JP2021152636A - Medical information management server, surgery training apparatus, surgery training system, image transmission method, display method, and program - Google Patents

Abstract

To provide a medical information management server, surgery training apparatus, surgery training system, image transmission method, and display method making it possible to gain experience equivalent to an actual surgery.SOLUTION: A medical information management server 70 includes a simulation data recording unit 75 in which at least plural surgical techniques are recorded in association with plural simulation data items including at least data items concerning living tissues, and a server control unit 79 that, when a client side selects one of the plural surgical techniques, transmits simulation data associated with the selected surgical technique to the client side.SELECTED DRAWING: Figure 3

Description

The present disclosure relates to medical information management servers, surgical training devices, surgical training systems, image transmission methods, display methods and programs that manage and distribute simulation data.

Conventionally, as a surgical training device, a technique has been known that reproduces a display during practice of how to see a teaching material image accompanying the movement of an endoscope (see, for example, Patent Document 1). In this technology, the movement history of the teaching material endoscope in the process of capturing the teaching material image by the teaching material endoscope is reproduced in the practice endoscope inserted in the practice device that provides the surgical practice space inside. At the same time, the teaching material image and the image captured by the practice endoscope are combined and displayed.

Japanese Unexamined Patent Publication No. 2016-148765

By the way, medical students and residents had to witness many actual surgeries and cases and gain various surgeries and experiences in order to become excellent surgeons. However, in the above-mentioned Patent Document 1, since the movement history of one teaching material endoscope is only reproduced by the practice endoscope, it is not possible to deal with various operations and cases, and the same case. Even so, medical students and residents were unable to trace the procedure performed by the surgeon or professor, or attempt a procedure using a different approach.

The present disclosure is made in view of the above, and provides a medical information management server, a surgical training device, a surgical training system, an image transmission method, a display method, and the like, which can gain experience equivalent to actual surgery. With the goal.

In order to solve the above-mentioned problems and achieve the object, the medical information management server according to the present disclosure includes a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data in association with each other, and a client side. When any one of the plurality of surgical procedures is selected, the control unit includes a control unit that transmits simulation data corresponding to the selected surgical procedure to the client side.

Further, the surgical training device according to the present disclosure includes a display unit capable of displaying a simulation movie based on the simulation data transmitted from a medical information management server that records a plurality of simulation data associated with a plurality of surgical procedures, and 1 The present invention includes one or more medical instruments, a detection unit for detecting the operation content of the operator on the medical instrument, and a control unit for displaying a medical instrument image related to the medical instrument on the simulation moving image on the display unit. The control unit controls the display mode of the medical device image and the simulation moving image based on the operation content detected by the detection unit.

Further, the surgical training system according to the present disclosure includes a medical information management server and a surgical training device capable of communicating with the medical information management server via a network, and the medical information management server includes at least a plurality of operations. The recording unit that records the procedure and the plurality of simulation data in association with each other, and when any one of the plurality of surgical procedures is selected on the client side, the simulation data corresponding to the selected surgical procedure is stored. The surgical training device includes a first control unit for transmitting to the client side, and the surgical training device imitates a display unit capable of displaying a simulation video based on the simulation data transmitted from the medical information management server and a subject. One or more medical devices that can be treated on the model, a detection unit that detects the operation content of the operator on the medical device, and a medical device image related to the medical device are displayed on the display unit on the simulation movie. A second control unit is provided, and the second control unit controls the display mode of the medical device image and the simulation moving image based on the operation content detected by the detection unit.

Further, the image transmission method according to the present disclosure is an image transmission method executed by a medical information management server including a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data in association with each other, and is on the client side. When any one of the plurality of surgical procedures is selected in, simulation data corresponding to the selected surgical procedure is transmitted to the client side.

Further, the display method according to the present disclosure includes a display unit capable of displaying a simulation movie based on the simulation data transmitted from a medical information management server that records a plurality of simulation data associated with a plurality of surgical procedures, and a subject. It is a display method executed by a surgical training device including one or more medical instruments that can be treated on a model imitating the above, and the operation contents by the operator on the medical instruments are detected and displayed on the simulation movie. The medical device image related to the medical device is displayed on the display unit, and the display mode of the medical device image and the simulation moving image is controlled based on the operation content.

Further, the program according to the present disclosure is a program executed by a medical information management server including a recording unit that records at least a plurality of surgical procedures and a plurality of simulation data in association with each other, and the plurality of programs are executed on the client side. When any one of the surgical procedures is selected, the simulation data corresponding to the selected surgical procedure is transmitted to the client side.

In addition, the program according to the present disclosure includes a display unit capable of displaying a simulation movie based on the simulation data transmitted from a medical information management server that records a plurality of simulation data associated with a plurality of surgical procedures, and a subject. It is a program to be executed by a surgical training device including one or more medical instruments capable of treating a imitated model, in which the operation contents of the operator on the medical instruments are detected, and the medical treatment is performed on the simulation movie. The medical device image related to the device is displayed on the display unit, and the display mode of the medical device image and the simulation moving image is controlled based on the operation content.

According to the present disclosure, it is possible to gain experience equivalent to actual surgery.

FIG. 1 is a diagram showing a schematic configuration of a surgical training system according to an embodiment. FIG. 2 is a block diagram showing a functional configuration of a main part of the surgical training device according to the embodiment. FIG. 3 is a block showing a functional configuration of the medical information management server according to the embodiment. FIG. 4 is a diagram showing an example of a setting screen displayed when performing surgical training displayed by the display device according to the embodiment. FIG. 5 is a diagram showing an example of a setting screen displayed when performing surgical training displayed by the display device according to the embodiment. FIG. 6 is a diagram showing an example of a setting screen displayed when performing surgical training displayed by the display device according to the embodiment. FIG. 7 is a diagram showing an example of a setting screen displayed when performing surgical training displayed by the display device according to the embodiment. FIG. 8 is a diagram showing an example of a setting screen displayed when performing surgical training displayed by the display device according to the embodiment. FIG. 9 is a diagram showing an example of a setting screen displayed when performing surgical training displayed by the display device according to the embodiment. FIG. 10 is a diagram showing an example of a setting screen displayed when performing surgical training displayed by the display device according to the embodiment. FIG. 11 is a diagram showing an example of a setting screen displayed when performing surgical training displayed by the display device according to the embodiment. FIG. 12 is a flowchart showing an outline of the processing executed by the medical information management server according to the embodiment. FIG. 13 is a flowchart showing an outline of the simulation data transmission process of FIG. FIG. 14 is a flowchart showing an outline of the simulation data generation process of FIG. FIG. 15 is a flowchart showing an outline of the process executed by the surgical training device according to the embodiment.

A form for carrying out the present disclosure will be described in detail together with the drawings. The present disclosure is not limited to the following embodiments. Further, in the following description, each of the referenced figures merely schematically shows the shape, size, and positional relationship to the extent that the contents of the present disclosure can be understood. That is, the present disclosure is not limited to the shape, size, and positional relationship exemplified in each figure. Further, in the description of the drawings, the same parts will be described with the same reference numerals. Furthermore, as an example of the surgical training system according to the present disclosure, a surgical training system using an endoscopic system using a laparoscope will be described, but the surgical training system using a surgical microscope system is also applicable. Can be done.

[Outline configuration of surgical training system]
FIG. 1 is a diagram showing a schematic configuration of a surgical training system according to an embodiment. In the surgical training system 1 shown in FIG. 1, a state in which a plurality of holes are formed in the abdomen of the _{model O 1 that imitates a subject such as a human or an animal by using a trocar, and the abdomen of the model O 1} is inflated by carbon dioxide or the like. in the operator U1 (trainee (trainee)) inserts the medical imaging apparatus (video scope such rigid endoscope), any one or more medical instruments energy devices such as forceps and the electric knife in the model O ₁ It is a system that trains laparoscopic surgery while observing and treating. In FIG. 1, are formed at two holes against model O _1, can be appropriately changed in accordance with the training content, for example, cases to train, operative procedures and surgical procedure.

The surgical training system 1 shown in FIG. 1 includes a trocar 10, a trocar 20, a medical device 30, a medical device 40, a display device 50, a control device 60, a medical information management server 70, and a plurality of hospital management servers. 80 and. In one embodiment, the trocar 10, the trocar 20, the medical device 30, the medical device 40, the display device 50, and the control device 60 function as the surgical training device 2.

Trocar 10 and 20, a cylindrical shape, is inserted into the model _{O 1.} The medical device 30 or the medical device 40, which will be described later, is inserted into the trocars 10 and 20.

The medical device 30 is inserted into the _{model O 1 via the trocar 10.} The medical device 30 simulates treatment for the observation target according to the operation of an operator such as a doctor or a trainee. Further, the medical device 30 detects the operation content of the operator, and outputs the detection result to the control device 60 via the transmission cable. The medical device 30 is, for example, an injection, forceps, a knife, a high frequency snare, a treatment tool, and an energy device such as a high frequency scalpel or an electric scalpel.

The medical device 40 is inserted into the _{model O 1 via the trocar 20.} The medical device 40 simulates treatment for the observation target according to the operation of an operator such as a doctor or a trainee. Further, the medical device 40 detects the operation content of the operator, and outputs the detection result to the control device 60 via the transmission cable. The medical device 40 is, for example, an injection, forceps, a knife, a high frequency snare, a treatment tool, and an energy device such as a high frequency scalpel or an electric scalpel.

Under the control of the control device 60, the display device 50 displays a two-dimensional image or a three-dimensional image based on the image data input from the control device 60. The display device 50 is realized by using, for example, a liquid crystal display, an organic electroluminescent display, an audio speaker, or the like.

The control device 60 controls each part of the surgical training device 2. The control device 60 performs various image processing on the simulation data input from the medical information management server 70 via the network 90 and outputs the simulation data to display the simulation moving image based on the simulation data on the display device 50. The control device 60 is realized by using a processor including a memory and hardware such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field-Programmable Gate Array). Here, the simulation data is data for performing a surgical simulation in a three-dimensional space using an organ, and includes data on an organ or a medical instrument. The simulation data for each organ and medical device includes, for example, the following parameters and is defined by polygon data in a three-dimensional space or the like.
(1) Position (organ position (coordinate information))
(2) Shape (shape of organ, shape of each medical device)
Furthermore, the simulation data for each organ and medical device may include the following parameters (3) to (7).
(3) Mass density (4) Rigidity (5) Friction (6) Viscosity (7) Tactile sensation Further, the simulation data can further include information on time-series changes related to the movement or deformation of the medical device or organ over time. ..

The medical information management server 70 transmits the simulation data requested by the control device 60 via the network 90. Further, the medical information management server 70 records two-dimensional image data or three-dimensional image data including data on at least organs acquired by a CT (Computed Tomography) device, an MRI (Magnetic Resonance Imaging) device, or the like for each of a plurality of patients. do. Further, the medical information management server 70 is a two-dimensional image data and a three-dimensional image including intraoperative moving image data captured during the operation of the subject by the medical imaging device and data on at least an organ captured by the CT device or the MRI device. Record simulation image data and the like based on at least one of the data. The medical information management server 70 is realized by using a processor including hardware such as a CPU, GPU, and FPGA, and an HDD (Hard Disk Drive), SSD (Solid State Drive), and the like.

The plurality of hospital management servers 80 transmit teaching material data for simulation data generated in the medical information management server 70 to the medical information management server 70 via the network 90. Here, the teaching material data includes two-dimensional image data or three-dimensional image data obtained by acquiring each of a plurality of subjects (patients) by a CT device, an MRI device, or the like, and a medical imaging device (for example, an endoscope or a rigid mirror). Intraoperative video data including at least organs (for example, stomach, intestine, etc.) imaged during each operation of a plurality of subjects (patients) by a video microscope such as, and subject information (patient) regarding each of the plurality of subjects. Information), multiple surgeons (operators and hospital names) who operated each of multiple subjects, procedure information on each surgical procedure of multiple subjects, and multiple medical instruments used in the surgical procedure. It is a video file associated with medical device information for each of the above. The plurality of hospital management servers 80 are realized by using a processor having hardware such as a CPU, GPU, and FPGA, and an HDD, SSD, and the like.

[Functional configuration of the main parts of the surgical training device]
Next, the functional configuration of the main part of the surgical training device 2 will be described. FIG. 2 is a block diagram showing a functional configuration of a main part of the surgical training device 2.

[Composition of medical equipment]
First, the configuration of the medical device 30 will be described. The medical device 30 includes a first detection unit 31, a second detection unit 32, a recording unit 33, and a communication unit 34. Since the medical device 40 has the same configuration as the medical device 30, detailed description thereof will be omitted.

The first detection unit 31 detects the insertion / removal of the medical device 30 held by the operator with respect to the trocar 10, for example, the push-in amount or the pull-out amount, and outputs the detection result to the communication unit 34. The first detection unit 31 is realized by using, for example, a gyro sensor, an acceleration sensor, or the like.

The second detection unit 32 detects the pressure value (pressing value) or speed associated with the opening / closing operation of the medical device 30 operated by the operator, and outputs the detection result to the communication unit 34. The second detection unit 32 is realized by using a pressure sensor, a gyro sensor, an acceleration sensor, or the like.

The recording unit 33 is realized by using a volatile memory or a non-volatile memory, and records medical device information regarding the medical device 30. Here, the medical device information includes the model name, shape, product number, type information of the medical device (for example, the type of forceps, electric knife, etc.), the date of manufacture, and the like of the medical device 30. The recording unit 33 outputs medical device information to the communication unit 34.

The communication unit 34 can communicate with the first communication unit 61, and controls the detection result of the first detection unit 31, the detection result of the second detection unit 32, and the medical device information recorded by the recording unit 33. Send to. The communication unit 34 is realized by using a communication module or the like.

[Control device configuration]
Next, the configuration of the control device 60 will be described. The control device 60 includes a first communication unit 61, an image processing unit 62, an input unit 63, a recording unit 64, a second communication unit 65, and a control unit 66.

The first communication unit 61 transmits various control data to the medical devices 30 and 40 under the control of the control unit 66, and various types of control data transmitted from the medical devices 30 and 40 and the medical information management server 70. The data is output to the image processing unit 62 or the control unit 66. The first communication unit 61 is realized by using, for example, an S / P conversion circuit, a communication module, or the like. Further, the first communication unit 61 may transmit various data to the medical devices 30 and 40 by wireless communication or optical communication.

Under the control of the control unit 66, the image processing unit 62 displays a simulation moving image based on the simulation data (polygon moving image data) input from the medical information management server 70 via the first communication unit 61. Specifically, the image processing unit 62 generates a simulation image (simulation moving image in which polygon data moving image data is graphicized) of an observation target in the subject, and outputs the simulation moving image to the display device 50. Display the simulation video. The image processing unit 62 is realized by using a memory and a processor having hardware such as a GPU (Graphics Processing Unit) and an FPGA.

The input unit 63 receives an input of an operation from the user and outputs a signal to the control unit 66 according to the received operation content. The input unit 63 is realized by using a mouse, a keyboard, a touch panel, switches, buttons, and the like.

The recording unit 64 records various data related to the surgical training device 2. The recording unit 64 has a program recording unit 641 that records various programs executed by the surgical training device 2. The recording unit 64 is realized by using a RAM (Random Access Memory), a ROM (Read Only Memory), an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like.

The second communication unit 65 acquires simulation data from the medical information management server 70 via the network 90 under the control of the control unit 66, and outputs the acquired simulation data to the control unit 66 and the image processing unit 62. do. The second communication unit 65 is realized by using a communication module or the like.

The control unit 66 controls each unit constituting the control device 60. The control unit 66 causes the display device 50 to display a medical device image related to the medical device on a simulation moving image based on simulation data acquired from the medical information management server 70 via the second communication unit 65. Specifically, the control unit 66 displays a medical device image and a simulation movie displayed by the display device 50 based on the operation contents of the medical device 30 detected by each of the first detection unit 31 and the second detection unit 32. Controls the display mode of. Further, the control unit 66 is an operation detected by each of the first detection unit 31 and the second detection unit 32 when a stop signal for stopping the simulation moving image displayed by the display device 50 is input from the input unit 63. Change the viewpoint of the simulation video based on the content. Specifically, the control unit 66 causes the display device 50 to display the simulation moving image of the viewpoint rotated by 90 degrees around the right side from the lower layer of the simulation moving image and the visual field area of the current viewpoint. The control unit 66 is realized by using a processor having hardware such as a memory and a CPU. In one embodiment, the control unit 66 functions as a second control unit.

[Medical information management server configuration]
Next, the configuration of the medical information management server 70 will be described. FIG. 3 is a block showing the functional configuration of the medical information management server 70. The medical information management server 70 shown in FIG. 3 includes a communication unit 71, a recording unit 72, a generation unit 73, an inference unit 74, a simulation data recording unit 75, an image data recording unit 76, and a user data recording unit 77. A medical device data recording unit 78 and a server control unit 79 are provided.

Under the control of the server control unit 79, the communication unit 71 transmits the simulation data requested by the control device 60 (client side) to the control device 60 via the network 90. The communication unit 71 is realized by using a communication module or the like.

The recording unit 72 records various data related to the medical information management server 70. The recording unit 72 has a program recording unit 721 that records various programs executed by the medical information management server 70. The recording unit 72 is realized by using a RAM, a ROM, or the like.

Under the control of the server control unit 79, the generation unit 73 receives intraoperative video data of the subject imaged during the operation transmitted from each of the plurality of hospital management servers 80 via the network 90, and the subject. Simulation data is generated based on at least one of the two-dimensional image data and the three-dimensional data previously captured by the CT device or the MRI device, and the simulation data is recorded in the simulation data recording unit 75. Further, the generation unit 73 aligns the organs included in each of the intraoperative moving image data and at least one of the two-dimensional image data and the three-dimensional data based on the inference result of the inference unit 74 described later, and performs simulation data. To generate. Further, the generation unit 73 reflects the orientation and crushing condition of the organs included in each of the intraoperative moving image data and at least one of the two-dimensional image data and the three-dimensional data based on the inference result of the inference unit 74 described later. Generate the simulated data. The generation unit 73 is realized by using a processor having hardware such as a memory and a GPU.

Under the control of the server control unit 79, the inference unit 74 includes intraoperative moving image data transmitted from each of the plurality of hospital management servers 80 via the network 90, and at least one of the two-dimensional image data and the three-dimensional data. , The organ and position of the subject appearing in the intraoperative moving image data are estimated, and the estimation result is output to the generation unit 73. As learning parameters, the inference unit 74 includes intraoperative video data, organs and positions reflected in the intraoperative video data, distance information indicating the distance from the tip of the medical imaging device (videoscope) to the organ, and feature quantities of the intraoperative video data. A plurality of simulation data (for example, polygon data based on at least one of CT image data and MRI image data) and the like are used as learning parameters (teacher data), and these learning parameters are machine-learned by deep learning or the like and used as output parameters. It is realized by using a trained model that outputs the estimation result of estimating the organ and position of the subject to be observed and dealt with in the intraoperative moving image data. Here, as machine learning, DNN (Deep Neural Network), CNN (Convolutional Neural Network), RNN (Recurrent Neural Network) and the like are used. Further, when the intraoperative moving image data transmitted from each of the plurality of hospital management servers 80 via the network 90 is the multispectral image data irradiated with the multispectral light by the light source device, the inference unit 74 is a multispectral image. Using the difference in the reflected light frequency (for each spectrum) included in the data, the biological tissue type (for example, organs such as bile sac and pancreas, blood vessels, nerves, lymph, urine) of each part of the subject to be observed in the multispectral image data (Tube, fat, etc.) and their positions (regions) are estimated, and the estimation result is output to the generation unit 73. Further, the inference unit 74 indicates the position and orientation of the medical imaging device associated with the intraoperative moving image data transmitted from each of the plurality of hospital management servers 80 via the network 90, the insertion position of the subject, and the subject. Orientation of organs, blood vessels, nerves, etc. in the intraoperative moving image data when used as a reference with respect to the optical axis (insertion direction of the trocar) of the optical system of the medical imaging device based on the amount of operation (distance) inserted into (Direction) and the degree of crushing are estimated, and the estimation result is output to the generation unit 73. The inference unit 74 is realized by using a processor having a memory and hardware such as a GPU.

The simulation data recording unit 75 records a plurality of surgical procedures and a plurality of simulation data in association with each other. Specifically, as shown in Table T1 of FIG. 4, the simulation data recording unit 75 uses the surgical technique, the simulation data, the operator information for identifying the operator who performed the simulation data, and the simulation data. The medical device information regarding the used medical device and the level information indicating the level of this simulation data are recorded in association with each other. The simulation data recording unit 75 is realized by using an HDD, an SSD, or the like.

The image data recording unit 76 includes two-dimensional image data captured by a CT device transmitted from each of a plurality of hospital management servers 80 via the network 90, three-dimensional image data captured by an MRI examination, and a hospital name. The hospital identification information indicating the above and the subject information (patient information) relating to the subject are recorded in association with each other. Here, the subject information includes the name of the subject, the name of the disease, the medical examination history, the biological information, the doctor in charge, the surgeon, and the like. The image data recording unit 76 is realized by using an HDD, an SSD, or the like.

The user data recording unit 77 records identification information about each of the plurality of operators U1 who have been trained by using the surgical training device 2 on the medical information management server 70 via the network 90. Specifically, as shown in Table T2 of FIG. 5, the user data recording unit 77 sets the user ID, the level (difficulty level) of the operator U1 by surgical training, as the identification information for identifying the operator U1. Record in association with the training content. Further, the user data recording unit 77 may record the number of trainings, the hospital or academic society to which the user ID belongs, the address, and the bank account in association with the user ID. The user data recording unit 77 is realized by using an HDD, an SSD, or the like.

The medical device data recording unit 78 records medical device information related to the medical device. Here, the medical device information is a model name, model number, animation data, CAD data (polygon data), etc. of the medical device.

The server control unit 79 controls each unit constituting the medical information management server 70. The server control unit 79 is realized by using a processor having hardware such as a memory and a CPU. When any one of the plurality of surgical procedures is selected by the control device 60 (client side), the server control unit 79 transmits simulation data corresponding to the selected surgical procedure to the control device 60 (client side). The server control unit 79 includes a determination unit 791, a billing unit 792, and a point giving unit 793. In one embodiment, the server control unit 79 functions as the first control unit.

The determination unit 791 determines the identification information of the operator U1 who has accessed the medical information management server 70 via the control device 60 and the network 90, and the identification information recorded by the user data recording unit 77, and this determination result. Is output to the charging unit 792 and the point giving unit 793.

Based on the determination result of the determination unit 791, the billing unit 792 downloads simulation data to the control device 60 for the registered address or affiliated hospital registered by the user corresponding to the user ID recorded by the user data recording unit 77. Charge by charging based on the level or number of times.

Based on the determination result of the determination unit 791, the score giving unit 793 scores the user ID recorded by the user data recording unit 77 according to the level associated with the simulation video or the number of times training is performed with the simulation data. Is given.

[Outline of training method]
Next, the outline of the surgical training method by the surgical training system 1 will be described. 6 to 11 are diagrams showing an example of a setting screen displayed by the display device 50 when performing surgical training.

First, the operator U1 accesses the browser of the medical information management server 70 using the input unit 63 of the control device 60, and inputs identification information (user ID, password, etc.) for identifying the operator U1. In this case, the medical information management server 70 transmits the training screen data to the control device 60 (client side) that has accessed the browser. As a result, as shown in FIG. 6, the display device 50 displays the training screen W1 corresponding to the training screen data under the control of the control device 60. The training screen W1 includes at least the level of the surgical procedure, the surgical target of the surgical procedure (eg rectum), the mission (eg tumor removal and suturing), and the thumbnail image SM1 corresponding to the simulation data. As shown in FIG. 6, the operator U1 operates a pointer or the like using the input unit 63 to set a desired level and a desired surgical target in each predetermined cell. In this case, the control device 60 transmits the setting contents set by the operator U1 to the medical information management server 70.

Further, as shown in FIG. 7, the medical information management server 70 receives a request signal requesting the realism setting of the surgical procedure and the setting of various modes during training from the control device 60, and the realism setting and the panic mode. The setting screen data is transmitted to the control device 60. As a result, as shown in FIG. 7, the display device 50 displays the realism setting and panic mode setting screen W2 under the control of the control device 60. On the realism setting and panic mode setting screen W2, at least, bleeding speed, blood pressure lowering speed, O2 lowering, malformation band / blood vessel, hardness / brittleness, vitals, presence / absence of panic during surgery, and so on. Is included.

Further, as shown in FIG. 8, when the request signal requesting the processing tool selection and setting for selecting the medical device is input from the control device 60, the medical information management server 70 displays the processing tool selection and setting setting screen data. It is transmitted to the control device 60. As a result, as shown in FIG. 8, the display device 50 displays the processing tool selection and setting setting screen W3 under the control of the control device 60. The processing tool selection and setting setting screen W3 includes a plurality of medical instruments M1 to M5 (plurality of treatment tools) used for the surgical procedure selected by the operator U1. For example, the processing tool selection and setting setting screen W3 includes forceps, an electric knife, a medical imaging device (rigid mirror), and the like. The operator U1 selects a medical device to be used for a surgical procedure performed in training. Further, the operator U1 can increase the number of medical devices (selected items) that can be selected by charging the medical information management server 70 or the like. In this case, the medical information management server 70 charges a fee according to the number of selected medical devices based on the identification information of the operator U1.

Further, as shown in FIG. 9, when a request signal requesting a surgical procedure or a case base is input from the control device 60, the medical information management server 70 corresponds to each of a plurality of surgical procedures or a plurality of cases. The case selection screen data displaying a list of thumbnail images of the simulation data of the above is transmitted to the control device 60. As a result, as shown in FIG. 9, the display device 50 displays the case selection screen W4 under the control of the control device 60. The case selection screen W4 includes thumbnail images P1 to P5 corresponding to each of the plurality of simulation data. The operator U1 selects a desired thumbnail image. In this case, the medical information management server 70 transmits the operator data regarding the operator (surgeon) of the simulation data of the thumbnail image corresponding to the selection signal input from the control device 60 to the control device 60. As a result, as shown in FIG. 10, the display device 50 displays the operator screen W5 regarding the operator who is an associate professor or the like under the control of the control device 60. The surgeon screen W5 includes a portrait of the surgeon (surgeon), a case operated by the surgeon, a surgical technique, a level and a message by the surgical training system 1.

Further, as shown in FIG. 11, when the request signal requesting the simulation data of the new approach is input from the control device 60, the medical information management server 70 outputs the simulation data screen data of the new approach to the control device 60. Send to. As a result, as shown in FIG. 10, the display device 50 displays the simulation data screen W6 of the novel approach under the control of the control device 60. The simulation data screen W6 includes thumbnail images SM2 and the like corresponding to the simulation moving image.

In this way, the operator U1 performs training by causing the control device 60 to download simulation data of a desired surgical case according to the level from the medical information management server 70 using the input unit 63 of the control device 60. Can be done.

[Medical information management server processing]
Next, the process executed by the medical information management server 70 will be described. FIG. 12 is a flowchart showing an outline of the processing executed by the medical information management server 70.

First, the server control unit 79 determines whether or not there is a request from the client side (step S1). Specifically, the server control unit 79 determines whether or not a request signal requesting various simulation data has been input from the control device 60 (client side) via the network 90. When the server control unit 79 determines that there is a request from the client side (step S1: Yes), the medical information management server 70 proceeds to step S2, which will be described later. On the other hand, when the server control unit 79 determines that there is no request from the client side (step S1: No), the medical information management server 70 proceeds to step S4, which will be described later.

In step S2, the medical information management server 70 executes a simulation data transmission process for transmitting simulation data to the client side. The details of the simulation video transmission process will be described later.

Subsequently, the server control unit 79 determines whether or not the access from the client side has been completed (step S3). When the server control unit 79 determines that the access from the client side has been completed (step S3: Yes), the medical information management server 70 ends this process. On the other hand, when the server control unit 79 determines that the access from the client side has not been completed (step S3: No), the medical information management server 70 returns to the above-mentioned step S1.

In step S4, the server control unit 79 determines whether or not the intraoperative moving image data, the two-dimensional image data, and the three-dimensional image data are received from any one of the plurality of hospital management servers 80 via the network 90. .. When it is determined by the server control unit 79 that the intraoperative moving image data, the two-dimensional image data, and the three-dimensional image data are received from any one of the plurality of hospital management servers 80 (step S4: Yes), the medical information management server 70 Goes to step S5, which will be described later. On the other hand, when it is determined by the server control unit 79 that the intraoperative moving image data, the two-dimensional image data, and the three-dimensional image data are not received from any one of the plurality of hospital management servers 80 (step S4: No). ), The medical information management server 70 proceeds to step S3.

In step S5, the medical information management server executes a simulation data generation process for generating simulation data based on either the intraoperative moving image data, the two-dimensional image data, or the three-dimensional image data. The details of the simulation data generation process will be described later. After step S5, the medical information management server 70 shifts to step S3.

[Overview of simulation data transmission processing]
Next, the outline of the simulation data transmission process described in step S2 of FIG. 12 described above will be described. FIG. 13 is a flowchart showing an outline of the simulation data transmission process.

As shown in FIG. 13, first, the server control unit 79 acquires the identification information for identifying the operator U1 transmitted from the control device 60 (client side) accessed via the network 90 and the communication unit 71. (Step S101). In this case, the determination unit 791 determines the identification information recorded by the user data recording unit 77 and the identification information of the operator U1 transmitted from the control device 60.

Subsequently, the determination unit 791 determines whether or not a setting signal for setting a mission has been input from the control device 60 in the surgical training (step S102). When it is determined by the determination unit 791 that the setting signal for setting the mission is input from the control device 60 in the surgical training (step S102: Yes), the medical information management server 70 proceeds to step S103, which will be described later. On the other hand, when the determination unit 791 determines that the setting signal for setting the mission in the surgical training has not been input from the control device 60 (step S102: No), the medical information management server 70 determines in step S105, which will be described later. Move to.

In step S103, the server control unit 79 causes the communication unit 71 to transmit simulation data corresponding to the mission set in the surgical training from the control device 60 (step S103). Specifically, the server control unit 79 acquires simulation data corresponding to the mission set in the surgical training from the control device 60 from the plurality of simulation data recorded by the simulation data recording unit 75, and the acquired simulation data. Is transmitted to the communication unit 71. As a result, the operator U1 can perform training according to the set mission while watching the simulation video displayed by the display device 50.

Subsequently, the server control unit 79 assigns points to the charge and identification information to the operator U1 based on the identification information, the level of the simulation data downloaded to the control device 60, and the like (step S104). Specifically, the billing unit 792 was downloaded to the control device 60 to the registered address or affiliated hospital registered by the user matching the user ID recorded by the user data recording unit 77 determined by the determination unit 791. Billing is done by billing based on the level or number of simulation data. Further, the score giving unit 793 assigns a score to the user ID recorded by the user data recording unit 77 determined by the determination unit 791 according to the level associated with the simulation video or the number of times training is performed with the simulation data. Give. After step S104, the medical information management server 70 returns to the main routine of FIG. The billing method by the billing unit 792 is an example, and in addition to the simulation data level, download of medical device information of a new medical device, download of simulation data of a new surgical procedure, simulation data by a predetermined surgeon. You may charge for the download of.

In step S105, the determination unit 791 determines whether or not a request signal for selecting a desired surgical procedure case from a plurality of surgical procedure cases has been input from the control device 60. When it is determined by the determination unit 791 that a request signal has been input from the control device 60 to select a desired surgical procedure case from a plurality of surgical procedure cases (step S105: Yes), the medical information management server 70 will be described later. Step S106 is performed. On the other hand, when it is determined by the determination unit 791 that the request signal for selecting the desired surgical procedure case from the plurality of surgical procedure cases has not been input from the control device 60 (step S105: No), the medical information. The management server 70 proceeds to step S109, which will be described later.

In step S106, the server control unit 79 causes the communication unit 71 to transmit surgical procedure case information summarizing a plurality of surgical procedure cases corresponding to each of the plurality of simulation data recorded by the simulation data recording unit 75.

Subsequently, the server control unit 79 acquires medical device information used by the operator U1 in the surgical training from the control device 60 from the control device 60 via the network 90 and the communication unit 71 (step S107).

After that, the server control unit 79 causes the communication unit 71 to transmit simulation data based on the surgical procedure case and medical device information selected by the operator U1 from the plurality of simulation data recorded by the simulation data recording unit 75. (Step S108). Thereby, the operator U1 can perform the surgical training of the desired surgical procedure case using the medical instruments 30 and 40 while watching the simulation moving image displayed on the display device 50. After step S108, the medical information management server 70 shifts to step S104.

In step S109, the determination unit 791 determines whether or not a request signal for selecting a desired surgeon from a plurality of surgeons has been input from the control device 60. When it is determined by the determination unit 791 that a request signal for selecting a desired surgeon from a plurality of surgeons has been input from the control device 60 (step S109: Yes), the medical information management server 70 is set to step S110, which will be described later. Move to. On the other hand, when the determination unit 791 determines that the request signal for selecting a desired surgeon from a plurality of surgeons has not been input from the control device 60 (step S109: No), the medical information management server. 70 proceeds to step S104.

In step S110, the server control unit 79 causes the communication unit 71 to transmit surgeon information summarizing a plurality of surgeons corresponding to each of the plurality of simulation data recorded by the simulation data recording unit 75.

Subsequently, the server control unit 79 acquires the surgeon information selected by the operator U1 from the control device 60 via the network 90 and the communication unit 71 (step S111), and the simulation data recording unit 75 records the plurality of information. The simulation data based on the surgeon information selected by the operator U1 from the simulation data of the above is transmitted to the communication unit 71 (step S112). As a result, the operator U1 can trace and learn the surgical procedure by the surgeon while watching the simulation video displayed on the display device 50. After step S112, the medical information management server 70 shifts to step S104.

[Overview of simulation data generation processing]
Next, the outline of the simulation data generation process described in step S5 of FIG. 12 described above will be described. FIG. 14 is a flowchart showing an outline of the simulation data generation process.

As shown in FIG. 14, first, the inference unit 74 first receives intraoperative moving image data transmitted from each of the plurality of hospital management servers 80 via the network 90, and two-dimensional image data and three-dimensional data including at least data on organs. Based on at least one of the data, the organ and position of the subject appearing in the intraoperative moving image data is estimated (step S201). Specifically, the inference unit 74 determines that the intraoperative moving image data transmitted from each of the plurality of hospital management servers 80 via the network 90 is multispectral image data irradiated with multispectral light by the light source device. Observation of the subject reflected in the multispectral image data using the difference in the reflected light frequency (for each spectrum) included in the multispectral image data Type of biological tissue of each part (for example, organs such as bile sac and pancreas, blood vessels, nerves) , Lymph, urinary tract, fat, etc.) and their positions (regions) are estimated, and the estimation result is output to the generation unit 73. In addition, the inference unit 74 includes the position and orientation of the medical imaging device associated with the intraoperative moving image data transmitted from each of the plurality of hospital management servers 80 via the network 90, the insertion position of the subject, and the subject. Orientation of organs, blood vessels, nerves, etc. in the intraoperative moving image data when used as a reference with respect to the optical axis (insertion direction of the trocar) of the optical system of the medical imaging device based on the amount of operation (distance) inserted into (Direction), the degree of crushing, and the like may be estimated, and the estimation result may be output to the generation unit 73.

Subsequently, the generation unit 73 generates simulation data by aligning the organs included in each of the intraoperative moving image data, the two-dimensional image data, and the three-dimensional data based on the inference result of the inference unit 74. (Step S202). Specifically, the generation unit 73 determines the orientation of the organ, the position of the organ, and the orientation of the organ included in each of the intraoperative moving image data and at least one of the two-dimensional image data and the three-dimensional data, based on the inference result of the inference unit 74. After identifying the organs, while aligning the organs, simulation data that reflects the degree of collapse of the organs assumed from the current viewpoint is generated.

After that, the server control unit 79 acquires the operator information from the metadata (header information) included in the intraoperative video file that stores the intraoperative video data (step S203), and converts the intraoperative video file into the intraoperative video file that stores the intraoperative video data. Medical device information is acquired from the included metadata (header information) (step S204).

Subsequently, the server control unit 79 generates a simulation file in which the simulation data generated by the generation unit 73, the operator information, and the medical device information are associated with each other, and records the simulation file in the simulation data recording unit 75 (step S205). ). After step S205, the medical information management server 70 returns to the main routine of FIG.

[Processing of surgical training equipment]
Next, the process executed by the surgical training device 2 will be described. FIG. 15 is a flowchart showing an outline of the process executed by the surgical training device 2.

First, the control unit 66 causes the second communication unit 65 to transmit the identification information of the operator U1 input via the input unit 63 (step S301).

Subsequently, when the operator U1 selects the mission mode via the input unit 63 (step S302: Yes), the control unit 66 causes the display device 50 to display the mission information that can be set in the surgical training (step S303). ).

After that, the control unit 66 causes the second communication unit 65 to transmit the mission information set by the operator U1 using the input unit 63 (step S304).

Subsequently, the control unit 66 causes the display device 50 to display a simulation video based on the simulation data corresponding to the mission set in the operator U1 transmitted from the medical information management server 70 (step S305).

After that, the control unit 66 determines whether or not a stop signal for stopping the reproduction of the simulation moving image has been input from the input unit 63 (step S306). When it is determined by the control unit 66 that the stop signal for stopping the reproduction of the simulation moving image has been input from the input unit 63 (step S306: Yes), the surgical training device 2 proceeds to step S307, which will be described later. On the other hand, when it is determined by the control unit 66 that the stop signal for stopping the reproduction of the simulation moving image has not been input from the input unit 63 (step S306: No), the surgical training device 2 goes to step S309 described later. Transition.

In step S307, the control unit 66 controls the viewpoint of the simulation moving image based on the operation contents of the medical devices 30 and 40. Specifically, the control unit 66 controls the image processing unit 62 based on the operation contents of the medical instruments 30 and 40, so that the display layer on the lower side of the image of the current display layer is used as the viewpoint of the simulation movie. The viewpoint of the simulation moving image displayed by the display device 50 is controlled by generating an image of a display layer that can be observed with the viewpoint changed 90 degrees clockwise from the current viewpoint. Of course, the control unit 66 may perform enlargement or reduction, for example, in addition to changing the viewpoint of the simulation moving image by controlling the image processing unit 62 based on the operation contents of the medical instruments 30 and 40.

Subsequently, when the reproduction of the simulation moving image is completed (step S308: Yes), the surgical training device 2 ends this process. On the other hand, when the reproduction of the simulation moving image is not completed (step S308: No), the surgical training device 2 shifts to the above-mentioned step S306.

In step S309, the control unit 66 controls the display mode of the simulation moving image based on the simulation data based on the operation contents of the medical devices 30 and 40. Specifically, the control unit 66 moves the medical devices 30 and 40 shown in the simulation movie according to the operation contents of the operator U1 based on the operation contents of the medical devices 30 and 40, and the medical devices 30 and 40 are used. Reflect the procedure in the simulation video. After step S309, the surgical training device 2 proceeds to step S308.

Next, in step S302, a case where the operator U1 does not select the mission mode via the input unit 63 (step S302: No) will be described. At this time, the surgical training device 2 shifts to step S310.

Subsequently, when the operator U1 selects the surgical procedure case mode via the input unit 63 (step S310: Yes), the surgical training device 2 shifts to step S311 described later. On the other hand, when the operator U1 does not select the surgical procedure case mode via the input unit 63 (step S310: No), the surgical training device 2 shifts to step S315, which will be described later.

In step S311 the control unit 66 causes the display device 50 to display the surgical procedure case information that can be selected in the surgical training.

Subsequently, the control unit 66 causes the second communication unit 65 to transmit the surgical procedure case information selected by the operator U1 using the input unit 63 (step S312).

After that, the control unit 66 causes the second communication unit 65 to transmit the medical device information regarding the medical devices 30 and 40 (step S313).

Subsequently, the control unit 66 causes the display device 50 to display a simulation moving image based on the simulation data corresponding to the surgical procedure case selected by the operator U1 transmitted from the medical information management server 70 (step S314). After step S314, the surgical training device 2 shifts to step S306.

In step S315, when the operator U1 selects the operator mode via the input unit 63 (step S315: Yes), the surgical training device 2 shifts to step S316 described later.

In step S316, the control unit 66 causes the display device 50 to display operator information on which a plurality of operators can be selected in the operation mode.

Subsequently, the operator information selected by the operator U1 is transmitted to the second communication unit 65 via the input unit 63 (step S317).

After that, the control unit 66 causes the display device 50 to display a simulation moving image based on the simulation data of the operator information selected by the operator U1 transmitted from the medical information management server 70 (step S318). After step S318, the surgical training device 2 shifts to step S306.

According to the embodiment described above, when the server control unit 79 selects any one of the plurality of surgical procedures on the control device 60 (client side), the server control unit 79 controls the simulation data corresponding to the selected surgical procedure. Since the data is transmitted to the device 60, the operator U1 can gain experience equivalent to actual surgery, so that the learning curve of the operator U1 can be improved.

Further, according to one embodiment, the surgical technique can be trained in the free time regardless of the location.

Further, according to one embodiment, the generation unit 73 receives the intraoperative moving image data imaged during the operation of the subject by the medical imaging device transmitted from each of the hospital management servers 80, and the CT device for the subject in advance. Alternatively, simulation data is generated based on at least one of two-dimensional image data and three-dimensional data including data on at least an organ imaged by an MRI apparatus, so that simulation data of a plurality of surgical procedures and a plurality of cases can be easily performed. Can be collected in.

Further, according to one embodiment, the generation unit 73 aligns the organs included in each of the intraoperative moving image data and at least one of the two-dimensional image data and the three-dimensional data based on the estimation result of the inference unit 74. To generate simulation data, it is possible to generate realistic simulation data.

Further, according to one embodiment, the simulation data recording unit 75 records a plurality of simulation data in association with a plurality of operator information regarding a plurality of operators who have operated each of the plurality of simulation data. Therefore, the operator U1 can select and train the simulation data performed by the desired operator.

Further, according to one embodiment, the simulation data recording unit 75 further records a plurality of simulation data in association with a plurality of medical device information relating to the plurality of medical devices used in each of the plurality of simulation data. Therefore, the operator U1 can select simulation data using a desired medical device and perform training.

Further, according to one embodiment, when the server control unit 79 selects any one or more of the plurality of medical device information described in the control device 60 (client side), the selected medical device information is transmitted to the control device 60. Since it is transmitted to the side, the operator U1 can perform training with simulation data using a desired medical device.

Further, according to one embodiment, when the billing unit 792 of the server control unit 79 transmits the simulation data to the control device 60 (client side), the operation is performed based on the identification information of the operator U1 who operates the client side. Since the person U1 is charged, a profit can be obtained.

Further, according to one embodiment, a surgical procedure related to a surgical procedure that can be selected by the control device 60 based on the level associated with the identification information of the operator U1 transmitted from the control device 60 by the server control unit 79. Since the information is transmitted to the control device 60, it is possible to select the simulation data suitable for the level of the operator U1.

Further, according to one embodiment, the server control unit 79 gives points to the number of times the operator U1 virtually trains using the simulation data based on the identification information of the operator U1. , The leveling of the procedure by the operator U1 can be easily grasped.

Further, according to one embodiment, since the simulation data recording unit 75 further associates and records each level of the plurality of simulation data, the operator U1 selects and trains the simulation data according to his / her own level. It can be performed.

Further, according to one embodiment, the medical treatment displayed by the display device 50 based on the operation content of the operator U1 detected by the control unit 66 by each of the first detection unit 31 and the second detection unit 32. Intuitive operation can be performed to control the display mode of the instrument image and the simulation moving image.

Further, according to one embodiment, when the control unit 66 inputs a stop signal for stopping the simulation moving image displayed by the display device 50 from the input unit 63, the first detection unit 31 and the second detection unit 31 and the second detection unit 66. When the operation content detected by each of the units 32 is detected, the viewpoint of the simulation movie is changed based on the operation content, so that the operator U1 can observe the simulation movie from the desired viewpoint.

Further, according to one embodiment, the control unit 66 causes the display device 50 to display a plurality of surgical procedure information relating to each of the plurality of surgical procedures transmitted from the medical information management server 70, and also via the input unit 63. When any one or more of the plurality of surgical procedure information is selected, the operator U1 is desired to request the medical information management server 70 to transmit the simulation data corresponding to the selected surgical procedure information. Simulation data corresponding to the case to be treated can be acquired from the medical information management server 70.

Further, according to one embodiment, since the control unit 66 transmits the identification information of the operator U1 to the medical information management server 70, it is possible to acquire the simulation data corresponding to the operator U1.

Further, according to one embodiment, since the control unit 66 transmits medical device information regarding each of the medical devices 30 and 40 used by the operator U1 to the medical information management server 70, the operator U1 uses it for training. It is possible to acquire simulation data suitable for the medical devices 30 and 40 to be used.

In one embodiment, one operator U1 is training, but the training is not limited to this, and a plurality of operators U1 are trained while watching the same simulation movie displayed by the display device 50. Or, a plurality of operators U1 may perform training using surgical training devices installed at different locations from each other.

(Other embodiments)
Various inventions can be formed by appropriately combining a plurality of components disclosed in the surgical training system according to the embodiment of the present disclosure described above. For example, some components may be removed from all the components described in the surgical training system according to the embodiment of the present disclosure described above. Further, the components described in the medical observation system according to the embodiment of the present disclosure described above may be appropriately combined.

Further, in the surgical training system according to the embodiment of the present disclosure, the above-mentioned "part" can be read as "means", "circuit" and the like. For example, the control unit can be read as a control means or a control circuit.

Further, the program to be executed by the surgical training system according to the embodiment of the present disclosure is a file data in an installable format or an executable format, such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD (Digital). It is recorded and provided on a computer-readable recording medium such as a Versatile Disk), a USB medium, or a flash memory.

Further, the program to be executed by the surgical training system according to the embodiment of the present disclosure may be stored on a computer connected to a network such as the Internet and provided by downloading via the network. ..

In the description of the flowchart in the present specification, the context of the processing between steps has been clarified by using expressions such as "first", "after", and "continued", but in order to carry out the present invention. The order of processing required is not uniquely defined by those representations. That is, the order of processing in the flowchart described in the present specification can be changed within a consistent range.

Although some of the embodiments of the present application have been described in detail with reference to the drawings, these are examples, and various modifications based on the knowledge of those skilled in the art, including the embodiments described in the columns of the present disclosure. It is possible to implement the present disclosure in other modified forms.

The present technology can also have the following configurations.
(Appendix 1)
A recording unit that records at least a plurality of surgical procedures in association with and at least a plurality of simulation data including data on a living tissue.
When any one of the plurality of surgical procedures is selected on the client side, a control unit that transmits simulation data corresponding to the selected surgical procedure to the client side.
To prepare
Medical information management server.
(Appendix 2)
The medical information management server described in (Appendix 1).
The simulation data is generated based on the intraoperative moving image data captured during the operation of the subject and at least one of the two-dimensional image data and the three-dimensional data previously captured by the CT device or the MRI device on the subject. Further equipped with a generator to generate,
Medical information management server.
(Appendix 3)
The medical information management server described in (Appendix 2).
An inference unit for estimating the organ and position of the subject reflected in the intraoperative moving image data based on the intraoperative moving image data and at least one of the two-dimensional image data and the three-dimensional data is further provided.
The generator
Based on the estimation result of the inference unit, the simulation data is generated by aligning the organs contained in at least one of the intraoperative moving image data, the two-dimensional image data, and the three-dimensional data.
Medical information management server.
(Appendix 4)
The medical information management server described in (Appendix 3).
The intraoperative video data is
The subject is irradiated with multispectral light by the multispectral light source, and by receiving the return light from the subject, the spectrum image data including at least the reflection frequency for each biological tissue is included.
The inference unit
At least the type and position of the biological tissue are estimated using the reflection frequency of each biological tissue included in the spectral image data.
Medical information management server.
(Appendix 5)
The medical information management server according to any one of (Appendix 1) to (Appendix 4).
The recording unit
A plurality of operator information regarding a plurality of operators who have performed each of the plurality of simulation data are further associated with the plurality of simulation data and recorded.
The control unit
When any one of the plurality of operator information is selected on the client side, the plurality of simulation data corresponding to the selected operator information is transmitted to the client side.
Medical information management server.
(Appendix 6)
The medical information management server according to any one of (Appendix 1) to (Appendix 5).
The recording unit
A plurality of medical device information relating to the plurality of medical devices used in each of the plurality of simulation data is further associated and recorded with the plurality of simulation data.
The control unit
When the simulation data is transmitted to the client side, the medical device information associated with the simulation data is also transmitted to the client side.
Medical information management server.
(Appendix 7)
The medical information management server described in (Appendix 6).
The control unit
When any one or more of the plurality of medical device information is selected on the client side, the selected medical device information is transmitted to the client side.
Medical information management server.
(Appendix 8)
The medical information management server according to any one of (Appendix 1) to (Appendix 7).
The control unit
When the simulation data is transmitted to the client side, the operator is charged based on the identification information of the operator who operates the client side.
Medical information management server.
(Appendix 9)
The medical information management server described in (Appendix 8).
The control unit
Based on the level associated with the identification information, the surgical procedure information regarding the surgical procedure selectable on the client side is transmitted to the client side.
Medical information management server.
(Appendix 10)
The medical information management server described in (Appendix 9).
The control unit
Based on the identification information, a score is given to the number of times that the operator virtually trained using the simulation data.
Medical information management server.
(Appendix 11)
The medical information management server according to any one of (Appendix 8) to (Appendix 10).
The recording unit
Each level of the plurality of simulation data is further associated and recorded.
Medical information management server.
(Appendix 12)
A display unit capable of displaying a simulation video based on the simulation data transmitted from a medical information management server that records a plurality of surgical procedures and a plurality of simulation data including at least data related to a living tissue in association with each other.
With one or more medical devices
A detection unit that detects the operation content of the operator on the medical device, and
A control unit that displays a medical device image related to the medical device on the simulation movie on the display unit,
With
The control unit controls the display mode of the medical device image and the simulation moving image based on the operation content detected by the detection unit.
Surgical training equipment.
(Appendix 13)
The surgical training device according to (Appendix 12).
The control unit
When a stop signal for stopping the simulation video displayed by the display unit is input and the detection unit detects the operation content, the viewpoint of the simulation video is changed based on the operation content.
Surgical training equipment.
(Appendix 14)
The surgical training device according to (Appendix 13).
The control unit
When a plurality of surgical procedure information relating to each of the plurality of surgical procedures is displayed on the display unit and one or more of the plurality of surgical procedure information is selected, the selected surgical procedure information is supported. Request the medical information management server to transmit the simulation data.
Surgical training equipment.
(Appendix 15)
The surgical training device according to (Appendix 14).
Further provided with an input unit that accepts the input of the operator's identification information.
The control unit
Sending the identification information to the medical information management server,
Surgical training equipment.
(Appendix 16)
The surgical training device according to any one of (Appendix 12) to (Appendix 15).
The control unit
Sending medical device information related to the medical device to the medical information management server,
Surgical training equipment.
(Appendix 17)
Medical information management server and
A surgical training device that can communicate with the medical information management server via a network,
With
The medical information management server
A recording unit that records at least a plurality of surgical procedures in association with and at least a plurality of simulation data including data on a living tissue.
When any one of the plurality of surgical procedures is selected on the client side, a first control unit that transmits simulation data corresponding to the selected surgical procedure to the client side, and
With
The surgical training device
A display unit capable of displaying a simulation movie based on the simulation data transmitted from the medical information management server, and a display unit.
One or more medical devices that can be treated on a model that mimics the subject,
A detection unit that detects the operation content of the operator on the medical device, and
A second control unit that displays a medical device image related to the medical device on the simulation movie on the display unit,
With
The second control unit controls the display mode of the medical device image and the simulation moving image based on the operation content detected by the detection unit.
Surgical training system.
(Appendix 18)
An image transmission method executed by a medical information management server including a recording unit that records at least a plurality of surgical procedures and at least a plurality of simulation data including data on a living tissue in association with each other.
When any one of the plurality of surgical procedures is selected on the client side, simulation data corresponding to the selected surgical procedure is transmitted to the client side.
Image transmission method.
(Appendix 19)
A display unit capable of displaying a simulation video based on the simulation data transmitted from the medical information management server, which records at least a plurality of surgical procedures and a plurality of simulation data including at least data on a living tissue in association with each other, and a subject. A display method performed by a surgical training device comprising one or more medical instruments capable of treating a model that mimics a specimen.
Detecting the operation contents of the medical device by the operator,
A medical device image related to the medical device is displayed on the simulation movie on the display unit, and the display mode of the medical device image and the simulation movie is controlled based on the operation content.
Display method.
(Appendix 20)
A program executed by a medical information management server having a recording unit that records at least a plurality of surgical procedures and at least a plurality of simulation data including data on living tissue in association with each other.
When any one of the plurality of surgical procedures is selected on the client side, simulation data corresponding to the selected surgical procedure is transmitted to the client side.
program.
(Appendix 21)
A display unit capable of displaying a simulation video based on the simulation data transmitted from the medical information management server, which records at least a plurality of surgical procedures and a plurality of simulation data including at least data on a living tissue in association with each other, and a subject. A program that is run by a surgical training device equipped with one or more medical instruments that can treat a model that mimics a specimen.
The operation content by the operator for the medical device is detected, and the operation content is detected.
A medical device image related to the medical device is displayed on the simulation movie on the display unit, and the display mode of the medical device image and the simulation movie is controlled based on the operation content.
program.

1 Surgical training system 2 Surgical training device 10, 20 Tracker 30, 40 Medical equipment 31 First detection unit 32 Second detection unit 33, 64, 72 Recording unit 34, 71 Communication unit 50 Display device 60 Control device 61 1 Communication unit 62 Image processing unit 63 Input unit 65 Second communication unit 66 Control unit 70 Medical information management server 71 Communication unit 73 Generation unit 74 Reasoning unit 75 Simulation data recording unit 76 Image data recording unit 77 User data recording unit 78 Medical equipment data recording unit 79 server controller 80 hospital management server 83 input unit 90 network 641 program recording unit 721 program storage unit 791 determination unit 792 billing unit 793 number giving unit O ₁ model U1 operator

Claims (21)

A recording unit that records at least a plurality of surgical procedures in association with and at least a plurality of simulation data including data on a living tissue.
When any one of the plurality of surgical procedures is selected on the client side, a control unit that transmits simulation data corresponding to the selected surgical procedure to the client side.
To prepare
Medical information management server. The medical information management server according to claim 1.
The simulation data is generated based on the intraoperative moving image data captured during the operation of the subject and at least one of the two-dimensional image data and the three-dimensional data previously captured by the CT device or the MRI device on the subject. Further equipped with a generator to generate,
Medical information management server. The medical information management server according to claim 2.
An inference unit for estimating the organ and position of the subject reflected in the intraoperative moving image data based on the intraoperative moving image data and at least one of the two-dimensional image data and the three-dimensional data is further provided.
The generator
Based on the estimation result of the inference unit, the simulation data is generated by aligning the organs contained in at least one of the intraoperative moving image data, the two-dimensional image data, and the three-dimensional data.
Medical information management server. The medical information management server according to claim 3.
The intraoperative video data is
The subject is irradiated with multispectral light by the multispectral light source, and by receiving the return light from the subject, the spectrum image data including at least the reflection frequency for each biological tissue is included.
The inference unit
At least the type and position of the biological tissue are estimated using the reflection frequency of each biological tissue included in the spectral image data.
Medical information management server. The medical information management server according to claim 1.
The recording unit
A plurality of operator information regarding a plurality of operators who have performed each of the plurality of simulation data are further associated with the plurality of simulation data and recorded.
The control unit
When any one of the plurality of operator information is selected on the client side, the plurality of simulation data corresponding to the selected operator information is transmitted to the client side.
Medical information management server. The medical information management server according to claim 1.
The recording unit
A plurality of medical device information relating to the plurality of medical devices used in each of the plurality of simulation data is further associated and recorded with the plurality of simulation data.
The control unit
When the simulation data is transmitted to the client side, the medical device information associated with the simulation data is also transmitted to the client side.
Medical information management server. The medical information management server according to claim 6.
The control unit
When any one or more of the plurality of medical device information is selected on the client side, the selected medical device information is transmitted to the client side.
Medical information management server. The medical information management server according to claim 1.
The control unit
When the simulation data is transmitted to the client side, the operator is charged based on the identification information of the operator who operates the client side.
Medical information management server. The medical information management server according to claim 8.
The control unit
Based on the level associated with the identification information, the surgical procedure information regarding the surgical procedure selectable on the client side is transmitted to the client side.
Medical information management server. The medical information management server according to claim 9.
The control unit
Based on the identification information, a score is given to the number of times that the operator virtually trained using the simulation data.
Medical information management server. The medical information management server according to claim 8.
The recording unit
Each level of the plurality of simulation data is further associated and recorded.
Medical information management server. A display unit capable of displaying a simulation video based on the simulation data transmitted from a medical information management server that records a plurality of surgical procedures and a plurality of simulation data including at least data related to a living tissue in association with each other.
With one or more medical devices
A detection unit that detects the operation content of the operator on the medical device, and
A control unit that displays a medical device image related to the medical device on the simulation movie on the display unit,
With
The control unit controls the display mode of the medical device image and the simulation moving image based on the operation content detected by the detection unit.
Surgical training equipment. The surgical training device according to claim 12.
The control unit
When a stop signal for stopping the simulation movie displayed by the display unit is input and the detection unit detects the operation content, the viewpoint of the simulation movie is changed based on the operation content.
Surgical training equipment. The surgical training device according to claim 13.
The control unit
When a plurality of surgical procedure information relating to each of the plurality of surgical procedures is displayed on the display unit and one or more of the plurality of surgical procedure information is selected, the selected surgical procedure information is supported. Request the medical information management server to transmit the simulation data.
Surgical training equipment. The surgical training device according to claim 14.
Further provided with an input unit that accepts the input of the operator's identification information.
The control unit
Sending the identification information to the medical information management server,
Surgical training equipment. The surgical training device according to claim 12.
The control unit
Sending medical device information related to the medical device to the medical information management server,
Surgical training equipment. Medical information management server and
A surgical training device that can communicate with the medical information management server via a network,
With
The medical information management server
A recording unit that records at least a plurality of surgical procedures in association with and at least a plurality of simulation data including data on a living tissue.
When any one of the plurality of surgical procedures is selected on the client side, a first control unit that transmits simulation data corresponding to the selected surgical procedure to the client side, and
With
The surgical training device
A display unit capable of displaying a simulation movie based on the simulation data transmitted from the medical information management server, and a display unit.
One or more medical devices that can be treated on a model that mimics the subject,
A detection unit that detects the operation content of the operator on the medical device, and
A second control unit that displays a medical device image related to the medical device on the simulation movie on the display unit,
With
The second control unit controls the display mode of the medical device image and the simulation moving image based on the operation content detected by the detection unit.
Surgical training system. An image transmission method executed by a medical information management server including a recording unit that records at least a plurality of surgical procedures and at least a plurality of simulation data including data on a living tissue in association with each other.
When any one of the plurality of surgical procedures is selected on the client side, simulation data corresponding to the selected surgical procedure is transmitted to the client side.
Image transmission method. A display unit capable of displaying a simulation video based on the simulation data transmitted from the medical information management server, which records at least a plurality of surgical procedures and a plurality of simulation data including at least data on a living tissue in association with each other, and a subject. A display method performed by a surgical training device comprising one or more medical instruments capable of treating a model that mimics a specimen.
Detecting the operation contents of the medical device by the operator,
A medical device image related to the medical device is displayed on the simulation movie on the display unit, and the display mode of the medical device image and the simulation movie is controlled based on the operation content.
Display method. A program executed by a medical information management server having a recording unit that records at least a plurality of surgical procedures and at least a plurality of simulation data including data on living tissue in association with each other.
When any one of the plurality of surgical procedures is selected on the client side, simulation data corresponding to the selected surgical procedure is transmitted to the client side.
program. A display unit capable of displaying a simulation video based on the simulation data transmitted from the medical information management server, which records at least a plurality of surgical procedures and a plurality of simulation data including at least data on a living tissue in association with each other, and a subject. A program that is run by a surgical training device equipped with one or more medical instruments that can treat a model that mimics a specimen.
The operation content by the operator for the medical device is detected, and the operation content is detected.
A medical device image related to the medical device is displayed on the simulation movie on the display unit, and the display mode of the medical device image and the simulation movie is controlled based on the operation content.
program.

Images