KR102014351B1 - Method and apparatus for constructing surgical information - Google Patents

Abstract

A relay server for constructing surgical information is provided. The relay server is for relaying communication between clients, receives actual surgical information generated on the basis of an actual surgical image from a first client, transmits the received actual surgical information to a second client, receives virtual surgical information generated by performing a virtual surgery on the basis of the actual surgical information from the second client, and transmits the virtual surgical information to the first client.

Description

Method and apparatus for constructing surgical information {METHOD AND APPARATUS FOR CONSTRUCTING SURGICAL INFORMATION}

The present invention relates to a method and apparatus for constructing surgical information.

Recently, surgery is performed using various methods, as well as when the medical staff directly performs the surgery. For example, a robotic surgery system using a surgical robot, a virtual surgery system that provides simulation of virtual surgery by providing the same virtual environment as in actual surgery, and a remote surgery system that can guide surgery remotely during surgery. Method can be used.

Accordingly, there is a need for a method that allows the operation process to proceed by using various methods during actual surgery.

The problem to be solved by the present invention is to provide a surgical information construction method and apparatus.

Problems to be solved by the present invention are not limited to the above-mentioned problems, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

The relay server for constructing surgery information according to an embodiment of the present invention, while relaying the communication between the client, receives the actual surgery information generated based on the actual surgery image from the first client, the received actual surgery information Transmits the virtual surgery information generated by performing a virtual surgery based on the actual surgery information from the second client, and transmits the virtual surgery information to the first client.

In one embodiment of the present invention, the relay server may store the actual surgery information received from the first client and the virtual surgery information received from the second client.

In an embodiment of the present invention, the relay server may generate cue sheet data on a surgery procedure based on the actual surgery information and the virtual surgery information.

In one embodiment of the present invention, the cue sheet data may include a plurality of detailed surgery operations generated by dividing a surgery procedure corresponding to the actual surgery image.

In an embodiment of the present invention, the relay server may calculate optimized cuesheet data corresponding to the actual surgical image by performing learning based on the cuesheet data.

In one embodiment of the present invention, the relay server, receiving the surgical guide information from a third client, and transmits the received surgical guide information to the first client or the second client, the third client is Receiving the actual surgery information and the virtual surgery information from the relay server, it is possible to generate the surgical guide information corresponding to the actual surgery image.

In one embodiment of the present invention, the actual surgery information, at least one of the camera information, surgical site information, and surgical tool information obtained from the actual surgery image, the virtual surgery information, the virtual surgery information It may include simulation data generated as performed.

Surgical information building method performed by the relay server according to an embodiment of the present invention, receiving the actual surgery information on the actual surgery image from the first client, transmitting the actual surgery information to the second client, Receiving virtual surgery information generated based on the actual surgery information from the second client, and storing the actual surgery information and the virtual surgery information.

A computer program according to an embodiment of the present invention is combined with a computer, which is hardware, and stored in a computer-readable recording medium to perform the method for constructing surgical information.

According to an embodiment of the present invention, surgical information can be collected from various clients in real time at the time of an actual operation, so that surgical medical staff can effectively proceed with the surgical procedure. In addition, according to an embodiment of the present invention, since the operation can be performed through a variety of surgical information and surgical system as compared to the case of performing the operation only by the surgical medical staff, it is possible to perform a more accurate operation.

According to the embodiment of the present invention, rather than transmitting and receiving the surgical information for each client, it is possible to reduce the communication load because the surgical information can be transmitted and received to each other through the relay server. In addition, since each client only needs to perform communication through a relay server, even if the number of clients increases, communication between each other can be easily performed, and an additional load does not occur. Thus, increasing the number of clients does not affect performance.

According to the embodiment of the present invention, since all the surgical information can be constructed through the relay server, it can be used as learning data later.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a schematic diagram of a system capable of performing robot surgery according to an embodiment of the present invention.
2 is a view schematically showing the configuration of a surgical information building system according to an embodiment of the present invention.
3 is a flowchart illustrating a method of building surgery information using a relay server according to an embodiment of the present invention.
4 is a diagram schematically showing the configuration of an apparatus 300 for performing surgery information construction according to an embodiment of the present invention.

Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but can be embodied in various different forms, and the present embodiments only make the disclosure of the present invention complete, and those of ordinary skill in the art to which the present invention belongs. It is provided to fully inform the skilled worker of the scope of the invention, which is defined only by the scope of the claims.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, "comprises" and / or "comprising" does not exclude the presence or addition of one or more other components in addition to the mentioned components. Like reference numerals refer to like elements throughout, and "and / or" includes each and all combinations of one or more of the mentioned components. Although "first", "second", etc. are used to describe various components, these components are of course not limited by these terms. These terms are only used to distinguish one component from another. Therefore, of course, the first component mentioned below may be a second component within the technical spirit of the present invention.

Unless otherwise defined, all terms used in the present specification (including technical and scientific terms) may be used in a sense that can be commonly understood by those skilled in the art. In addition, terms that are defined in a commonly used dictionary are not ideally or excessively interpreted unless they are specifically defined clearly.

As used herein, the term "part" or "module" refers to a hardware component such as software, FPGA, or ASIC, and the "part" or "module" plays certain roles. However, "part" or "module" is not meant to be limited to software or hardware. The “unit” or “module” may be configured to be in an addressable storage medium or may be configured to play one or more processors. Thus, as an example, a "part" or "module" may include components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, Procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. Functions provided within components and "parts" or "modules" may be combined into smaller numbers of components and "parts" or "modules" or into additional components and "parts" or "modules". Can be further separated.

As used herein, the term "computer" includes all the various devices capable of performing arithmetic processing to provide a result to a user. For example, a computer can be a desktop PC, a notebook, as well as a smartphone, a tablet PC, a cellular phone, a PCS phone (Personal Communication Service phone), synchronous / asynchronous The mobile terminal of the International Mobile Telecommunication-2000 (IMT-2000), a Palm Personal Computer (PC), a Personal Digital Assistant (PDA), and the like may also be applicable. In addition, when a head mounted display (HMD) device includes a computing function, the HMD device may be a computer. Also, the computer may correspond to a server that receives a request from a client and performs information processing.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a schematic diagram of a system capable of performing robot surgery according to an embodiment of the present invention.

According to FIG. 1, the robotic surgical system includes a medical imaging apparatus 10, a server 100, a control unit 30 provided in an operating room, a display 32, and a surgical robot 34. According to an embodiment, the medical imaging apparatus 10 may be omitted in the robot surgery system according to the disclosed embodiment.

In one embodiment, surgical robot 34 includes imaging device 36 and surgical instrument 38.

In one embodiment, the robot surgery is performed by the user controlling the surgical robot 34 using the control unit 30. In one embodiment, the robot surgery may be automatically performed by the controller 30 without the user's control.

The server 100 is a computing device including at least one processor and a communication unit.

The controller 30 includes a computing device including at least one processor and a communication unit. In one embodiment, the control unit 30 includes hardware and software interfaces for controlling the surgical robot 34.

The imaging device 36 includes at least one image sensor. That is, the imaging device 36 includes at least one camera device and is used to photograph an object, that is, a surgical site. In one embodiment, the imaging device 36 includes at least one camera coupled with a surgical arm of the surgical robot 34.

In an embodiment, the image photographed by the photographing apparatus 36 is displayed on the display 340.

In one embodiment, surgical robot 34 includes one or more surgical tools 38 that can perform cutting, clipping, fixing, grabbing operations, and the like, of the surgical site. Surgical tool 38 is used in conjunction with the surgical arm of the surgical robot 34.

The controller 30 receives information necessary for surgery from the server 100 or generates information necessary for surgery and provides the information to the user. For example, the controller 30 displays the information necessary for surgery, generated or received, on the display 32.

For example, the user performs the robot surgery by controlling the movement of the surgical robot 34 by manipulating the control unit 30 while looking at the display 32.

The server 100 generates information necessary for robotic surgery using medical image data of an object previously photographed from the medical image photographing apparatus 10, and provides the generated information to the controller 30.

The controller 30 displays the information received from the server 100 on the display 32 to provide the user, or controls the surgical robot 34 by using the information received from the server 100.

In one embodiment, the means that can be used in the medical imaging apparatus 10 is not limited, for example, other various medical image acquisition means such as CT, X-Ray, PET, MRI may be used.

In the disclosed embodiment, the surgical image obtained by the imaging device 36 is transmitted to the controller 30.

Hereinafter, a description will be given of the process of establishing the surgical information by using a relay server for transmitting and receiving information between the clients. The relay server may mean the server 100 of FIG. 1 or may be separately installed and operate in conjunction with the server 100 of FIG. 1.

In recent years, the medical staff may perform the surgery by various methods such as using a surgical robot or a simulator or guiding the operation remotely, as well as performing the surgery directly in the operating room. Accordingly, there is a need for a method of connecting various devices used during surgery and transmitting and receiving information therebetween to provide a more effective and optimized surgery to the patient during surgery. Therefore, the present invention is to provide a method for connecting each device using a relay server and through this to collect and utilize information.

2 is a view schematically showing the configuration of a surgical information building system according to an embodiment of the present invention.

2, the surgical information building system according to an embodiment of the present invention may include a relay server 200 and a plurality of clients. As an example, in FIG. 2, the first client 210, the second client 220, and the third client 230 are described as one example. However, the present invention is not limited thereto. . Of course, it can be configured to include fewer or more clients than shown in FIG.

The relay server 200 is a device that relays communication between a plurality of clients, and may perform a function such as, for example, a chat server. For example, the relay server 200 may relay a surgery by connecting a robot surgery system using a surgery robot, a virtual surgery system using a simulator, a remote system that guides the surgery remotely during surgery, and the like.

In addition, the relay server 200 may build learning data in response to receiving various surgical information from a plurality of clients. According to an embodiment, the relay server 200 may perform learning based on the learning data and calculate an optimized surgical process, or may provide the learning data to an external server. In this case, the external server may calculate the optimized surgical process by performing the learning based on the learning data provided from the relay server 200.

The first client 210 may be an apparatus for acquiring an actual surgery image and generating actual surgery information. In one embodiment, the first client 210 may be a device provided in the operating room (operating site). For example, it may correspond to the surgical robot 34 or the control unit 30 shown in FIG. 1, or may be provided in the operating room as a separate device and connected to the surgical robot 34 or the control unit 30.

The first client 210 may access the relay server 200 and transmit the actual surgery information generated at the time of operation to the relay server 200. In addition, the first client 210 may receive various information generated by another client from the relay server 200.

Here, the actual surgery image may mean data obtained as the actual medical staff performs the surgery. For example, it may be an image of an actual surgical scene actually performed by the surgical robot 34. In other words, the actual surgical image is data recorded on the surgical site and the operation during the actual surgical procedure.

The actual surgery information is information obtained from an actual surgery image, and may include, for example, surgical site information of a patient, surgical tool information, and information about a camera. In this case, the surgical site refers to a body part of the patient where the actual surgery is performed, and may be, for example, an organ or a blood vessel. The surgical tool may be a tool used during surgery, for example, a tool required for performing a robotic operation or a consumable used during surgery. The camera is a device for photographing an actual surgical scene. For example, the camera may be installed in an operating room to photograph a surgical scene, or may be provided in an endoscope that enters a patient's body when performing a robotic surgery to photograph an internal body.

For example, the first client 210 acquires a real surgery image from a camera or an external device included in a surgical robot, recognizes a surgical site, a surgical tool, and a camera from the acquired real surgery image, and based on the actual surgery information. Can be generated. That is, the first client 210 may generate the type and position information of the surgical site, the type, position, direction, and movement information of the surgical tool, the position and movement direction of the camera, and the like from the actual surgical image. Thereafter, the first client 210 may transmit the generated actual surgery information to the relay server 200, and provide the generated actual surgery information to other clients connected to the relay server 200.

The second client 220 may be an apparatus for generating virtual surgery information by performing virtual surgery. In one embodiment, the second client 220 may be a device provided in an operating room (operating site) or a remote location, for example, rehearsing or simulating a virtual body model modeled the same as the patient (ie, a patient). It may be a simulator capable of performing an action.

Here, the virtual surgery refers to a virtual surgery that performs rehearsal or simulation on a virtual body model that is implemented in the same manner as the physical state based on the medical image data (eg, CT, MRI, PET images, etc.) of the subject. . The virtual surgery information is information obtained based on the image data on which the rehearsal or simulation is performed on the virtual body model in the virtual space, and may include simulation data. For example, the simulation data may include surgical site information, surgical tool information, and information about a camera of a surgical subject. That is, the virtual surgery information may include all the information included in the data recorded for the surgical operation performed on the virtual body model.

In addition, the virtual body model may be 3D modeling data generated based on medical image data previously photographed inside the body of the subject. For example, the model may be modeled in accordance with the body of the surgical subject, and may be corrected to the same state as the actual surgical state.

The second client 220 may access the relay server 200 and transmit the virtual surgery information generated by performing the virtual surgery to the relay server 200. In addition, the second client 220 may receive various information generated by another client from the relay server 200.

For example, the second client 220 receives the actual surgery information generated by the first client 210 from the relay server 200 and performs virtual surgery based on the received actual surgery information to perform virtual surgery information. Can be generated.

In the case of virtual surgery, it is important to provide a virtual surgical condition that is identical to the actual surgical condition for the subject. Accordingly, in the present invention, the second client 220 receives the actual surgery information (eg, position, direction, and movement information of the surgical tool) obtained by performing the actual surgery on the first client 210 through the relay server 200. Since it can be transmitted in real time, the second client 220 can provide the surgical medical staff with more accurate virtual surgery information reflecting the actual surgery information in real time. Therefore, since the surgical medical staff can use both the information of the first client 210 and the second client 220 through the relay server 200, it is possible to effectively perform the actual surgery for the surgical subject.

The third client 230 may be a device for generating surgery guide information corresponding to the actual surgery image. In one embodiment, the third client 230 may be a device provided in the operating room (operation site) or a remote location, for example, when a specific event occurs during the operation may generate the surgical guide information in response thereto. Alternatively, another medical staff may access the third client 230 at the request of the medical staff performing the surgery in the operating room (operation site). In this case, by performing communication between the third client 230 through the relay server 200, other medical staff can provide information about the surgery to the medical staff performing the surgery in the operating room (operation site) in real time.

The third client 230 may access the relay server 200 to transmit the surgical guide information generated by the third client 230 to the relay server 200, and receive various information generated from other clients from the relay server 200. .

For example, the third client 230 may monitor the surgical situation according to the information (ie, actual surgery information and virtual surgery information) provided by the other clients 210 and 220 from the relay server 200. . In addition, the third client 230 may recognize whether a specific event occurs during the surgery based on the monitoring, and generate surgery guide information corresponding thereto. According to an embodiment, the third client 230 may automatically recognize whether a specific event occurs and generate surgery guide information determined according to the specific event. Alternatively, the medical staff may monitor the actual surgical situation through the third client 230 to recognize that a specific event occurs, and input surgery guide information corresponding thereto.

Relay server 200 according to an embodiment of the present invention, in addition to the above-described first, second, third client (210, 220, 230) can additionally connect the necessary client in accordance with the operation, and transmit and receive information between these clients It can play a role in controlling it.

On the other hand, when directly transmitting and receiving the surgical information between the client without passing through the relay server 200, since the surgical information is shared only between the client directly communicates with other clients can not obtain information about the operation process . Accordingly, there is a problem that other clients are difficult to intervene in the middle of the operation and cannot confirm the operation process in real time.

In the embodiment of the present invention to solve this problem, as described above, the relay server 200 serves to relay the communication between the client, the intermediate intervention through the relay server 200 during the actual operation to perform Can be. In addition, the relay server 200 may monitor the actual surgical process in real time. In addition, since the relay server 200 can obtain the surgical information from all the clients, it is possible to derive information about the entire surgical procedure. Accordingly, the relay server 200 may record the entire surgical procedure in order through such surgical information to form historical information about the corresponding surgery.

3 is a flowchart illustrating a method of building surgery information using a relay server according to an embodiment of the present invention. The method of FIG. 3 may be performed in the surgical information building system shown in FIG. 2.

According to an embodiment, first, the relay server 200 may generate at least one client required as a group in operation. For example, the relay server 200 may generate clients required for actual surgery as a group based on a specific patient, or may receive a group request signal directly from a client and create a group in response thereto. . Alternatively, when the relay server 200 receives an access request from the client, the relay server 200 may automatically determine whether the relay server belongs to the corresponding group and process the client to access the group.

Subsequently, the relay server 200 may transmit and receive information between clients in the group as the surgery proceeds. A detailed operation process thereof will be described below.

Referring to FIG. 3, the first client 210 may acquire an actual surgery image and generate real surgery information corresponding to the obtained actual surgery image (S100).

In one embodiment, the first client 210 obtains a real surgery image from a camera or an external device included in the surgical robot, and recognizes a surgical site, a surgical tool, and a camera from the acquired real surgery image, based on the actual surgery. Information can be generated. As described above, the actual surgical information may include surgical site information (e.g., type and location information of the surgical site), surgical tool information (e.g., type, position, direction, and movement information of the surgical tool), and camera information (e.g., camera). Location and motion information), and the like.

The first client 210 may transmit the actual surgery information to the relay server 200 (S105), and the relay server 200 may store the actual surgery information received from the first client 210 in the database ( S110).

In addition, the relay server 200 may transmit the actual surgery information to at least one client (eg, the second client 220 or / and the third client 230) in the same group (S115).

The second client 220 may perform virtual surgery to generate virtual surgery information (S120).

In an embodiment, the second client 220 receives the actual surgery information generated by the first client 210 through the relay server 200 and performs virtual surgery based on the received actual surgery information to perform virtual surgery. Information can be generated.

The second client 220 may transmit the virtual surgery information to the relay server 200 (S125), and the relay server 200 may store the virtual surgery information received from the second client 220 in a database ( S130).

In addition, the relay server 200 may transmit the virtual surgery information to at least one client (eg, the first client 210 or / and the third client 230) in the same group (S135).

The third client 230 may monitor the surgery situation based on the surgery information on the surgery subject received through the relay server 200, and may generate the surgery guide information based on the monitoring result (S140).

In an embodiment, the third client 230 generates a specific event during the surgery based on the actual surgery information of the first client 210 and the virtual surgery information of the second client 220 received from the relay server 200. Whether or not, and generate surgical guide information corresponding thereto. In another embodiment, the third client 230 may acquire the actual surgical image through the relay server 200 and provide the same to the medical staff to directly generate the surgical guide information corresponding to the actual surgical image. In this case, the third client 230 may receive the surgical guide information from the medical staff, and transmit it to the relay server 200.

The third client 230 may transmit the surgical guide information to the relay server 200 (S145), the relay server 200 may store the surgical guide information received from the third client 230 in the database ( S150).

In addition, the relay server 200 may transmit the surgical guide information to at least one client (eg, the first client 210 or / and the second client 220) in the same group (S155).

As described in the above steps S100 ~ S155, the relay server 200 acts as a relay to transmit and receive information between the clients belonging to the group as the operation performed on the operation target object, and stores the transmitted and received information in the database Can be. In addition, the relay server 200 may perform learning to build learning data and provide an optimized surgical process by using various surgical information on the operation targets stored in the database.

The relay server 200 may generate cue sheet data about a surgical procedure of a surgery target subject based on various surgery information received from a client (S160).

In an embodiment, the relay server 200 may include at least one of actual surgery information of the first client 210, virtual surgery information of the second client 220, and surgery guide information of the third client 230. Based on the cue sheet data for the surgical procedure of the surgical subject can be generated.

Here, the cue sheet data may refer to data recorded in order by dividing a specific surgical procedure into detailed surgical operations. The detailed surgical operation may be a minimum operation unit constituting the surgical process, and may be divided by various criteria. For example, the detailed operation may include the type of surgery (e.g., laparoscopic surgery, robotic surgery, etc.), the anatomical body area on which the surgery is performed, the surgical tools used, the number of surgical tools, the direction or location in which the surgical tools appear on the screen. This may be based on the movement of the surgical tool (e.g., forward / retract, etc.). Since the relay server 200 can grasp the information on the criteria for dividing the detailed operation through the actual surgery information, the virtual surgery information, the operation guide information, and the like stored in the database, the cue sheet data corresponding to the actual surgery of the patient Can be built accurately. According to an embodiment of the present invention, the relay server 200 may form a history of the entire surgical procedure by simply generating the cue sheet data. In addition, according to an embodiment of the present invention, the relay server 200 may perform step S165.

The relay server 200 may calculate the optimized cue sheet data corresponding to the surgical process of the surgical subject by performing the learning based on the cue sheet data (S165).

In one embodiment, the relay server 200 may configure the actual surgery information, virtual surgery information, surgery guide information, and the like stored in the database as the training data, or may configure the cue sheet data itself as the training data. Thereafter, the relay server 200 may perform learning based on the learning data and calculate an optimized surgical process for the surgical target. Here, the learning method may be a machine learning method such as supervised learning, unsupervised learning, reinforcement learning, for example, deep learning learning method.

The relay server 200 may store the optimized cuesheet data in a database and provide the optimized cuesheet data to at least one client in the group. Accordingly, the at least one client may use the surgical subject's surgery based on the received optimized cuesheet data.

In this embodiment, the relay server 200 calculates the optimized cue sheet data by performing the learning based on the training data, but according to the embodiment, the relay server 200 stores the actual surgical information and the virtual surgery stored in the database. Information, surgery guide information, etc. (ie, learning data) may be provided to an external server, and learning may be performed based on the learning data in the external server.

In addition, the embodiment of the present invention may implement a virtual body model of the surgical subject based on the cue sheet data for the surgical subject generated in step S160 or / and step S165, or conversely the operation of the surgery performed on the surgical subject You can also implement an image.

According to an embodiment of the present invention, surgical information can be collected from various clients in real time at the time of an actual operation, so that surgical medical staff can effectively proceed with the surgical procedure. In addition, according to an embodiment of the present invention, since the operation can be performed through a variety of surgical information and surgical system as compared to the case of performing the operation only by the surgical medical staff, it is possible to perform a more accurate operation.

According to the embodiment of the present invention, rather than transmitting and receiving the surgical information for each client, it is possible to reduce the communication load because the surgical information can be transmitted and received to each other through the relay server. In addition, since each client only needs to perform communication through a relay server, even if the number of clients increases, communication between each other can be easily performed, and an additional load does not occur. Thus, increasing the number of clients does not affect performance.

According to the embodiment of the present invention, since all the surgical information can be constructed through the relay server, it can be used as learning data later.

4 is a diagram schematically showing the configuration of an apparatus 300 for performing surgery information construction according to an embodiment of the present invention.

Referring to FIG. 4, the processor 310 may include a connection passage (for example, a bus or the like) that transmits and receives signals with one or more cores (not shown) and a graphic processor (not shown) and / or other components. ) May be included.

The processor 310 according to an embodiment executes one or more instructions stored in the memory 320 to perform the surgery information construction method described with reference to FIGS. 2 and 3.

For example, the processor 310 receives the actual surgery information on the actual surgery image from the first client, transmits the actual surgery information to the second client by executing one or more instructions stored in the memory 320, and the second client. The virtual surgery information generated based on the actual surgery information may be received from the client, and the actual surgery information and the virtual surgery information may be stored.

Meanwhile, the processor 310 may read random access memory (RAM) and read-only memory (ROM) for temporarily and / or permanently storing a signal (or data) processed in the processor 310. , Not shown) may be further included. In addition, the processor 310 may be implemented in the form of a system on chip (SoC) including at least one of a graphic processor, a RAM, and a ROM.

The memory 320 may store programs (one or more instructions) for processing and controlling the processor 310. Programs stored in the memory 320 may be divided into a plurality of modules according to their functions.

The surgical information building method according to an embodiment of the present invention described above may be implemented as a program (or an application) and stored in a medium to be executed in combination with a computer which is hardware.

The above-described program includes C, C ++, JAVA, machine language, etc. which can be read by the computer's processor (CPU) through the computer's device interface so that the computer reads the program and executes the methods implemented as the program. Code may be coded in the computer language of. Such code may include functional code associated with a function or the like that defines the necessary functions for executing the methods, and includes control procedures related to execution procedures necessary for the computer's processor to execute the functions according to a predetermined procedure. can do. In addition, the code may further include memory reference code for additional information or media required for the computer's processor to execute the functions at which location (address address) of the computer's internal or external memory should be referenced. have. Also, if the processor of the computer needs to communicate with any other computer or server remotely in order to execute the functions, the code may be used to communicate with any other computer or server remotely using the communication module of the computer. It may further include a communication related code for whether to communicate, what information or media should be transmitted and received during communication.

The stored medium is not a medium for storing data for a short time such as a register, a cache, a memory, but semi-permanently, and means a medium that can be read by the device. Specifically, examples of the storage medium include, but are not limited to, a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. That is, the program may be stored in various recording media on various servers to which the computer can access or various recording media on the computer of the user. The media may also be distributed over network coupled computer systems so that the computer readable code is stored in a distributed fashion.

The steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, in a software module executed by hardware, or by a combination thereof. Software modules may include random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any form of computer readable recording medium well known in the art.

In the above, embodiments of the present invention have been described with reference to the accompanying drawings, but those skilled in the art to which the present invention pertains may realize the present invention in other specific forms without changing the technical spirit or essential features thereof. I can understand that. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive.

Claims (9)

In the relay server for building the surgical information,
Relay communications between clients,
Receiving actual surgery information generated based on the actual surgery image from the first client, transmitting the received actual surgery information to the second client,
Receiving virtual surgery information generated by performing virtual surgery based on the actual surgery information from the second client, and transmitting the virtual surgery information to the first client,
The first client generates the actual surgery information based on the actual surgery image,
The second client generates the virtual surgery information based on the actual surgery information,
The relay server,
And a cue sheet data for a surgery procedure based on the actual surgery information received from the first client and the virtual surgery information received from the second client. The method of claim 1,
The relay server,
And storing the actual surgery information received from the first client and the virtual surgery information received from the second client. delete The method of claim 1,
The cue sheet data,
And a plurality of detailed surgery operations generated by dividing a surgery procedure corresponding to the actual surgery image. The method of claim 1,
The relay server,
And performing the learning based on the cue sheet data to calculate optimized cue sheet data corresponding to the actual surgical image. The method of claim 1,
The relay server,
Receiving surgery guide information from a third client, transmitting the received surgery guide information to the first client or the second client,
The third client,
And receiving the actual surgery information and the virtual surgery information from the relay server and generating surgery guide information corresponding to the actual surgery image. The method of claim 1,
The actual surgery information,
At least one of the camera information, surgical site information, and surgical instrument information obtained from the actual surgical image,
The virtual surgery information,
Relay server, characterized in that it comprises a simulation data generated by performing the virtual surgery. In the surgery information construction method performed by the relay server,
Receiving actual surgery information on the actual surgery image from the first client;
Transmitting the actual surgical information to a second client;
Receiving virtual surgery information generated based on the actual surgery information from the second client; And
Storing the actual surgery information and the virtual surgery information,
The first client generates the actual surgery information based on the actual surgery image,
The second client generates the virtual surgery information based on the actual surgery information,
And generating, by the relay server, cue sheet data on a surgery procedure based on the actual surgery information received from the first client and the virtual surgery information received from the second client. How to build information. A computer program, coupled to a computer, which is hardware, stored on a recording medium readable by a computer to perform the method of claim 8.

Images