KR20190133423A - Program and method for generating surgical simulation information - Google Patents

Abstract

Disclosed are a surgical simulation information generation method and a program thereof. The surgical simulation information generation method generates surgical simulation information which is optimized for each patient by using information obtained in advance and can assist surgery by providing the surgical simulation information to a doctor during the surgery. The surgery simulation information generation method comprises the steps of: allowing a computer to generate three-dimensional modeling data including a surgical part of a subject; allowing the computer to acquire virtual surgery data for the three-dimensional modeling data; and allowing the computer to generate cue sheet data including one or more detailed surgical motions by using the virtual surgical data.

Description

PROGRAM AND METHOD FOR GENERATING SURGICAL SIMULATION INFORMATION}

The present invention relates to a method and a program for generating surgical simulation information.

In the surgical procedure, there is a need for the development of technologies that can provide information to assist the surgeon's surgery. In order to provide information to assist the surgery, the operation must be recognizable.

Conventionally, in order to design a scenario for optimizing a surgical process, a previously taken medical image or a highly skilled doctor was consulted. However, the medical image alone was difficult to judge unnecessary processes. There was a difficult problem to consult.

Therefore, medical imaging and the advice of an experienced doctor were often difficult to be used as an aid for the optimization of the surgical process for the patient.

Thus, by optimizing the surgical process by minimizing unnecessary processes in performing surgery using a three-dimensional medical image (for example, a virtual image of the internal movement caused by the movement of the three-dimensional surgical instruments and tools) And, there is a need for the development of a method that can provide surgical assistance information based on this.

In recent years, deep learning has been widely used to analyze medical images. Deep learning is defined as a set of machine learning algorithms that attempts to achieve high levels of abstraction (summarizing key content or functions in large amounts of data or complex data) through a combination of several nonlinear transformations. Deep learning can be seen as a field of machine learning that teaches computers how people think in a large framework.

The problem to be solved by the present invention is to provide a method and program for generating surgical simulation information.

Problems to be solved by the present invention are not limited to the above-mentioned problems, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

Surgery simulation information generation method according to an aspect of the present invention for solving the above problems, the computer generating 3D modeling data including the surgical site of the object, to obtain the virtual surgery data for the 3D modeling data And generating cue sheet data including one or more detailed surgical operations using the virtual surgical data.

The generating of the cue sheet data may include: dividing the virtual surgical data into the one or more detailed surgical operations, including a surgical part included in the virtual surgical data, a type of surgical tool, the number of surgical tools, and the surgical tool. And dividing the virtual surgical data into the one or more subsurface operations based on at least one of a position, a direction of the surgical tool, and a movement of the surgical tool.

The acquiring of the 3D modeling data may include acquiring a medical image including a surgical part of the object and generating the 3D modeling data using the medical image.

The acquiring of the medical image may include acquiring a photographing posture of the object based on an actual surgical posture and acquiring the photographed medical image based on the acquired posture.

The generating of the 3D modeling data may include obtaining the actual surgical posture of the object and generating the 3D modeling data by correcting the obtained medical image based on the actual surgical posture. have.

The method may further include determining whether to optimize the generated cue sheet data.

The determining of the optimization may further include providing surgical guide data based on the cuesheet data determined to be optimized.

The determining of whether to optimize may include obtaining optimization cuesheet data and comparing the cuesheet data with the optimization cuesheet data.

The acquiring the optimized cuesheet data may include obtaining one or more learning cuesheet data, performing reinforcement learning using the one or more learning cuesheet data, and performing the optimization cuesheet data based on the reinforcement learning result. It may include the step of obtaining.

In accordance with an aspect of the present invention for solving the above problems, there is provided a computer program stored in a computer-readable recording medium to perform the method for generating surgical simulation information according to the disclosed embodiment.

Other specific details of the invention are included in the detailed description and drawings.

According to the disclosed embodiment, the surgery simulation information optimized for each patient is generated by using the information obtained in advance, and it is effective to assist the surgery by providing the same to the doctor during the surgery.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a view showing a robot surgery system according to the disclosed embodiment.
2 is a flowchart illustrating a method of generating surgical simulation information according to an exemplary embodiment.
3 is a flowchart illustrating a process of generating 3D modeling data by applying a surgical posture, according to an exemplary embodiment.
4 is a flowchart illustrating a method of generating surgical simulation information according to another exemplary embodiment.
5 is a flowchart illustrating a surgical process optimization method according to an exemplary embodiment.

Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but can be embodied in various different forms, and the present embodiments only make the disclosure of the present invention complete, and those of ordinary skill in the art to which the present invention belongs. It is provided to fully inform the skilled worker of the scope of the invention, which is defined only by the scope of the claims.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, "comprises" and / or "comprising" does not exclude the presence or addition of one or more other components in addition to the mentioned components. Like reference numerals refer to like elements throughout, and "and / or" includes each and all combinations of one or more of the mentioned components. Although "first", "second", etc. are used to describe various components, these components are of course not limited by these terms. These terms are only used to distinguish one component from another. Therefore, of course, the first component mentioned below may be a second component within the technical spirit of the present invention.

Unless otherwise defined, all terms used in the present specification (including technical and scientific terms) may be used in a sense that can be commonly understood by those skilled in the art. In addition, terms that are defined in a commonly used dictionary are not ideally or excessively interpreted unless they are specifically defined clearly.

As used herein, the term "part" or "module" refers to a hardware component such as software, FPGA, or ASIC, and the "part" or "module" plays certain roles. However, "part" or "module" is not meant to be limited to software or hardware. The “unit” or “module” may be configured to be in an addressable storage medium or may be configured to play one or more processors. Thus, as an example, a "part" or "module" may include components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, Procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. Functions provided within components and "parts" or "modules" may be combined into smaller numbers of components and "parts" or "modules" or into additional components and "parts" or "modules". Can be further separated.

As used herein, "image" may mean multi-dimensional data composed of discrete image elements (eg, pixels in a 2D image and voxels in a 3D image). For example, the image may include a medical image of the object obtained by the CT imaging apparatus.

As used herein, an "object" may be a person or an animal, or part or all of a person or an animal. For example, the subject may include at least one of organs such as the liver, heart, uterus, brain, breast, abdomen, and blood vessels.

As used herein, a "user" may be a doctor, a nurse, a clinical pathologist, a medical imaging professional, or the like, and may be a technician who repairs a medical device, but is not limited thereto.

In the present specification, "medical image data" is a medical image photographed by a medical imaging apparatus, and includes all medical images that can be implemented as a three-dimensional model of the body of an object. The medical image data may include a computed tomography (CT) image, a magnetic resonance image (MRI), a positron emission tomography (PET) image, and the like.

As used herein, the term "virtual body model" refers to a model generated according to the actual patient's body based on medical image data. The "virtual body model" may be generated by modeling medical image data in three dimensions as it is, or may be corrected as in actual surgery after modeling.

As used herein, "virtual surgery data" refers to data including rehearsal or simulation actions performed on a virtual body model. The "virtual surgery data" may be image data for which rehearsal or simulation has been performed on the virtual body model in the virtual space, or may be data recorded for a surgical operation performed on the virtual body model.

As used herein, the term "actual surgery data" refers to data obtained by performing a surgery by an actual medical staff. "Real surgery data" may be image data photographing the surgical site in the actual surgical procedure, or may be data recorded for the surgical operation performed in the actual surgical procedure.

As used herein, the term "detailed operation" refers to a minimum unit of a surgical operation divided according to a specific criterion.

As used herein, "Cue sheet data" refers to data recorded by sequentially dividing a specific surgical procedure into detailed surgical operations.

In this specification, "execution cue sheet data" means cue sheet data obtained based on virtual surgical data on which a user performs a simulation.

As used herein, "training virtual surgery cue sheet data" refers to cue sheet data generated based on virtual surgery data obtained by a user performing surgery simulation.

In the present specification, "reference virtual surgery cue sheet data" refers to cue sheet data for virtual surgery performed by a specific medical person for constructing learning big data or guiding a surgical process.

As used herein, "optimized cuesheet data" refers to cuesheet data for a surgical process optimized in terms of operation time or surgical prognosis.

In the present specification, "learning cue sheet data" means cue sheet data used for learning for calculating the optimized cue sheet data.

In the present specification, "surgical guide data" means data used as guide information in actual surgery.

As used herein, the term "computer" includes all the various devices capable of performing arithmetic processing to provide a result to a user. For example, a computer can be a desktop PC, a notebook, as well as a smartphone, a tablet PC, a cellular phone, a PCS phone (Personal Communication Service phone), synchronous / asynchronous The mobile terminal of the International Mobile Telecommunication-2000 (IMT-2000), a Palm Personal Computer (PC), a Personal Digital Assistant (PDA), and the like may also be applicable. In addition, when a head mounted display (HMD) device includes a computing function, the HMD device may be a computer. Also, the computer may correspond to a server that receives a request from a client and performs information processing.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a view showing a robot surgery system according to the disclosed embodiment.

Referring to FIG. 1, there is shown a simplified schematic diagram of a system capable of performing robotic surgery in accordance with the disclosed embodiments.

According to FIG. 1, the robotic surgical system includes a medical imaging apparatus 10, a server 20, and a control unit 30 provided in an operating room, an imaging unit 36, a display 32, and a surgical robot 34. do.

In one embodiment, the robot surgery is performed by the user controlling the surgical robot 34 using the control unit 30. In one embodiment, the robot surgery may be automatically performed by the controller 30 without the user's control.

The server 20 is a computing device including at least one processor and a communication unit.

The controller 30 includes a computing device including at least one processor and a communication unit. In one embodiment, the control unit 30 includes hardware and software interfaces for controlling the surgical robot 34.

The image capturing unit 36 includes at least one image sensor. That is, the image capturing unit 36 includes at least one camera device and is used to photograph the surgical site. In one embodiment, the imaging unit 36 is used in conjunction with the surgical robot 34. For example, the image capturing unit 36 may include at least one camera coupled with a surgical arm of the surgical robot 34.

In an embodiment, the image photographed by the image capturing unit 36 is displayed on the display 340.

The controller 30 receives information necessary for surgery from the server 20 or generates information necessary for surgery and provides the information to the user. For example, the controller 30 displays the information necessary for surgery, generated or received, on the display 32.

For example, the user performs the robot surgery by controlling the movement of the surgical robot 34 by manipulating the control unit 30 while looking at the display 32.

The server 20 generates information necessary for robotic surgery using medical image data of a subject (patient) previously photographed from the medical image photographing apparatus 10, and provides the generated information to the controller 30.

The control unit 30 provides the user with the information received from the server 20 on the display 32 or controls the surgical robot 34 using the information received from the server 20.

In one embodiment, the means that can be used in the medical imaging apparatus 10 is not limited, for example, other various medical image acquisition means such as CT, X-Ray, PET, MRI may be used.

Hereinafter, a method of generating surgical simulation information with reference to the drawings will be described in detail.

2 is a flowchart illustrating a method of generating surgical simulation information according to an exemplary embodiment.

Each step shown in FIG. 2 is performed in time series in the server 20 or the controller 30 shown in FIG. 1. Hereinafter, for the convenience of description, each step is described as being performed by a computer, but the performing agent of each step is not limited to a specific device, and all or part thereof may be performed by the server 20 or the controller 30. Can be.

Referring to FIG. 2, in the method of generating surgical simulation information according to an embodiment of the present invention, a computer generates 3D modeling data including a surgical part of the object based on medical image data of a surgical part of the subject. Step S200; Acquiring, by the computer, virtual surgical data of the 3D modeling data from a user (S400); And generating, by the computer, cue sheet data including one or more detailed surgical operations using the virtual surgical data (S600). It includes.

Hereinafter, a detailed description of each step will be described.

The computer generates 3D modeling data including the surgical site of the subject based on the medical image data photographing the surgical site of the subject (S200). In order to perform a simulation similar to a real surgery, a user needs 3D modeling data identical to a real patient.

In the past, medical imaging was performed in a posture different from the actual surgical posture, and since the medical image data was implemented as 3D modeling data, the preoperative simulation did not provide the effect of practicing the actual surgery in advance.

Specifically, the internal organ placement state or shape of the patient taken by the medical imaging apparatus while the patient is lying on the inside of the object 10 during surgery due to the influence of gravity according to the angle at which the object 10 is lying down. Because they differ from the organ placement or organ shape, they do not provide the same simulation situation as in actual surgery.

In addition, since the simulation is performed in a state inconsistent with the ups and downs of carbon dioxide inside the body for laparoscopic surgery or robotic surgery, the preoperative simulation did not provide the effect of practicing the actual surgery in advance. In order to solve this problem, it is necessary to generate 3D modeling data (ie, 3D relief models) that are identical to actual surgical conditions.

3 is a flowchart illustrating a process of generating 3D modeling data by applying a surgical posture, according to an exemplary embodiment.

In one embodiment, as shown in Figure 3, the computer generates the 3D modeling data based on the medical image data taken by applying the actual surgical posture. In detail, the computer calculates a photographing posture of the patient based on the surgical posture determined based on the location of the patient or the operation type (S202); And performing 3D modeling based on the medical image data photographed in the photographing posture (S204).

The computer calculates the photographing posture of the patient based on the surgical posture determined based on the location of the patient or the type of surgery (S202). The posture may vary depending on the location of the patient (ie, the surgical site), the type of surgery, and the like. The computer calculates a photographing posture capable of taking a medical image in the same posture as the patient's posture.

For example, when the upper body is operated while the patient's body is inclined 15 degrees, the computer may calculate the posture of the patient's back so that the upper body is only 15 degrees. By setting the posture in which only the upper body of the patient is set to 15 degrees in the photographing position, the surgical site may be photographed in the same state as during surgery by using an existing medical imaging apparatus (for example, a CT device).

Also, for example, the medical imaging apparatus may implement the same imaging posture as the surgical posture, including the function of adjusting the arrangement of the patient. For example, when the medical imaging apparatus is a computed tomography (CT) device, the table of the CT device may be tilted or rotated by a predetermined angle in a predetermined direction. The gantry may also be inclined by a predetermined angle in a predetermined direction.

Through this, the computer or the user controls the table and the gantry of the medical imaging apparatus to be inclined at the same angle as the operation posture, and the computer is taken from the medical imaging apparatus at the same angle as the actual surgery. Data can be obtained.

In addition, the computer may receive the operation posture of the patient from the medical staff, or may directly calculate the operation posture to be performed based on the location of the patient, the type of surgery.

Thereafter, the computer acquires 3D modeling data generated based on the medical image data photographed in the photographing position (S204). The computer may generate and render a 3D modeling image by using the 3D medical image of the object 10. Also, the computer may receive 3D modeling data generated after the medical image photographing apparatus is photographed in a specific posture. That is, the medical image capturing apparatus (for example, a CT apparatus) may generate 3D modeling data by performing 3D rendering based on the medical image data, and then transmit the generated 3D modeling data to a computer.

In another embodiment, the computer generates 3D modeling data by applying a correction algorithm that changes from medical image data photographed under general photographing conditions to a body state during a specific surgery. That is, the computer generates 3D modeling data by applying the body change formed as a relief model according to the body change due to gravity or the injection of carbon dioxide in a tilted state to the medical image data of the patient lying in the normal state.

For example, the correction algorithm may be calculated by learning big data formed by matching medical image data regarding general photographing conditions and physical conditions with physical state data during surgery of a specific physical condition and surgical condition. The body state data at the time of surgery may include image data of the shape or arrangement of the internal organs of the body at the time of surgery or image data of the body surface.

In one embodiment, when forming a relief model during laparoscopic surgery or robotic surgery based on medical image data (eg, CT data) taken under the general conditions, the computer is a medical image data of the patient and the body at the time of surgery Constructs big data to perform learning by matching surface data. The body surface data is taken by a medical staff prior to surgery.

For example, the medical staff acquires body surface data by photographing the modified abdominal surface by injecting carbon dioxide before performing laparoscopy or robotic surgery. As the computer learns big data, it generates correction criteria or correction algorithms that form 3D modeling data of general state into a relief model during laparoscopy or robotic surgery. Thereafter, when the medical image data of the new patient is acquired, the computer generates the relief model by modifying the 3D modeling data generated based on the medical image data based on the correction criteria or the correction algorithm.

In this way, the computer may generate and provide 3D modeling data similar to the physical state at the time of operation for a specific patient to the user.

The computer acquires virtual surgery data on the 3D modeling data from the user (S400). The user performs a simulation or rehearsal to conduct a virtual surgery on the 3D modeling image. For example, the computer may display a virtual surgical tool on the 3D modeling image, and the user may control the virtual surgical tool in various ways to perform a rehearsal to induce surgery on the 3D modeling image.

The virtual surgery data is generated as a user simulates surgery on a virtual body model of a 3D modeled patient. The process of acquiring the virtual surgery data may be a process in which the user simulates the surgery, or a process of rehearsal under the same conditions immediately before the surgery. The computer may receive virtual surgery data from a user in various ways.

In one embodiment, the computer may provide a virtual body model to a user through a virtual reality (VR) device, and input a surgical operation on the virtual body model through a controller. The controller may be in various forms capable of recognizing a user's hand movement. For example, as the controller is implemented in a glove type, it is possible to obtain detailed finger movements and real-time placement of the hand during the operation of the user.

Specifically, the computer provides a virtual body model to the HMD device worn by the user. The computer rotates or enlarges the virtual body model according to the user's operation, and accordingly, the user performs surgery planning and simulation while looking at the virtual body model implemented in the same manner as the scheduled surgery to the patient.

For example, depending on the physical characteristics of the subject, the fat, such as mentum, may not be able to see the vessels and organs that require the actual surgery, but modeling removes the mentum to accurately locate and shape the vessels and organs. In this way, it is possible to establish an optimal surgical plan for reaching a target that requires actual surgery. The computer acquires an input for selecting a specific surgical tool by the user, and acquires an operation for performing a surgery on a specific region of the virtual body model with the corresponding surgical tool through a controller.

Also, in one embodiment, the virtual surgery data may be stored as image data that implements a surgical operation performed by the user on the virtual body model. The computer may generate a simulation image based on real-time motion data obtained by the controller, a surgical tool that performs the motion, and a position in the virtual body model in which the motion is performed.

Further, in another embodiment, the virtual surgery data may be modeling data for the virtual body model itself, the entire surgical tools and motion data used at each point in the simulation or rehearsal process. That is, the virtual surgery data may be a plurality of data sets that can be received by an external computer to implement a simulation image.

The computer generates cue sheet data including one or more detailed surgery operations on the virtual surgery data (S600). That is, the computer not only generates an image of the virtual surgery performed by the user on the virtual body model, but also divides it into one or more detailed operation to generate a cue sheet representing the surgical process.

The detailed surgical operation constituting the cue sheet data is a minimum operation unit constituting the surgical process. The detailed operation can be divided by several criteria. For example, the detailed surgical motion may include the type of surgery (e.g., laparoscopic surgery, robotic surgery), the anatomical body portion on which the surgery is performed, the surgical tools used, the number of surgical tools, the direction in which the surgical tools appear on the screen, or Location, surgical instrument movement (e.g., forward / retract), and the like.

The division criteria and the detailed categories included in the division criteria may be directly set through the actual surgical data learning of the medical staff. The computer may perform the supervised learning according to the division criteria and the detailed categories set by the medical staff to divide the virtual surgical data into the smallest detailed operation.

In addition, the subdivision criteria and subcategories included in the subdivision criteria may be extracted through learning of a surgical image of a computer. For example, the computer may calculate a division criterion and a category within each division criterion by deep learning learning (ie, non-supervised learning) of the actual surgical data accumulated as big data. Thereafter, the computer generates the cue sheet data by dividing the virtual surgical data according to the division criteria generated through the actual surgical data learning.

In another embodiment, the virtual surgery data or the actual surgery data may be segmented by determining whether the virtual surgery data or the actual surgery data correspond to each division criterion through image recognition. That is, the computer recognizes the location of the anatomical organs corresponding to the division criteria, the appearing surgical tools, the number of each surgical tool, etc. on the screen in the image of the virtual surgical data or the actual surgical data, and divides them into detailed surgical operation units. Can be.

In another embodiment, the computer may perform a partitioning process for generating cue sheet data based on the surgical tool motion data included in the actual surgical data or the virtual surgical data. The actual surgery data may include various information input in the process of controlling the surgical robot, such as the type and number of surgical tools selected by the user, information on the movement of each surgical tool when the user performs the robot surgery, The virtual surgery data may also include information on the type and number of surgical tools selected by the user and the movement of each surgical tool when the virtual body model is simulated. Therefore, the computer may divide based on the information included in each time point of the actual surgical data or the virtual surgical data.

In addition, in one embodiment, the virtual surgical data or the actual surgical data includes various kinds of actions such as ablation and suture, and segmentation is performed according to the division criteria. Specifically, the process of generating the cue sheet data by dividing the actual surgical data (for example, the actual surgical image) or the virtual surgical data simulating the gastric cancer surgery into detailed surgical operations is as follows.

For example, gastric cancer surgery first includes an action of resecting some or all of the stomach, including a tumor, and an ablation of lymph nodes. In addition, depending on the state of gastric cancer, various resections and linkages are used. In addition, each action may be divided into a plurality of more detailed actions according to the specific position where the action is taken and the moving direction of the surgical tool.

For example, the detailed operation of gastric cancer surgery may be divided into an open stage, an ablation stage, a connection stage, and a suture stage.

In addition, the method of connecting the resected organs includes an in vitro anastomosis method of cutting and connecting 4-5 cm or more of the end of the tooth, or an internal anastomosis method in which an abdominal cavity is incised about 3 cm and excision and anastomosis are performed in the abdominal cavity. Can be divided in more detail according to such a specific connection method.

In addition, each surgical method may be divided into a plurality of detailed surgical operations according to the position and movement of the surgical tool.

Each of the divided suboperative operations may be given a standardized name based on the location where the suboperative operation is performed and the path of the surgical tool.

In one embodiment, terms used for standardized names may be variously defined. For example, when dealing with a specific area at the bottom right of the stomach, the name of the site may be a name commonly used in the medical field, and a more generic or refined name defined in the system according to the disclosed embodiments. Can be used.

Accordingly, the rehearsal image may be organized into cue sheet-shaped information sequentially arranged based on a standardized name of the plurality of actions. Similarly, a surgical image in which a user actually performs a surgery may also be divided into action units and arranged into cue sheet information.

In addition, as an example, the cue sheet data may be generated as code data having a specific number of digits based on a criterion of dividing into detailed surgical operations. That is, the computer divides the virtual surgical data into standardized detailed surgical operations by applying standardized segmentation criteria and designates detailed categories within the segmentation criteria, and distinguishes each detailed surgical operation by assigning standardized code values to each detailed category. Gives standardized code data.

The computer assigns digital code data to each subsurgical operation by assigning numbers or letters in order from the upper category to which the specific subsurface operation belongs according to the division criteria application order. Through this, the computer may generate the cue sheet data in a form in which standardized code data of each detailed surgery operation is listed, rather than a divided detailed surgery operation image. In addition, the user can share or transmit the simulated surgical procedure by providing only the cuesheet data composed of standardized code data.

Also, in one embodiment, the computer may assign a standardized name to the standardized code of each subsurface operation. Through this, the user can select and check only the desired surgical operation (or action) part in the entire cue sheet. Also, in this case, the user can easily grasp the progress of the surgery or rehearsal simply by looking at the cue sheets sequentially arranged based on the standardized name of each action, without seeing all the rehearsal or surgery images.

The cue sheet data may be converted into a surgical image using an image database for each detailed surgical operation. The image database may store images corresponding to each code data, and a plurality of images corresponding to each code data may be stored according to a situation. For example, certain code data may be loaded with different suboperative images in the image database, depending on the previously performed operation.

In addition, the computer may reproduce the surgical simulation image by sequentially applying each subsurgical operation included in the cue sheet data to the virtual body model, as each cue sheet data is matched and stored with a specific virtual body model.

Therefore, the image corresponding to the cue sheet may be reproduced at the same point of view as the surgical rehearsal image, or may be reconstructed and reproduced at another viewpoint. Alternatively, the image may be modeled in 3D, and a viewpoint and a position may be adjusted according to a user's manipulation.

In addition, as shown in FIG. 4, in another embodiment, the computer determines whether to optimize the trial cuesheet data generated based on the virtual surgical data of the user (S800).

The computer determines the appropriateness of the trial cuesheet data by comparing the trial cuesheet data with the optimization cuesheet data. For example, the computer checks whether the trial cue data generated by the user includes unnecessary detailed surgical operations that delay the operation time, and detailed surgical operations that must be included before or after the specific detailed surgical operation. It is judged whether or not it is missing, and whether or not it is properly generated to apply to the actual patient of the trial cuesheet data.

The computer may perform the process of calculating the optimized cuesheet data to perform evaluation of the trial cuesheet data.

5 is a flowchart illustrating a surgical process optimization method according to an exemplary embodiment.

The computer acquires one or more learning cue sheet data (S820). The learning cue sheet data is learning object data learned for calculating the optimization cue sheet data. The learning cue sheet data may include cue sheet data generated based on actual surgical data (ie, actual surgical cue sheet data) or cue sheet data generated based on simulated virtual surgery data for reference (ie, virtual surgical cue sheet data for reference). It may include. The actual surgery cue sheet data is generated by a computer dividing the actual surgery data according to the division criteria. The virtual surgery cue sheet data for reference is not obtained in a surgical training course of a user, but a simulation is performed for the purpose of constructing a learning target data or providing a reference to a trainee.

Thereafter, the computer performs reinforcement learning using the learning cue sheet data (S840). Reinforcement learning is an area of machine learning, in which an agent defined in an environment is aware of its current state and selects an action or sequence of actions that maximizes rewards from selectable actions. Reinforcement learning can be summarized as learning how to maximize rewards based on state transitions and the rewards of state transitions.

Thereafter, the computer calculates the optimization cuesheet data using the reinforcement learning result (S860). The optimized cuesheet data is calculated on the basis of the reinforcement learning result on the basis of the shortest operation time, the minimum bleeding amount, the required operation group, and the required performance order to reduce the anesthesia time.

The mandatory action group is a group of detailed operation operations that must be performed together to perform a specific detailed operation. The essential procedure is a surgical operation sequence that must be performed sequentially in the process of performing a specific surgery. For example, according to the type of operation or the type of operation, the operations and the order of operations that must appear sequentially may be determined.

In addition, the computer calculates contextual optimization cuesheet data according to the patient's physical condition, surgical site (eg, tumor tissue) condition (eg, tumor size, location, etc.) through reinforcement learning. To this end, the computer utilizes patient conditions, surgical site conditions, and the like along with the learning cuesheet data.

In one embodiment, the computer may perform virtual simulation surgery on its own. For example, the computer may generate a surgical process according to the type of surgery and the type of patient based on the disclosed surgical process optimization method, and perform a virtual surgical simulation based on the generated surgical process.

The computer may obtain an optimized surgical process by evaluating the virtual surgical simulation result and performing reinforcement learning based on the virtual surgical simulation information and the evaluation information on the result.

Even in a model trained to generate an optimal surgical process, it may be difficult to generate an optimized surgical process according to each patient and type of surgery since the actual surgery may have different body structures and types of surgery.

Therefore, the computer generates a surgical process based on the patient's body structure and the type of surgery using the learned model, and conducts reinforcement learning by performing a virtual surgery simulation, and optimizes the surgical process for each patient and type of surgery. Can be generated.

In addition, as shown in Figure 4, in another embodiment, the step of providing a surgical guide data based on the specific cue sheet data by the computer at the user's request (S1000); That is, the computer may provide the user with the cue sheet data generated by the user performing simulation or rehearsal during the operation.

The method for generating surgical simulation information according to an embodiment of the present invention described above may be implemented as a program (or an application) and stored in a medium to be executed in combination with a computer which is hardware.

The above-described program includes C, C ++, JAVA, machine language, etc. which can be read by the computer's processor (CPU) through the computer's device interface so that the computer reads the program and executes the methods implemented as the program. Code may be coded in the computer language of. Such code may include functional code associated with a function or the like that defines the necessary functions for executing the methods, and includes control procedures related to execution procedures necessary for the computer's processor to execute the functions according to a predetermined procedure. can do. In addition, the code may further include memory reference code for additional information or media required for the computer's processor to execute the functions at which location (address address) of the computer's internal or external memory should be referenced. have. Also, if the processor of the computer needs to communicate with any other computer or server remotely in order to execute the functions, the code may be used to communicate with any other computer or server remotely using the communication module of the computer. It may further include a communication related code for whether to communicate, what information or media should be transmitted and received during communication.

The stored medium is not a medium for storing data for a short time such as a register, a cache, a memory, but semi-permanently, and means a medium that can be read by the device. Specifically, examples of the storage medium include, but are not limited to, a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. That is, the program may be stored in various recording media on various servers to which the computer can access or various recording media on the computer of the user. The media may also be distributed over network coupled computer systems so that the computer readable code is stored in a distributed fashion.

The steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, in a software module executed by hardware, or by a combination thereof. The software module may be a random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any form of computer readable recording medium well known in the art.

In the above, embodiments of the present invention have been described with reference to the accompanying drawings, but those skilled in the art to which the present invention pertains may realize the present invention in other specific forms without changing the technical spirit or essential features thereof. I can understand that. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive.

10: medical imaging equipment
20: server
30: control unit
32: display
34: surgical robot
36: video recording unit

Claims (10)

Generating 3D modeling data including a surgical site of the subject by the computer;
Obtaining virtual surgery data on the 3D modeling data; And
Generating cue sheet data including one or more detailed surgical operations using the virtual surgical data; Comprising, surgical simulation information generation method. According to claim 1,
Generating the cue sheet data,
The virtual surgical data is divided into one or more detailed surgical operations, wherein the surgical site, the type of surgical tool, the number of surgical tools, the position of the surgical tool, the direction of the surgical tool, and the surgery included in the virtual surgical data. Dividing the virtual surgical data into the one or more subsurface operations based on at least one of movement of a tool; Comprising, surgical simulation information generation method. According to claim 1,
Acquiring the 3D modeling data,
Acquiring a medical image including a surgical part of the subject; And
Generating the 3D modeling data using the medical image; Comprising, surgical simulation information generation method. The method of claim 3, wherein
Acquiring the medical image,
Acquiring a photographing posture of the subject based on an actual surgical posture; And
Acquiring the photographed medical image based on the acquired photographing posture; Comprising, surgical simulation information generation method. The method of claim 3, wherein
Generating the 3D modeling data,
Acquiring an actual surgical posture of the subject; And
Generating the 3D modeling data by correcting the acquired medical image based on the actual surgical position; Comprising, surgical simulation information generation method. According to claim 1,
Determining whether to optimize the generated cuesheet data; Further comprising, surgery simulation information generation method. The method of claim 6,
The determining of the optimization is,
Providing surgical guide data based on the cuesheet data determined to be optimized; Further comprising, surgery simulation information generation method. The method of claim 6,
The determining of the optimization is,
Obtaining optimization cuesheet data; And
Comparing the cuesheet data with the optimization cuesheet data; Comprising, surgical simulation information generation method. The method of claim 8,
Acquiring the optimization cue sheet data,
Obtaining one or more learning cuesheet data;
Performing reinforcement learning using the one or more learning cue sheet data; And
Obtaining the optimization cuesheet data based on the reinforcement learning result; Comprising, surgical simulation information generation method. A computer program, coupled to a computer, which is hardware, stored on a recording medium readable by a computer to perform the method of claim 1.

Images