KR20190088419A - Program and method for generating surgical simulation information - Google Patents

Abstract

Disclosed is a method for generating surgical simulation information, which includes a step of allowing a computer to generate 3D modeling data including the surgical site of an object, a step of obtaining virtual surgery data for the 3D modeling data, and a step of generating cue sheet data including one or more detailed surgery operations using the virtual surgery data. It is possible to generate surgery simulation information.

Description

TECHNICAL FIELD [0001] The present invention relates to a surgical simulation information generating method,

The present invention relates to a method and a program for generating surgical simulation information.

In the surgical procedure, development of techniques capable of providing information for assisting the surgeon of the doctor is required. In order to provide information for assisting the operation, it is necessary to be able to recognize the operation.

Previously, for the scenario design to optimize the surgical process, consultation with pre-recorded medical images or consultation with a highly skilled physician was made, but it was difficult to judge the unnecessary process by the medical image only. There was a difficult problem to receive consultation.

Therefore, medical imaging and consultation with skilled physicians are often difficult to use as an adjunct for the optimization of surgical procedures for patients undergoing surgery.

Thus, by using a 3D medical image (for example, a virtual image of a change in internal organs generated due to movement of a three-dimensional surgical tool and a tool), unnecessary processes are minimized to optimize the surgical process And to develop a method for providing surgical assistance information based thereon.

In recent years, deep learning has been widely used in the analysis of medical images. Deep learning is defined as a set of machine learning algorithms that try to achieve high levels of abstraction (a task that summarizes key content or functions in large amounts of data or complex data) through a combination of several nonlinear transformation techniques. Deep learning can be viewed as a field of machine learning that teaches computers how people think in a big way.

A problem to be solved by the present invention is to provide a method and a program for generating surgical simulation information.

The problems to be solved by the present invention are not limited to the above-mentioned problems, and other problems which are not mentioned can be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided a method for generating a surgical simulation information, the method comprising: generating 3D modeling data including a surgical site of a target object; acquiring virtual surgery data on the 3D modeling data; And generating the cue chart data including one or more detailed surgery operations using the virtual surgery data.

The step of generating the cue chart data may further include dividing the virtual surgery data into the one or more detailed surgery operations, wherein the virtual surgery data includes at least one of a surgical site, a type of surgical tool, Dividing the virtual surgical data into the one or more detailed surgical operations based on at least one of a position of the surgical tool, a direction of the surgical tool, and a motion of the surgical tool.

The acquiring of the 3D modeling data may include acquiring a medical image including a surgical site of the target object, and generating the 3D modeling data using the medical image.

The step of acquiring the medical image may include acquiring a photographing posture of the object based on an actual posture and acquiring the medical image photographed based on the obtained photographing posture.

In addition, the step of generating the 3D modeling data may include generating the 3D modeling data by correcting the obtained medical image based on the actual posture of the object and the actual posture of the object have.

The method may further include determining whether the generated queue chart data is optimized.

The step of determining whether the optimization is performed may further include providing surgical guide data based on the cue chart data determined to be optimized.

The step of determining whether the optimization is performed may include acquiring optimized queue data and comparing the queue data and the optimized queue data.

The acquiring of the optimized queue data may further include acquiring one or more learning queue data, performing the reinforcement learning using the at least one learning queue data, and extracting the optimized queue data based on the reinforcement learning result. And a step of acquiring.

According to an aspect of the present invention, there is provided a computer program stored in a computer-readable recording medium for performing a method of generating simulation information for surgery according to an embodiment of the present invention, the computer program being stored in a computer readable by a computer.

Other specific details of the invention are included in the detailed description and drawings.

According to the disclosed embodiment, it is possible to assist surgery by generating surgical simulation information optimized for each patient using previously obtained information, and providing it to a doctor during surgery.

The effects of the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description.

1 is a diagram illustrating a robot surgery system in accordance with the disclosed embodiment.
2 is a flow chart illustrating a method for generating surgical simulation information according to one embodiment.
3 is a flowchart illustrating a process of generating 3D modeling data by applying an operation posture according to an embodiment.
4 is a flowchart illustrating a method of generating surgical simulation information according to another embodiment.
5 is a flow chart illustrating a method of optimizing a surgical procedure according to an embodiment.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and the manner of achieving them, will be apparent from and elucidated with reference to the embodiments described hereinafter in conjunction with the accompanying drawings. It should be understood, however, that the invention is not limited to the disclosed embodiments, but may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, Is provided to fully convey the scope of the present invention to a technician, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of illustrating embodiments and is not intended to be limiting of the present invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. The terms " comprises "and / or" comprising "used in the specification do not exclude the presence or addition of one or more other elements in addition to the stated element. Like reference numerals refer to like elements throughout the specification and "and / or" include each and every combination of one or more of the elements mentioned. Although "first "," second "and the like are used to describe various components, it is needless to say that these components are not limited by these terms. These terms are used only to distinguish one component from another. Therefore, it goes without saying that the first component mentioned below may be the second component within the technical scope of the present invention.

Unless defined otherwise, all terms (including technical and scientific terms) used herein may be used in a sense that is commonly understood by one of ordinary skill in the art to which this invention belongs. In addition, commonly used predefined terms are not ideally or excessively interpreted unless explicitly defined otherwise.

As used herein, the term "part" or "module" refers to a hardware component, such as a software, FPGA, or ASIC, and a "component" or "module" performs certain roles. However, "part" or " module " is not meant to be limited to software or hardware. A "module " or " module " may be configured to reside on an addressable storage medium and configured to play back one or more processors. Thus, by way of example, "a" or " module " is intended to encompass all types of elements, such as software components, object oriented software components, class components and task components, Microcode, circuitry, data, databases, data structures, tables, arrays, and variables, as used herein. Or " modules " may be combined with a smaller number of components and "parts " or " modules " Can be further separated.

As used herein, the term "image" may refer to multi-dimensional data composed of discrete image elements (e.g., pixels in a two-dimensional image and voxels in a 3D image). For example, the image may include a medical image or the like of the object obtained by the CT photographing apparatus.

As used herein, an " object "may be a person or an animal, or part or all of a person or an animal. For example, the subject may comprise at least one of the following: liver, heart, uterus, brain, breast, organs such as the abdomen, and blood vessels.

As used herein, the term "user" may be a doctor, a nurse, a clinical pathologist, a medical imaging specialist, or the like, and may be a technician repairing a medical device.

In the present specification, the term "medical image data" is a medical image captured by a medical image capturing apparatus, and includes all medical images capable of realizing a body of a subject as a three-dimensional model. "Medical image data" may include a computed tomography (CT) image, a magnetic resonance imaging (MRI), a positron emission tomography (PET) image, and the like.

As used herein, the term "virtual body model" refers to a model generated based on medical image data in accordance with an actual patient's body. The "virtual body model" may be generated by modeling the medical image data in three dimensions as it is, or may be corrected after modeling as in actual surgery.

As used herein, the term "virtual surgery data" means data including a rehearsal or simulation action performed on a virtual body model. The "virtual surgery data" may be image data that is rehearsed or simulated for a virtual body model in virtual space, or data recorded for a surgical operation performed on a virtual body model.

In the present specification, the term "actual operation data" means data obtained as an actual medical staff performs surgery. The "actual operation data" may be image data obtained by photographing a surgical site in an actual operation procedure, or may be data recorded on a surgical operation performed in an actual operation procedure.

As used herein, the term " detailed operation "means a minimum unit of a surgical operation divided according to a specific criterion.

In the present specification, "cue sheet data" means data in which a specific operation procedure is divided into detailed operation operations and recorded in order.

In the present specification, "trial queue data" means the queue sheet data obtained based on the virtual surgery data on which the user has performed the simulation.

In the present specification, the term " training virtual surgery < / RTI > queue chart data "is included in the enactment queue data and refers to the queue chart data generated based on the virtual surgery data obtained by performing the operation simulation.

In the present specification, "reference virtual surgery queue data" refers to cue chart data for a virtual surgery performed by a specific medical person for guidance of construction of a big data for learning or surgical procedure guidance.

As used herein, the term " optimized queue data "means queue sheet data for a surgical procedure optimized in terms of an operation time or an operation prognosis.

In the present specification, "learning cue chart data" means the cue chart data used for learning for calculating optimization cue sheet data.

In the present specification, "surgical guide data" means data used as guide information in actual surgery.

As used herein, the term "computer" includes all of the various devices that can perform computational processing to provide results to a user. For example, the computer may be a smart phone, a tablet PC, a cellular phone, a personal communication service phone (PCS phone), a synchronous / asynchronous A mobile terminal of IMT-2000 (International Mobile Telecommunication-2000), a Palm Personal Computer (PC), a personal digital assistant (PDA), and the like. In addition, when the head mounted display (HMD) device includes a computing function, the HMD device can be a computer. Also, the computer may correspond to a server that receives a request from a client and performs information processing.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating a robot surgery system in accordance with the disclosed embodiment.

Referring to FIG. 1, there is shown a simplified schematic representation of a system capable of performing robotic surgery in accordance with the disclosed embodiments.

1, the robot surgery system includes a medical imaging apparatus 10, a server 20, a control unit 30 provided in an operating room, an image capturing unit 36, a display 32, and a surgical robot 34 do.

In one embodiment, robotic surgery is performed by the user controlling the surgical robot 34 using the control unit 30. [ In one embodiment, robot surgery may be performed automatically by the control unit 30 without user control.

The server 20 is a computing device including at least one processor and a communication unit.

The control unit 30 includes a computing device including at least one processor and a communication unit. In one embodiment, the control unit 30 includes hardware and software interfaces for controlling the surgical robot 34.

The image capturing unit 36 includes at least one image sensor. That is, the image capturing unit 36 includes at least one camera device, and is used to photograph a surgical site. In one embodiment, the imaging section 36 is used in combination with the surgical robot 34. For example, the image capturing unit 36 may include at least one camera coupled with a surgical arm of the surgical robot 34. [

In one embodiment, the image photographed by the image photographing section 36 is displayed on the display 340. [

The control unit 30 receives information necessary for surgery from the server 20, or generates information necessary for surgery and provides the information to the user. For example, the control unit 30 displays on the display 32 information necessary for surgery, which is generated or received.

For example, the user operates the control unit 30 while viewing the display 32 to perform the robot surgery by controlling the movement of the surgical robot 34.

The server 20 generates information necessary for robot surgery using the medical image data of the object (patient) photographed beforehand from the medical imaging apparatus 10, and provides the generated information to the control unit 30. [

The control unit 30 provides the information received from the server 20 to the user by displaying the information on the display 32 or controls the surgical robot 34 using the information received from the server 20. [

In one embodiment, the means that can be used in the medical imaging equipment 10 is not limited, and various other medical imaging acquiring means such as CT, X-Ray, PET, MRI and the like may be used.

Hereinafter, a method of generating surgical simulation information will be described in detail with reference to the drawings.

2 is a flow chart illustrating a method for generating surgical simulation information according to one embodiment.

Each step shown in FIG. 2 is performed in a time-series manner in the server 20 or the control unit 30 shown in FIG. Hereinafter, for convenience of description, each step is described as being performed by a computer, but the subject of each step is not limited to a specific device, and all or some of the steps may be performed in the server 20 or the control unit 30 .

Referring to FIG. 2, a method for generating simulation information for surgery according to an exemplary embodiment of the present invention includes generating 3D modeling data including a surgical region of a target object based on medical image data of a surgical region of a target object Step S200; The computer obtaining virtual surgery data for the 3D modeling data from a user (S400); And generating (S600) the cue chart data including one or more detailed surgery operations using the virtual surgery data; .

Hereinafter, a detailed description of each step will be described.

The computer generates 3D modeling data including a surgical region of the target object based on the medical image data obtained by photographing the surgical target region of the target object (S200). In order for a user to perform a simulation similar to an actual operation, the same 3D modeling data as the actual patient is required.

Previously, medical imaging was performed in a different posture from the actual posture, and the medical image data was directly embodied as 3D modeling data, so that the pre-operation simulation did not provide the effect of practicing the actual surgery in advance.

In general, an internal organ placement state or a long-term shape of a patient, which is photographed by a medical imaging apparatus while the patient is lying down, Long-term placement, or long-term appearance, and thus does not provide the same simulation situation as the actual surgery.

In addition, since the simulation is performed in a state where the inside of the body is filled with carbon dioxide for the laparoscopic surgery or the robot operation, the pre-operation simulation does not provide the effect of practicing the actual surgery in advance. In order to solve this problem, it is necessary to generate 3D modeling data (i.e., a 3D undulation model) that is the same as the actual operation state.

3 is a flowchart illustrating a process of generating 3D modeling data by applying an operation posture according to an embodiment.

In one embodiment, as shown in FIG. 3, the computer generates 3D modeling data based on the medical image data photographed by applying the actual posture. Specifically, the computer calculates a photographing posture of the patient on the basis of the operation posture determined based on the affected location or the operation type of the patient (S202); And performing 3D modeling based on the medical image data photographed in the photographing posture (S204).

The computer calculates a photographing posture of the patient on the basis of the operation posture determined based on the affected location or the operation type of the patient (S202). The position of the affected part of the patient (i.e., the surgical site), the type of operation, and the like. The computer calculates a posture in which the medical imaging can be performed in the same posture as the patient's posture.

For example, when the patient's body is tilted at 15 degrees and the upper body is operated, the computer can calculate the back posture of the patient so that the upper body is raised by only 15 degrees. By setting the attitude of the patient's upper body only 15 degrees to the photographing posture, the surgical site can be photographed in the same state as the operation using the conventional medical image photographing apparatus (for example, CT apparatus).

In addition, for example, the medical image photographing apparatus can implement the same photographing posture as the operation posture, including a function of adjusting the placement state of the patient. For example, when the medical imaging apparatus is a computed tomography (CT) apparatus, the table of the CT apparatus can be tilted or rotated by a predetermined angle in a predetermined direction , The gantry can also be inclined by a predetermined angle in a predetermined direction.

Thus, the computer or the user controls the table and the gantry of the medical image capturing apparatus to be inclined at the same angle as the operation posture, and the computer displays the medical image captured from the medical image capturing apparatus in the tilted posture Data can be acquired.

In addition, the computer may receive the operation posture of the patient from the medical staff, directly calculate the posture to be performed based on the location of the patient, the operation type, and the like.

Thereafter, the computer acquires 3D modeling data generated based on the medical image data photographed in the photographing posture (S204). The computer can generate a 3D modeling image by rendering the 3D medical image of the object 10. Further, the computer may receive the 3D modeling data generated after the medical image photographing apparatus is photographed in a specific posture. That is, a medical image photographing device (for example, a CT device) can perform 3D rendering based on medical image data to generate 3D modeling data and transmit the data to a computer.

Further, in another embodiment, the computer generates 3D modeling data by applying a correction algorithm for changing from medical image data photographed under general photographing conditions to a specific physical state at the time of surgery. That is, the computer generates 3D modeling data by applying a body change based on gravity or a body change formed by an undulation model in accordance with gravity-fed body change or carbon dioxide injection in a state inclined to medical image data of a patient lying in a normal state.

For example, the correction algorithm can be calculated by learning big data formed by matching medical image data on general imaging conditions and physical condition with physical condition data during surgery of a specific physical condition and surgical conditions. The physical condition data at the time of the surgery may include image data of a body internal organ shape or arrangement at the time of surgery or image data of a body surface.

In one embodiment, when the laparoscopic surgery or robotic surgery relief model is formed based on medical image data (for example, CT data) photographed under a general condition, the computer displays the medical image data of the patient and the body The surface data is matched and constructed as big data to be learned. The body surface data is taken by the medical staff before surgery.

For example, a medical practitioner takes body surface data by capturing a deformed abdominal surface as it is injected with carbon dioxide prior to performing laparoscopic surgery or robotic surgery. As the computer learns the big data, it creates a correction criterion or correction algorithm that forms the generalized 3D modeling data as a relief model in laparoscopic surgery or robotic surgery. Thereafter, when the medical image data of the new patient is acquired, the computer generates the undulation model by modifying the 3D modeling data generated based on the medical image data based on the modification reference or modification algorithm.

Through this, the computer can generate 3D modeling data similar to the body condition at the time of operation for a specific patient, and provide it to the user.

The computer obtains virtual surgery data for the 3D modeling data from the user (S400). The user performs a simulation or rehearsal of the virtual surgery on the 3D modeling image. For example, a computer may display a virtual surgical tool on a 3D modeling image, and a user may perform a rehearsal on the 3D modeling image by controlling a virtual surgical tool in various ways.

The virtual surgery data is generated as the user simulates the operation on the virtual body model of the 3D modeled patient. The process of acquiring virtual surgery data may be a process of simulating the operation by the user or a process of rehearsal under the same condition immediately before surgery. The computer can receive virtual surgery data from the user in various ways.

In one embodiment, a computer may provide a virtual body model to a user via a Virtual Reality (VR) device, and may enter a surgical operation for a virtual body model via a controller. The controller may be of various forms capable of recognizing the movement of the user's hand. For example, since the controller is implemented in a glove shape, it is possible to obtain detailed finger movements and a real-time arrangement state of a hand at the time of a user's operation.

Specifically, the computer provides a virtual body model to the HMD device worn by the user. The computer rotates or magnifies the virtual body model according to the user's operation. Accordingly, the user performs the operation plan and the simulation while closely examining the virtual body model implemented in the same manner as the operation scheduled for the patient.

For example, vessels and organs that require actual surgery due to fat such as omentum (curtain) may not be seen depending on the body characteristics of the subject, but in modeling, omentum is removed to accurately identify the location and shape of actual blood vessels and organs So that an optimal surgical plan can be established to reach an object requiring actual surgery. The computer obtains an input from which a particular surgical tool is selected by the user and acquires an operation to perform surgery on a specific area of the virtual body model with the surgical tool through the controller.

In addition, in one embodiment, the virtual surgery data may be stored as image data that implements a surgical operation performed by the user on the virtual body model. The computer can generate a simulation image based on real-time motion data obtained by the controller, the surgical tool performing the motion, the position in the virtual body model in which the motion is performed, and the like.

Further, in another embodiment, the virtual surgery data may be modeling data for the virtual body model itself, all the surgical tools and motion data used at each point in the simulation or rehearsal process. That is, the virtual surgery data may be a plurality of sets of data that an external computer can receive and implement a simulation image.

The computer generates the virtual surgery data and the queue sheet data including one or more detailed surgery operations (S600). That is, the computer generates a virtual surgical procedure performed on a virtual body model by a user as an image, and divides the virtual surgery process into one or more detailed operation operations to generate a cue sheet representing a surgical process.

The detailed surgery operation constituting the cue chart data is a minimum operation unit constituting the surgical process. The detailed surgical operation can be divided by several criteria. For example, the detailed surgical operation may be performed in a variety of ways, including the type of surgery (e.g., laparoscopic surgery, robotic surgery), the anatomical body part undergoing surgery, the surgical tool used, the number of surgical tools, Position, movement of the surgical tool (e.g., forward / retract), and the like.

The subcategories included in the segmentation criterion and the segmentation criterion can be set directly by learning the actual operation data of the medical staff. The computer can perform the supervised learning according to the division criteria and the subcategories set by the medical staff to divide the virtual surgery data into the minimum unit detailed operation.

Further, the subcategories included in the segmentation criterion and the segmentation criterion can be extracted through the surgical image learning of the computer. For example, the computer can calculate the division criterion and the category within each division criterion by deep learning learning (i.e., non-coaching learning) of the actual operation data accumulated in the big data. Thereafter, the computer divides the virtual surgery data according to the division criterion generated through the actual operation data learning to generate the cue sheet data.

In another embodiment, the virtual surgery data or the actual surgery data may be segmented by determining whether the virtual surgery data or the actual surgery data corresponds to each division criterion through image recognition. That is, the computer recognizes the anatomical long-term position corresponding to the segmentation criterion, the surgical tool appearing, the number of each surgical tool, and the like in the image of the virtual surgery data or the actual operation data, .

In another embodiment, the computer may perform a segmentation process for generating the cue sheet data based on the surgical tool motion data included in the actual surgery data or the virtual surgery data. The actual operation data may include various information inputted in the process of controlling the surgical robot such as the type and number of the surgical tools selected by the user and information about the motion of each surgical tool when the user performs the robot surgery , And the virtual surgery data may also include information on the type and number of surgical tools selected by the user at the time of simulation for the virtual body model, and the motion of each surgical tool. Thus, the computer can perform segmentation based on the actual surgical data or the information contained at each time point of the virtual surgery data.

In addition, in one embodiment, the virtual surgery data or the actual operation data includes various kinds of actions such as ablation and suture, and segmentation is performed according to the division criterion. More specifically, a process of generating actual surgical data (for example, an actual surgical image) actually performed on stomach cancer surgery or virtual surgery data simulating stomach cancer surgery by dividing the virtual surgery data into detailed surgery operations and generating it as a cue chart data will be described below.

For example, stomach cancer surgery includes an operation to ablate some or all of the stomach, including a tumor, and an operation to ablate the lymph nodes. In addition, various resection and connection techniques are used depending on the condition of the stomach. Further, each action can be divided into a plurality of more detailed actions according to the specific position at which the action is taken and the direction of movement of the surgical tool.

For example, the detailed operation of stomach cancer surgery can be divided into a laparotomy, a resectional step, a connection step, and a suturing step.

In addition, the method of connecting the resected organs includes an ex vivo anastomosis method of incising and connecting 4-5 cm or more of the gut tip, or a method of incision in the abdominal cavity by abdominal incision and anastomosis in the body cavity. May be further divided according to such a specific connection method.

Further, each surgical method can be divided into a plurality of more detailed surgical operations according to the position and movement of the surgical tool.

Each of the detailed surgical operations divided may be given a standardized name based on the position at which the detailed operation is performed and the pathway of the surgical tool.

In one embodiment, the terms used in the standardized nomenclature may be variously defined. For example, when dealing with a lower right specific region of a stomach, the name of the site may be a name that is commonly used in the medical field, and a more comprehensive or subdivided name, as defined in the system according to the disclosed embodiment, Can be used.

Therefore, the rehearsal image can be organized into information of a cue sheet type in which a plurality of actions are sequentially listed based on the standardized names. Likewise, the operation image in which the user actually performed surgery can also be divided into action units and can be organized into cue sheet type information.

Also, in one embodiment, the cue chart data may be generated as code data of a certain number of digits according to a criterion for dividing into detailed surgery operations. That is, the computer divides the virtual surgery data into standardized detailed surgery operations by applying the standardized division criteria and specifying the subcategories within the division criterion, assigns standardized code values to the respective subcategories, Standardized code data that can be used.

The computer assigns numbers or letters to the detailed operation data in order from the upper category to which the specific detailed operation belongs in accordance with the dividing reference application sequence. Accordingly, the computer can generate the cue sheet data in a form in which the standardized code data of each detailed operation operation is listed instead of the divided detailed operation operation image. In addition, the user can share or transmit the simulated surgical procedure by providing only the cue chart data composed of the standardized code data.

Further, in one embodiment, the computer may assign a standardized name to a standardized code of each detailed surgery operation. Through this, the user can selectively check only the desired operation (or action) portion within the entire cuesheet. In this case, the user can easily grasp the progress of surgery or rehearsal even if he or she sees a sequenced sequence chart sequentially based on the standardized name of each action, without viewing the rehearsal or the surgical image at all.

The cue chart data can be converted into a surgical image using an image database for each detailed operation. The image database may store an image corresponding to each code data, and a plurality of images corresponding to each code data may be stored according to a situation. For example, specific code data may be loaded with different detailed operation motion images in the image database according to previously performed operations.

Further, as each of the cue chart data is stored and matched with a specific virtual body model, the computer can reproduce the surgical simulation image by sequentially applying each detailed operation included in the cue sheet data to the virtual body model.

Thus, the image corresponding to the cue sheet may be reproduced at the same point as the rehearsal image, or may be reconstructed and reproduced at another point in time. Alternatively, the image may be modeled in 3D, and the viewpoint and position may be adjusted according to the user's operation.

In addition, as shown in FIG. 4, the method further includes determining (S800) whether the computer optimizes the trial queue data generated based on the user's virtual surgery data.

The computer judges the appropriateness of the trial queue data by comparing the trial queue data with the optimized queue data. For example, the computer may be configured to perform a detailed surgery operation that may include unnecessary detail surgery operations that delay the operation time within the generated trial sequence data as the user simulates, And judges whether or not it is generated appropriately for the actual patient of the trial queue data.

The computer can perform the process of calculating optimized queue data to perform evaluation of trial queue data.

5 is a flow chart illustrating a method of optimizing a surgical procedure according to an embodiment.

The computer acquires one or more learning queue data (S820). The learning queue data is learning target data learned for calculating optimized queue data. The learning cue chart data may include cue chart data generated based on actual surgery data (i.e., actual surgery cue chart data) or cue chart data generated based on simulated virtual surgery data (i.e., reference virtual cue scan data) . The actual surgery queue chart data is generated by dividing the actual operation data by a computer according to a division criterion. The reference virtual surgery queue data is generated not by the user in the surgical training process but by simulation for the purpose of constructing the learning data or providing the reference data to the practitioners.

Thereafter, the computer performs reinforcement learning using the learning queue data (S840). Reinforcement learning is an area of machine learning, in which an agent defined in an environment recognizes the current state and selects a behavior or sequence of actions that maximizes compensation among selectable behaviors. Reinforcement learning can be summarized as learning how to maximize compensation based on state transitions and compensation based on state transitions.

Thereafter, the computer calculates optimized queue data using the enhanced learning result (S860). The optimized queue data is calculated on the basis of the shortest operation time, the minimum amount of blood loss, the required action group, and the mandatory execution order that can reduce the anesthetic time of the patient based on the reinforcement learning result.

The mandatory group of operations is a group of detailed surgical operations that must be performed together essentially to perform a specific detailed surgical operation. The essential procedure is a sequence of operations that must be performed sequentially in the course of performing a specific operation. For example, the surgical operations to be sequentially appeared depending on the type of operation or the type of operation, and the order may be determined.

The computer also calculates optimized context queue data according to the patient's physical condition, surgical site (e.g., tumor tissue) conditions (e.g., tumor size, location, etc.) through reinforcement learning. To do this, the computer uses the patient condition and the surgical site condition together with the learning queue data at the time of learning.

In one embodiment, the computer can perform a simulated virtual surgery on its own. For example, the computer may generate a surgical procedure according to the type of surgery and the type of patient based on the disclosed surgical process optimization method, and may perform a virtual surgery simulation based on the generated surgical procedure.

The computer can acquire the optimized surgical process by evaluating the virtual surgical simulation result and performing reinforcement learning based on the virtual surgical simulation information and the evaluation information on the result.

Even if the model is learned so as to create the optimal surgical process, since the body structure and the type of operation are different for each patient in the actual operation, it may be difficult to generate an optimized surgical process depending on each patient and the type of operation.

Therefore, the computer uses the learned model to create a surgical process based on the patient's body structure and the type of surgery, performs reinforcement learning by performing virtual surgery simulation, and optimizes the surgical process according to each patient and type of surgery Can be generated.

In addition, as shown in FIG. 4, in another embodiment, the computer (S1000) of providing surgical guide data based on the specific queue chart data at the request of the user (S1000). That is, the computer can perform simulation or rehearsal during the operation to provide the generated queue sheet data to the user.

As described above, the surgical simulation information generating method according to an embodiment of the present invention may be implemented as a program (or an application) to be executed in combination with a hardware computer and stored in a medium.

The above-described program may be stored in a computer-readable medium such as C, C ++, JAVA, machine language, or the like that can be read by the processor (CPU) of the computer through the device interface of the computer, And may include a code encoded in a computer language of the computer. Such code may include a functional code related to a function or the like that defines necessary functions for executing the above methods, and includes a control code related to an execution procedure necessary for the processor of the computer to execute the functions in a predetermined procedure can do. Further, such code may further include memory reference related code as to whether the additional information or media needed to cause the processor of the computer to execute the functions should be referred to at any location (address) of the internal or external memory of the computer have. Also, when the processor of the computer needs to communicate with any other computer or server that is remote to execute the functions, the code may be communicated to any other computer or server remotely using the communication module of the computer A communication-related code for determining whether to communicate, what information or media should be transmitted or received during communication, and the like.

The medium to be stored is not a medium for storing data for a short time such as a register, a cache, a memory, etc., but means a medium that semi-permanently stores data and is capable of being read by a device. Specifically, examples of the medium to be stored include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like, but are not limited thereto. That is, the program may be stored in various recording media on various servers to which the computer can access, or on various recording media on the user's computer. In addition, the medium may be distributed to a network-connected computer system so that computer-readable codes may be stored in a distributed manner.

The steps of a method or algorithm described in connection with the embodiments of the present invention may be embodied directly in hardware, in software modules executed in hardware, or in a combination of both. The software module may be a random access memory (RAM), a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a hard disk, a removable disk, a CD- May reside in any form of computer readable recording medium known in the art to which the invention pertains.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, You will understand. Therefore, it should be understood that the above-described embodiments are illustrative in all aspects and not restrictive.

10: Medical imaging equipment
20: Server
30:
32: Display
34: Surgical robot
36:

Claims (10)

The computer generating 3D modeling data including a surgical site of a subject;
Obtaining virtual surgery data for the 3D modeling data; And
Using the virtual surgical data to generate queue sheet data comprising one or more detailed surgical operations; And generating a surgical simulation information. The method according to claim 1,
Wherein the step of generating the cue chart data comprises:
The virtual surgical data is divided into the one or more detailed surgical operations, wherein the virtual surgery data includes at least one of a surgical site, a type of surgical tool, a number of the surgical tool, a position of the surgical tool, Dividing the virtual surgical data into the one or more detailed surgical operations based on at least one of a movement of the tool; And generating a surgical simulation information. The method according to claim 1,
Wherein the obtaining of the 3D modeling data comprises:
Obtaining a medical image including a surgical site of the object; And
Generating the 3D modeling data using the medical image; And generating a surgical simulation information. The method of claim 3,
Wherein the acquiring of the medical image comprises:
Obtaining a photographing posture of the object based on an actual posture; And
Acquiring the medical image photographed based on the obtained photographing posture; And generating a surgical simulation information. The method of claim 3,
Wherein the step of generating the 3D modeling data comprises:
Obtaining an actual posture of the object; And
Generating the 3D modeling data by correcting the obtained medical image based on the actual posture; And generating a surgical simulation information. The method according to claim 1,
Determining whether the generated cue chart data is optimized; Further comprising the steps < RTI ID = 0.0 > of: < / RTI > The method according to claim 6,
Wherein the step of determining whether to optimize comprises:
Providing surgical guide data based on the cue chart data determined to be optimized; Further comprising the steps < RTI ID = 0.0 > of: < / RTI > The method according to claim 6,
Wherein the step of determining whether to optimize comprises:
Obtaining optimization queue data; And
Comparing the cue chart data with the optimized cue chart data; And generating a surgical simulation information. 9. The method of claim 8,
Wherein acquiring the optimized queue data comprises:
Acquiring at least one learning queue data;
Performing reinforcement learning using the at least one learning queue data; And
Acquiring the optimized queue data based on the enhanced learning result; And generating a surgical simulation information. A computer program stored in a computer readable recording medium in combination with a computer which is hardware and which is capable of performing the method of claim 1. Description:

Images