KR20190099999A - Method, apparatus and program for constructing surgical simulation information - Google Patents

Abstract

Provided is a method for establishing surgical simulation information performed by a computer. The method comprises steps of: acquiring a pre-generated virtual body model based on medical image data of an object; acquiring actual surgical data photographed during actual surgery on the subject; extracting texture information of the subject from the actual surgical data; and reflecting the extracted texture information to the virtual body model.

Description

METHOD, APPARATUS AND PROGRAM FOR CONSTRUCTING SURGICAL SIMULATION INFORMATION}

The present invention relates to a method, apparatus and program for constructing surgical simulation information.

In the surgical procedure, there is a need for the development of technologies that can provide information to assist the surgeon's surgery. In order to provide information to assist the operation, the operation should be recognizable.

Conventionally, in order to design a scenario for optimizing a surgical process, a previously taken medical image or a highly skilled doctor was consulted. However, the medical image alone was difficult to judge unnecessary processes. There was a difficult problem to consult.

Therefore, medical imaging and the advice of an experienced doctor were often difficult to be used as an aid for the optimization of the surgical process for the patient.

Thus, by optimizing the surgical process by minimizing unnecessary processes in performing surgery using a three-dimensional medical image (for example, a virtual image of the internal movement caused by the movement of the three-dimensional surgical instruments and tools) And, there is a need for the development of a method that can provide surgical assistance information based on this.

The problem to be solved by the present invention is to provide a method, apparatus and program for the construction of surgical simulation information.

Problems to be solved by the present invention are not limited to the above-mentioned problems, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

According to an embodiment of the present invention, a method for constructing surgery simulation information performed by a computer may include obtaining a parasitic virtual body model based on medical image data of an object, and performing an actual surgery taken during an actual operation on the object. Acquiring data, extracting texture information of the object from the actual surgery data, and reflecting the extracted texture information to the virtual body model.

In an embodiment of the present disclosure, the extracting of the texture information may extract texture information of the surgical region of the object in real time from the surgical region image of the object obtained based on the actual surgical data. .

In one embodiment of the present invention, the step of reflecting on the virtual body model, the step of obtaining a corresponding area corresponding to the surgical site image from the virtual image of the object in the virtual body model, and the corresponding area The method may include mapping texture information about a surgical site of the object.

In an embodiment of the present disclosure, the acquiring of the corresponding region may include matching camera position information of the virtual image with camera position information of the surgical site image, and based on the camera position information. And matching the virtual image with the surgical site image to obtain location information of the corresponding region.

The method may further include obtaining a standardized template image of the object, and matching the template image to a virtual image of an object in the virtual body model. The reflecting on the virtual body model may reflect the texture information based on the matched virtual image.

In one embodiment of the present invention, the method may further include reconstructing the virtual body model by reflecting the texture information.

In an embodiment of the present disclosure, the method may further include providing surgical information of the subject in real time during the actual surgery, compared to the actual surgery data based on the reconstructed virtual body model.

In one embodiment of the present invention, the virtual body model may be 3D modeling data generated based on medical image data of the surgical site of the subject.

In an embodiment of the present disclosure, the extracting of the texture information may include at least one of information representing surface color of the object, information representing surface roughness and height, and information representing surface reflectivity. Can be extracted.

In one embodiment of the present invention, the actual surgery data, may be obtained by a surgical camera to enter the surgical site of the subject during the actual surgery.

Surgical simulation information building apparatus according to an embodiment of the present invention includes a memory for storing one or more instructions, and a processor for executing the one or more instructions stored in the memory, the processor by executing the one or more instructions, Acquiring a parasitic virtual body model based on the medical image data of the object, acquiring the actual surgical data photographed during the actual operation of the object, and texture of the object from the actual surgical data Extracting information, and reflecting the extracted texture information to the virtual body model.

A computer program according to an embodiment of the present invention is combined with a computer, which is hardware, and stored in a computer-readable recording medium to perform the method for constructing surgical simulation information.

According to the present invention, real surgery information is obtained in real time during surgery, and based on this, by constructing the same surgery simulation information as the actual surgery information of each patient, it is possible to provide an optimized patient surgery information to the doctor during surgery.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a view showing a robot surgery system according to an embodiment of the present invention.
2 is a flowchart illustrating a method of constructing surgical simulation information according to an embodiment of the present invention.
3 is a flowchart illustrating a process of reflecting texture information on an actual surgical part of an object in a virtual body model according to an exemplary embodiment of the present invention.
4 is a diagram illustrating an example of applying a method for constructing surgical simulation information according to an embodiment of the present invention.
5 is a diagram schematically showing the configuration of an apparatus 300 for building surgical simulation information according to an embodiment of the present invention.

Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but can be embodied in various different forms, and the present embodiments only make the disclosure of the present invention complete, and those of ordinary skill in the art to which the present invention belongs. It is provided to fully inform the skilled worker of the scope of the invention, which is defined only by the scope of the claims.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, "comprises" and / or "comprising" does not exclude the presence or addition of one or more other components in addition to the mentioned components. Like reference numerals refer to like elements throughout, and "and / or" includes each and all combinations of one or more of the mentioned components. Although "first", "second", etc. are used to describe various components, these components are of course not limited by these terms. These terms are only used to distinguish one component from another. Therefore, of course, the first component mentioned below may be a second component within the technical spirit of the present invention.

Unless otherwise defined, all terms used in the present specification (including technical and scientific terms) may be used in a sense that can be commonly understood by those skilled in the art. In addition, terms that are defined in a commonly used dictionary are not ideally or excessively interpreted unless they are specifically defined clearly.

As used herein, the term "part" or "module" refers to a hardware component such as software, FPGA, or ASIC, and the "part" or "module" plays certain roles. However, "part" or "module" is not meant to be limited to software or hardware. The “unit” or “module” may be configured to be in an addressable storage medium or may be configured to play one or more processors. Thus, as an example, a "part" or "module" may include components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, Procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. Functions provided within components and "parts" or "modules" may be combined into smaller numbers of components and "parts" or "modules" or into additional components and "parts" or "modules". Can be further separated.

As used herein, “image” may mean multidimensional data composed of discrete image elements (eg, pixels in a 2D image and voxels in a 3D image). For example, the image may include a medical image of the object obtained by the CT imaging apparatus.

As used herein, an "object" may be a person or an animal, or part or all of a person or an animal. For example, the subject may include at least one of organs such as the liver, heart, uterus, brain, breast, abdomen, and blood vessels.

As used herein, a "user" may be a doctor, a nurse, a clinical pathologist, a medical imaging professional, or the like, and may be a technician who repairs a medical device, but is not limited thereto.

In the present specification, "medical image data" is a medical image photographed by a medical imaging apparatus, and includes all medical images that can be implemented as a three-dimensional model of the body of an object. "Medical image data" may include computed tomography (CT) images, magnetic resonance imaging (MRI), positron emission tomography (PET) images, and the like.

As used herein, the term "virtual body model" refers to a model generated according to the actual patient's body based on medical image data. The "virtual body model" may be generated by modeling medical image data in three dimensions as it is, or may be corrected as in actual surgery after modeling.

As used herein, "virtual surgery data" refers to data including rehearsal or simulation actions performed on a virtual body model. The "virtual surgery data" may be image data for which rehearsal or simulation has been performed on the virtual body model in the virtual space, or may be data recorded for a surgical operation performed on the virtual body model.

As used herein, the term "actual surgery data" refers to data obtained by performing a surgery by an actual medical staff. "Real surgery data" may be image data photographing the surgical site in the actual surgical procedure, or may be data recorded for the surgical operation performed in the actual surgical procedure.

As used herein, the term "computer" includes all the various devices capable of performing arithmetic processing to provide a result to a user. For example, a computer can be a desktop PC, a notebook, as well as a smartphone, a tablet PC, a cellular phone, a PCS phone (Personal Communication Service phone), synchronous / asynchronous The mobile terminal of the International Mobile Telecommunication-2000 (IMT-2000), a Palm Personal Computer (PC), a Personal Digital Assistant (PDA), and the like may also be applicable. In addition, when a head mounted display (HMD) device includes a computing function, the HMD device may be a computer. Also, the computer may correspond to a server that receives a request from a client and performs information processing.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a view showing a robot surgery system according to an embodiment of the present invention.

1, there is shown a schematic diagram of a system capable of performing robotic surgery in accordance with one embodiment of the present invention.

According to FIG. 1, the robotic surgical system includes a medical imaging apparatus 10, a server 20, and a control unit 30 provided in an operating room, an imaging unit 36, a display 32, and a surgical robot 34. do.

In one embodiment, the robot surgery is performed by the user controlling the surgical robot 34 using the control unit 30. In one embodiment, the robot surgery may be automatically performed by the controller 30 without the user's control.

The server 20 is a computing device including at least one processor and a communication unit.

The controller 30 includes a computing device including at least one processor and a communication unit. In one embodiment, the control unit 30 includes hardware and software interfaces for controlling the surgical robot 34.

The image capturing unit 36 includes at least one image sensor. That is, the image capturing unit 36 includes at least one camera device and is used to photograph the surgical site. In one embodiment, the imaging unit 36 is used in conjunction with the surgical robot 34. For example, the image capturing unit 36 may include at least one camera coupled with a surgical arm of the surgical robot 34.

In an embodiment, the image photographed by the image capturing unit 36 is displayed on the display 340.

The controller 30 receives information necessary for surgery from the server 20 or generates information necessary for surgery and provides the information to the user. For example, the controller 30 displays the information necessary for surgery, generated or received, on the display 32.

For example, the user performs the robot surgery by controlling the movement of the surgical robot 34 by manipulating the control unit 30 while looking at the display 32.

The server 20 generates information necessary for robotic surgery using medical image data of a subject (patient) previously photographed from the medical image photographing apparatus 10, and provides the generated information to the controller 30.

The control unit 30 provides the user with the information received from the server 20 on the display 32 or controls the surgical robot 34 using the information received from the server 20.

In one embodiment, the means that can be used in the medical imaging apparatus 10 is not limited, for example, other various medical image acquisition means such as CT, X-Ray, PET, MRI may be used.

Hereinafter, a method for constructing surgical simulation information will be described in detail with reference to the drawings.

2 is a flowchart illustrating a method of constructing surgical simulation information according to an embodiment of the present invention.

Each step shown in FIG. 2 is performed in time series in the server 20 or the controller 30 shown in FIG. 1. Hereinafter, for the convenience of description, each step is described as being performed by a computer, but the performing agent of each step is not limited to a specific device, and all or part thereof may be performed by the server 20 or the controller 30. Can be.

Referring to FIG. 2, in the method for constructing surgery simulation information according to an embodiment of the present invention, the computer acquires a parasitic virtual body model based on medical image data of the object (S100), and actually Acquiring real surgery data photographed at the time of operation (S110), extracting texture information on the object from the actual surgery data (S120), and reflecting the extracted texture information to the virtual body model It includes a step (S130). Hereinafter, a detailed description of each step will be described.

The computer may acquire a parasitic virtual body model based on the medical image data of the object (S100). Here, the subject refers to the body of the patient, and may be a body organ or a blood vessel. For example, the subject may include the liver, heart, uterus, brain, breast, abdomen, and the like.

In an embodiment, the computer may acquire medical image data of the object photographed from the medical imaging apparatus 10 such as CT, MRI, PET, and the like, and generate a virtual body model based on the acquired medical image data in advance. have. However, the virtual body model generated based on the medical image data accurately represents the appearance of the object (eg, an organ), but the surface features (eg, texture features such as color, material, and texture) of the object ) Is not expressed. Therefore, it is necessary to provide information reflecting the actual surface characteristics of the object to a virtual body model previously generated based on the medical image data.

Thus, during the actual surgery of the patient, the computer may first obtain a parasitic virtual body model based on the medical image data. As described above, the virtual body model may be 3D modeling data generated based on medical image data of all or a part of a body organ (ie, an object) that is a target of a surgical site of a patient.

Thereafter, the computer may acquire the actual surgical data photographed during the actual surgery on the subject (S110). Here, the actual surgery data is data obtained by performing a surgery by the medical staff, and may be image data photographing the actual surgical region of the object.

In one embodiment, when performing minimally invasive surgery, such as laparoscopic surgery, robotic surgery, surgery is performed by inserting only a surgical tool and a camera in the surgical site of the patient. In this case, the computer may acquire, as the actual surgical data, an image captured by a camera entering the patient's body. For example, the actual surgical data may be taken of the process of moving from the patient's inside of the body to the surgical part of the body, and from the start of the action by the surgical tool to the end of the body. It may be image data.

The computer may extract surface feature information of the object, that is, texture information such as color, material, and texture represented on the surface of the object from the actual surgery data (S120), and extract the extracted texture information from the parasitic acquired in step S100. Can be reflected in the virtual body model (S130).

In detail, the computer may acquire a virtual image of the object in the virtual body model, and obtain an image of the actual surgical site of the object from the actual surgery data. Thereafter, the computer may acquire a corresponding region corresponding to the actual surgical region image from the virtual image, and map texture information of the actual surgical region of the object to the corresponding region. Here, the virtual image may mean an image acquired at the same point of time as viewed by an actual camera in a virtual body model constructed in three dimensions.

In one embodiment, the camera entering the patient's body during surgery is to shoot all the objects in the direction the camera looks. That is, the camera records the surgical scene from the start of the action by the surgical tool to the end of the specific surgical site of the patient. The computer may select image data of a desired time point from among image data (actual surgery data) captured by a camera entering the patient's body during surgery, and reflect the image data to the virtual body model using the selected image data.

For example, if several medical staff practice the same surgery, the virtual body model should be implemented in the same manner as the actual patient's operation. To this end, the computer may generate a virtual body model using the actual surgical image data obtained at the beginning of the total actual surgical data. For example, a virtual body model at the initial stage of surgery may be generated by reflecting texture, fat placement, initial organ state, etc. from the actual surgical image data acquired at the beginning. On the other hand, if a reoperation is required after performing the first operation, the medical staff may perform a surgery simulation using a virtual body model in advance before the reoperation. In this case, the computer may configure the virtual body model by reflecting the actual surgical image data obtained later in the entire actual surgical data. Subsequently, when the medical staff simulates the reoperation, virtual surgery data may be generated by performing a process of following the actual surgical operation in the virtual body model.

3 is a flowchart illustrating a process of reflecting texture information on an actual surgical part of an object in a virtual body model according to an exemplary embodiment of the present invention.

In one embodiment, as shown in Figure 3, the computer is to match the camera position information between the virtual image of the virtual body model and the surgical site image of the actual surgical data in order to make the rendering of the virtual body model and the actual surgical data the same. It may be (S131).

For example, the position of the virtual camera of the virtual image and the position of a camera (eg, a camera included in a surgical tool such as an endoscope) of the actual surgical site image may be matched.

In this way, by matching the camera position information between the two images, the position between the objects in the 2D or 3D, when mapping from one space to another space can be equally converted.

The computer may match the virtual image and the actual surgical site image based on the location information between the cameras (S132). That is, the computer may allow the part of the object included in the virtual image corresponding to the surgical site to be matched with the surgical site based on the location information between the cameras.

For example, the computer may extract a portion corresponding to the object in the virtual image. In addition, the computer may adjust the enlargement and reduction of the portion extracted from the virtual image to correspond to the surgical region included in the actual surgical region image. Through such adjustment, an object of the virtual image corresponding to the surgical region of the actual surgical region image may be matched. Accordingly, a corresponding region corresponding to the surgical site may be obtained from the object of the virtual image.

The computer may acquire the location information of the corresponding region by detecting a corresponding region in the virtual image corresponding to the surgical region from the actual surgical region image through matching between the images (S133).

For example, the computer may derive the position information of the corresponding area in the virtual image based on the conversion relationship between the camera position information on the actual surgical site image and the camera position information on the virtual image. In this case, location information of the corresponding area may be calculated using a perspective value of the camera.

The computer may extract the surgical site from the actual surgical site image, and acquire texture information about the surgical site (S134).

The texture information may include information representing a surface color of an object (for example, a diffuse map), information representing a surface roughness and a height of an object (for example, a normal map and a height map), and information representing a surface reflectivity of an object (for example, and occlusion maps. In addition to the normalized image-based texture information, a shader capable of generating a texture in real time through calculation may be applied. For example, applying a shader can generate changes in real time on an object's surface, such as all pixels, vertices, texture locations, hue, saturation, brightness, contrast, etc. that are used to compose an image. Texture information can be calculated using information on normal, vertical) values, general surface properties such as transparency, reflectivity, and object color, and the type, number, and direction of lighting.

The computer may map texture information extracted from the actual surgical site image to the virtual image based on the location information of the corresponding region in the virtual image (S135).

For example, the computer may extract the surgical region including texture information from the actual surgical region image, and mount the extracted region on the surface of the object based on the position information of the corresponding region in the virtual image.

When the above-described process of FIG. 3 is repeatedly executed by moving the position of the camera, texture information of the entire object can be extracted and mapped to the corresponding object, thereby obtaining a virtual body model identical to the actual organ of the patient.

Meanwhile, according to an embodiment of the present invention, the computer may acquire a standardized template image of the object. The standardized template image of the object may be image data in which a template of each object is constructed as a 3D model by standardizing anatomical features such as appearance, size, and position of each object. In this case, the computer may acquire a standardized template image and match it with the parasitic virtual body model obtained in step S100.

For example, the computer may extract the missing portion from the virtual image including the object in the virtual body model. The computer may receive a standardized template image of the object from a database in which the standardized template image is previously stored and stored, and correct the missing portion in the virtual image based on the received standardized template image. Through this, it is possible to provide a virtual body model such as the appearance of the actual object more.

Thereafter, the computer may perform a process of reflecting the texture information as described above using the virtual body model that is matched with the standardized template image.

In addition, according to an embodiment of the present invention, the computer may reconstruct the virtual body model by reflecting the texture information in step S130.

In addition, according to an embodiment of the present invention, the computer may provide surgical information of the subject in comparison with the actual surgery data in real time during the actual surgery based on the reconstructed virtual body model.

That is, since the reconstructed virtual body model maps texture information extracted from the actual surgical data of the patient under actual surgery, the reconstructed virtual body model includes the same information as the actual surgical site of the patient. Therefore, if the surgery is performed using the virtual body model reconstructed during the actual surgery, the virtual image can be obtained in the same visual aspect as the actual surgery image, so that the location of the actual surgical site can be specified or additional information related to the surgery (e.g., Location, direction, movement, etc. of the surgical instruments) can be effective.

4 is a diagram illustrating an example of applying a method for constructing surgical simulation information according to an embodiment of the present invention.

Referring to FIG. 4, the computer may acquire a parasitic virtual body model 102 based on the medical image data 100 of the object (S200). In addition, the computer may acquire the standardized template image 112 for the object from the database 110 in which the standardized template image is previously constructed and stored (S210). In addition, the computer may acquire the actual surgical data 120 photographing the surgical site of the patient during the actual operation (S220).

The computer then corrects for the missing portion or correction in the virtual body model 102 through comparison with the virtual body model 102 (ie, the 3D modeled virtual image including the subject of the patient) based on the standardized template image 112. This necessary part can be detected. Accordingly, the computer may match the missing part of the virtual body model 102 or the part requiring correction using the standardized template image 112. Therefore, the computer may derive the matched virtual body model (S230).

Next, the computer may detect a region corresponding to the actual surgical region from the actual surgery data 120, and extract texture information of the detected actual surgical region (S240).

Next, the computer may acquire a corresponding region corresponding to the detected actual surgical region from the matched virtual body model, and map texture information of the actual surgical region to the acquired corresponding region (S250). Therefore, the virtual body model 130 having the same texture information as the actual surgical region of the patient may be finally obtained. That is, the finally obtained virtual body model 130 is reconstructed by reflecting the texture information of the actual surgical site on the parasitic virtual body model 102 generated based on the medical image data 100 of the object.

The process (steps S240 to S250) of mapping texture information about the actual surgical site of the patient to the matched virtual body model may be performed by applying the method of FIG. 3.

As described above, the present invention uses the same information (actual surgery data, virtual body model, standardized template image) about the surgical site of the patient during the actual surgery to build the surgical simulation information to the same information as the actual surgical site of the patient Can be provided in real time during surgery.

However, in the case of the conventional 3D organ model constructed using images obtained from CT, MRI, PET, etc., the appearance of the organ is relatively accurate, but the texture information of the surface such as the color and material of the organ is not represented. There is no way to know before you operate the organ. Therefore, conventionally, a method of applying a color, a material, or the like commonly used for each organ, referring to a picture book, or obtaining a texture of color, material, etc., from an operation image of another patient has been used. In the case of a surgical simulation model constructed in this way, it is difficult to visually determine the organs of the same patient because surface features are expressed differently from those of the patient in the actual surgical image.

In order to improve such a conventional surgical simulation model, as described above, in the present invention, the texture information of the patient organ extracted from the image data during the actual surgery is mapped to a virtual body model previously constructed for a specific patient in real time. to provide. That is, the actual surgical image of the patient and the surgical simulation (that is, the virtual body model) can be made identical in terms of visual. Through this visual identity, it is effective to compare the surgical image and the surgical simulation image to the surgeon during the operation and to obtain the information necessary for the operation (for example, the location of the surgical site). In addition, since the postoperative surgical simulation model can reproduce the same organ image as the actual surgical image, it is possible to express all the procedures of the surgery (eg, endoscope or surgical tool movement) as if the actual surgery is in progress. Can be. Therefore, the surgical simulation model in the present invention can be used not only in real time during the operation, but also try to reoperate later, or can be used as educational material. For example, if the physics engine is applied to an organ in a surgical simulation model, it can be visually reproduced in the same manner as the actual patient's organ when attempting a reoperation.

5 is a diagram schematically showing the configuration of an apparatus 300 for building surgical simulation information according to an embodiment of the present invention.

Referring to FIG. 5, the processor 310 may include a connection passage (for example, a bus or the like) that transmits and receives a signal with one or more cores (not shown) and a graphic processor (not shown) and / or other components. ) May be included.

The processor 310 according to an embodiment executes one or more instructions stored in the memory 320 to perform the method of constructing surgical simulation information described with reference to FIGS. 2 to 4.

For example, the processor 310 obtains a parasitic virtual body model based on the medical image data of the object by executing one or more instructions stored in the memory 320, and performs the actual surgery taken during the actual surgery on the object. Data may be obtained, texture information of the object may be extracted from the actual surgical data, and the extracted texture information may be reflected in the virtual body model.

Meanwhile, the processor 310 may read random access memory (RAM) and read-only memory (ROM) for temporarily and / or permanently storing a signal (or data) processed in the processor 310. , Not shown) may be further included. In addition, the processor 310 may be implemented in the form of a system on chip (SoC) including at least one of a graphic processor, a RAM, and a ROM.

The memory 320 may store programs (one or more instructions) for processing and controlling the processor 310. Programs stored in the memory 320 may be divided into a plurality of modules according to their functions.

The method for constructing surgical simulation information according to an embodiment of the present invention described above may be implemented as a program (or an application) and stored in a medium to be executed in combination with a computer which is hardware.

The above-described program includes C, C ++, JAVA, machine language, etc. which can be read by the computer's processor (CPU) through the computer's device interface so that the computer reads the program and executes the methods implemented as the program. Code may be coded in the computer language of. Such code may include functional code associated with a function or the like that defines the necessary functions for executing the methods, and includes control procedures related to execution procedures necessary for the computer's processor to execute the functions according to a predetermined procedure. can do. In addition, the code may further include memory reference code for additional information or media required for the computer's processor to execute the functions at which location (address address) of the computer's internal or external memory should be referenced. have. Also, if the processor of the computer needs to communicate with any other computer or server remotely in order to execute the functions, the code may be used to communicate with any other computer or server remotely using the communication module of the computer. It may further include a communication related code for whether to communicate, what information or media should be transmitted and received during communication.

The stored medium is not a medium for storing data for a short time such as a register, a cache, a memory, but semi-permanently, and means a medium that can be read by the device. Specifically, examples of the storage medium include, but are not limited to, a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. That is, the program may be stored in various recording media on various servers to which the computer can access or various recording media on the computer of the user. The media may also be distributed over network coupled computer systems so that the computer readable code is stored in a distributed fashion.

The steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, in a software module executed by hardware, or by a combination thereof. Software modules may include random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any form of computer readable recording medium well known in the art.

In the above, embodiments of the present invention have been described with reference to the accompanying drawings, but those skilled in the art to which the present invention pertains may realize the present invention in other specific forms without changing the technical spirit or essential features thereof. I can understand that. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive.

10: medical imaging equipment
20: server
30: control unit
32: display
34: surgical robot
36: video recording unit
300: surgical simulation information building device
310: processor
320: memory

Claims (12)

In the computer-assisted surgical simulation information construction method,
Obtaining a parasitic virtual body model based on medical image data of the object;
Acquiring the actual surgical data photographed during the actual surgery on the subject;
Extracting texture information of the object from the actual surgery data; And
Surgical simulation information construction method comprising the step of reflecting the extracted texture information to the virtual body model. The method of claim 1,
Extracting the texture information,
Surgical simulation information construction method, characterized in that for extracting the texture information on the surgical site of the subject in real time from the surgical site image of the subject obtained based on the actual surgery data. The method of claim 2,
Reflecting in the virtual body model,
Obtaining a corresponding area corresponding to the surgical site image from the virtual image of the object in the virtual body model; And
And mapping the texture information of the surgical site of the object to the corresponding area. The method of claim 3,
Acquiring the corresponding area,
Matching camera position information of the virtual image with camera position information of the surgical site image; And
And acquiring position information of the corresponding region by matching the virtual image with the surgical region image based on the camera position information. The method of claim 1,
Obtaining a standardized template image of the object; And
And matching the template image to a virtual image of an object in the virtual body model.
Reflecting in the virtual body model,
Surgical simulation information construction method, characterized in that for reflecting the texture information based on the matched virtual image. The method of claim 1,
And reconstructing the virtual body model by reflecting the texture information. The method of claim 6,
And providing surgical information of the subject in real time during the actual surgery in comparison with the actual surgery data based on the reconstructed virtual body model. The method of claim 1,
The virtual body model,
Surgical simulation information construction method characterized in that the 3D modeling data generated based on the medical image data of the surgical site of the subject. The method of claim 1,
Extracting the texture information,
And at least one of information representing surface color of the object, information representing surface roughness and height, and information representing surface reflectivity. The method of claim 1,
The actual surgery data,
Surgical simulation information building method, characterized in that obtained by the surgical camera to enter the surgical site of the subject during the actual surgery. Memory for storing one or more instructions; And
A processor for executing the one or more instructions stored in the memory,
The processor executes the one or more instructions,
Obtaining a parasitic virtual body model based on medical image data of the object;
Acquiring the actual surgical data photographed during the actual surgery on the subject;
Extracting texture information of the object from the actual surgery data; And
And applying the extracted texture information to the virtual body model. A computer program, coupled to a computer, which is hardware, stored on a recording medium readable by a computer so as to perform the method of claim 1.

Images