KR101208191B1 - Method and apparatus of simulating incision treatment in computer-based virtual surgery training environment and computer-readable recording medium thereof - Google Patents

Abstract

A method and apparatus for simulating incisional treatment in a computer-based virtual medical training environment is disclosed. Incision treatment simulation method is a three-dimensional wound model display step of displaying a three-dimensional affected model consisting of a plurality of polygons in the virtual space, and displays a user tool model operated by the user in the virtual space, the user input received through the input unit User tool manipulation step of manipulating the user tool model in the virtual space based on the method. When the user tool model contacts the 3D affected model in the virtual space, the 3D affected model is determined according to the incision pattern determined using a predetermined decision technique. And a feedback step of generating at least one feedback of visual, tactile, and auditory and providing the feedback to the user through the output unit while the division step of splitting and the cutting step are performed. According to the present invention, it is possible to reproduce the cutting process performed in the virtual space close to the actual, and can increase the training effect of the user through multi-modal feedback.

Description

Method and apparatus of simulating incision treatment in computer-based virtual surgery training environment and computer-readable recording medium

TECHNICAL FIELD The present invention relates to a technique for providing a computer-based virtual medical training environment, and more particularly, to a simulation technique for enabling surgical treatment technology training including incision in a computer-based virtual medical training environment.

Thanks to the advancement of modern medicine, the various medical treatments practiced by doctors are becoming increasingly refined, and even more delicate procedures are being developed that can not be performed by the naked eye and the hands of doctors. Thus, while a surgeon needs to be highly skilled, a surgeon can not achieve this proficiency with limited clinical experience and experience.

Therefore, one way to make medical practice training more effective is to use computer simulations. For example, before the procedure, the medical image of the patient is reconstructed in three dimensions to establish the surgical plan, and the simulation plan is applied to the simulation environment to conduct the simulation training. The physician performing the training performs the necessary treatment through the input device simulating the surgical tool and prepares for the actual operation based on the output result.

As such, contemporary computer engineering makes it possible to provide realistic training situations in a virtual environment created by a computer program. That is, a three-dimensional model of the object related to the simulation is provided by the computer. As such, many complex processes in the surgical field are not impossible to fully simulate on a computer, but this is a very expensive technique. In addition, the computer-based training environment developed to solve these problems is also limited in training range and slow commercialization due to lack of accuracy and presence.

In addition, known techniques for providing reliable simulation are very complex and costly in terms of computer utility in the form of processors and memory. Furthermore, the prior art is not sufficient to provide a simulation environment where the results are feasible. In practice, the visual characteristics of anatomy are difficult and time-consuming to reproduce the simulation.

Therefore, in order to make up for the shortcomings of the conventional computer-based incision simulation technique and to apply it to a wide range of practical training, a more immersive virtual environment and an accurate interaction technique for the training target are urgently required.

[1] Special 2003-0083695 "Simulation methods and systems for surgical treatment" [2] 10-2009-0043513 "Computerized Medical Training Systems"

[3] Turkiyyah, G., Karam, WB, Ajami, Z., and Nasri, A., "Mesh cutting during real-time physical simulation," in SPM '09: 2009 SIAM / ACM Joint Conference on Geometric and Physical Modeling , 159 ?? 168 (2009).

SUMMARY OF THE INVENTION An object of the present invention is to provide an incision treatment simulation method for calculating a curvature of a real progress path in real time in cutting a 3D model based on a user input and using the same for model segmentation to more accurately represent an incision path. will be.

It is also an object of the present invention to provide an incision treatment simulation apparatus that can provide a more immersive training environment by mathematically analyzing and reproducing visual, auditory, and tactile feedback that may occur in an actual incision process.

One aspect of the present invention for achieving the above objects relates to a method for simulating an incision procedure in a computer-based virtual medical training environment. According to an embodiment of the present invention, a method for simulating incision treatment may include displaying a three-dimensional wound model displaying a three-dimensional wound model including a plurality of polygons in a virtual space, and displaying a user tool model manipulated by a user in the virtual space. A user tool manipulation step of manipulating the user tool model in the virtual space based on the user input received through the input unit; and a predetermined decision technique when the user tool model contacts the 3D affected model in the virtual space. An incision step of dividing the three-dimensional affected model according to the incision pattern determined by using and a feedback of generating at least one feedback of sight, tactile, and hearing while the incision step is performed and providing the feedback to the user through the output unit Characterized in that it comprises a step. In particular, the display of the three-dimensional affected model may include displaying candidate affected models stored in a training scenario database, allowing the user to select an affected lesion model suitable for a desired medical training environment among the candidate affected models through an input unit, and Reading the selected lesion model from the training scenario database and displaying it in the virtual space. The user tool manipulation step may include extracting motion information of at least one of three degrees of freedom (3DOF) and six degrees of freedom (6DOF) of the user tool model from the user input and extracting the extracted motion information. Converting the user tool model into position and state information in the virtual space, and updating the position and state of the user tool model in the virtual space according to the position and state information. Further, the cutting step may include determining whether the user tool model is in contact with the three-dimensional affected model in the virtual space, determining whether to start the cutting operation when in contact, and when the cutting operation is initiated, the three-dimensional Determining an entry point, which is the point where each polygon included in the affected model and the user tool model meet, and an exit point where the user tool model leaves the polygon, an entry point on the movement path of the user tool model, and A turning point determination step of determining a turning point located between the entry points, and a straight line connecting the entry point and the entry point via the turning point as an incision path, and all included in the three-dimensional wound model along the incision path. And dividing the polygon. In particular, the feedback step provides as a tactile feedback at least one of a vertical reaction force that makes a boundary to the affected area, a resistive force acting in the opposite direction to the incision direction, and a torque indicative of a grip between the user tool and the incision surface. And an auditory feedback step of providing the auditory effect as an auditory feedback that may occur in a tactile feedback step or an incision situation.

Another aspect of the present invention for achieving the above objects relates to an apparatus for simulating an incision procedure in a computer-based virtual medical training environment. An incision treatment simulation apparatus according to another aspect of the present invention is a central processing unit for implementing a virtual medical training environment in a virtual space, a three-dimensional lesion model implemented in a virtual space and consisting of a plurality of polygons, and a user manipulated by a user. A display for displaying a tool model, and an input for receiving user input for manipulating the user tool model in the virtual space and passing the input to the central processing unit, wherein the central processing unit contacts the three-dimensional affected model with the user tool model in the virtual space. A collision detection unit for detecting whether a contact is detected, an incision processing unit for dividing the three-dimensional affected model according to an incision pattern determined using a predetermined decision technique when a contact is detected, and at least one of visual, tactile, and auditory while the incision operation is performed. Generate one feedback and user through output It includes a feedback generator for providing to. In particular, the incision processor determines whether to initiate the incision operation when the user tool model contacts the three-dimensional affected model in the virtual space, and when the incision operation is initiated, each polygon and the user tool included in the three-dimensional affected model. The entry point, where the model meets, and the action of determining the exit point from which the user tool model leaves the polygon, and the turning point located between the entry point and the exit point on the user tool model's movement path. A transition point determination operation for determining the transition point, and a straight line connecting the entry point and the exit point via the transition point, are regarded as an incision path, and are adapted to perform a division operation for dividing all polygons included in the three-dimensional wound model along the incision path. It is characterized by. Furthermore, the incision processor divides the moving path in the polygon of the user tool model into a predetermined number of sections to perform the turning point determination operation, and uses the starting point, midpoint, and end point of each divided section to curvature the corresponding section. And an operation of determining an intermediate point of a section in which the calculated curvature becomes a threshold or more using a predetermined criterion, as the switching point. In addition, in order to perform the split operation, the cutting processing unit duplicates the current vertex on the cutting path and updates the geometry information of the current vertex by referring to a finite state machine (FSM) in which an abnormal element according to the polygon type is excluded. The operation of determining the dividing direction by calculating two vector cross products according to the order of the incision direction vector, which is the direction of the path, and the normal vector of the current vertex, and the geometry information of both sides of the incision path of the three-dimensional wound model according to the dividing directions, respectively. It is characterized in that it is adapted to perform the segmentation simulation operation which represents the updating operation and the incision path as an incision line, and represents the incision surface which divided | segmented the 3D affected model according to the updated geometry information according to the incision line. Preferably, the incision processor is adapted to further perform the operation of further modifying the geometry information to reflect the physical characteristics of the three-dimensional affected model for the realistic incision plane representation to perform the segmentation simulation operation. Alternatively, the cut processing unit may further change the geometry information by reflecting a visualization element including at least one of a texture, a particle, and a shader to implement a physical effect occurring at the cut surface in order to perform a split simulation operation. And adapted to perform. Further, the feedback generating unit may include at least one of a vertical reaction force for feeling a boundary to the affected area, a resistance force acting in a direction opposite to the incision direction, and a torque representing a locking phenomenon between the user tool and the incision surface as tactile feedback. Or as auditory feedback for auditory effects that may occur in an incision situation.

According to the present invention, the cutting process performed in the virtual space can be reproduced close to the actual one, and this advantage is maximized, especially in a model expressed in low resolution.

In addition, according to the present invention, the training effect of the user can be increased through the multi-modal feedback accompanying the incision.

Furthermore, although most of the prior arts focused on minimally invasive surgery, the present invention is applicable to various medical training environments by suggesting a mathematical technique that is independent of the simulation environment or the type of procedure.

1 is a flow diagram conceptually illustrating a method for simulating an incision procedure in a computer-based virtual medical training environment in accordance with an aspect of the present invention.
2 is a diagram illustrating an entry point, a turning point, and an exit point on a moving path of a user tool model.
Figure 3a shows the definition of curvature used in the present invention.
3B is a diagram for explaining a method of determining a turning point according to the present invention.
4 is a diagram for describing a process of determining an incision path according to the presence of a turning point.
5 is a diagram for explaining a method of determining a split direction based on a cutting path.
6 is a block diagram conceptually illustrating one embodiment of an apparatus for simulating an incision treatment in a computer-based virtual medical training environment in accordance with another aspect of the present invention.

In order to fully understand the present invention, operational advantages of the present invention, and objects achieved by the practice of the present invention, reference should be made to the accompanying drawings and the accompanying drawings which illustrate preferred embodiments of the present invention.

Hereinafter, the present invention will be described in detail with reference to the preferred embodiments of the present invention with reference to the accompanying drawings. However, the present invention can be implemented in various different forms, and is not limited to the embodiments described. In order to clearly describe the present invention, parts that are not related to the description are omitted, and the same reference numerals in the drawings denote the same members.

Throughout the specification, when a part is said to "include" a certain component, it means that it may further include other components, without excluding the other components unless otherwise stated. In addition, the terms "... unit", "... unit", "module", "block", etc. described in the specification mean a unit for processing at least one function or operation, which means hardware, software, or hardware. And software.

1 is a flow diagram conceptually illustrating a method for simulating an incision procedure in a computer-based virtual medical training environment in accordance with an aspect of the present invention.

According to the present invention, a user performs an incision training by operating an input device, and when a contact with a surgical tool manipulated by a user occurs, the user enters an incision state and performs an incision and division process on a three-dimensional affected model. . In this process, the curvature information of the actual incision path is utilized to assist the path determination to faithfully reproduce the incision path. It also provides immersion into the training environment by providing visual, auditory, and tactile feedback that accompanies the incision.

Hereinafter, the present invention will be described in detail with reference to FIG. 1.

First, in operation S110, a 3D affected area model including a plurality of polygons and a user tool model manipulated by a user are displayed in a computer-based virtual space. In the present specification, the virtual space refers to a virtual space displayed on a display monitor by a computer, and the user visually recognizes the three-dimensional affected model displayed on the display monitor and uses the user tool model on the virtual space. By direct manipulation, you can experience the same as performing various treatments yourself. In this case, various lesions and various training scenarios may be stored in a training scenario database. Then, the user can directly select a desired one of the training scenarios and the lesions stored in the database through the user manipulation unit. As such, using a database in which various training scenarios are stored, various treatment trainings can be performed using the same device. When a new procedure is developed, maintenance costs are reduced because only the database needs to be updated.

When the 3D affected model and the user tool model are displayed in the virtual space, the user tool model is manipulated based on the user input received through the input unit (S120).

In order to manipulate the user tool model, motion information of at least one degree of freedom of the user tool model is obtained from the user input. In one embodiment of the present invention, three degrees of freedom (3DOF) or six degrees of freedom (6DOF) may be obtained. 6 Degrees of freedom refers to all motion elements used in robotics or virtual reality systems, i.e. X (horizontal), Y (vertical), Z (depth), pitch, yaw, roll To indicate. In addition, three degrees of freedom show only X, Y, and Z.

When motion information based on 3 degrees of freedom or 6 degrees of freedom is extracted as described above, the extracted motion information is converted into position and state information of the user tool model in the virtual space. This conversion operation may be implemented by a central processing unit that implements a virtual space, or may be performed by a separate information conversion device.

The central processing unit then updates the position and state of the user tool model in the virtual space according to the translated position and state information. Through this process, the user can confirm that the user tool model moves in the virtual space as he manipulates and commands. The input unit manipulated by the user may have a joystick shape or may be the same as the user tool model. For example, a user performing a cutting process using a scalpel may manipulate a user tool model using an input unit having the same shape as the scalpel. In this case, of course, the input unit having a shape like a scalpel should include a sensor for detecting a three degrees of freedom or a six degrees of freedom input.

Then, it is detected whether the user tool model moving in the virtual space is in contact with the 3D affected model (S130). If the user tool model is in contact with the three-dimensional affected model, this means that an incision can be initiated. However, contact does not necessarily initiate the incision procedure and may require a separate trigger action. For example, when performing an incision treatment using a laser beam, even if the beam irradiator is in contact with the 3D affected model, the incision does not start unless the beam is irradiated. Also, even if one blade of the scalpel is in contact with the three-dimensional affected model, it must overlap with the other blade of the scalpel to initiate incision. Therefore, in consideration of these points, it is determined whether the cutting operation is started (S140).

Once the incision is initiated, the three-dimensional lesion model is segmented according to the incision pattern determined using a predetermined decision technique. This process is described in detail as follows.

That is, when the incision is initiated, an entry point, which is a point where each polygon included in the 3D affected model and the user tool model meet, and an exit point where the user tool model leaves the polygon are first determined (S150). . Then, a turning point located between the entry point and the exit point on the moving path of the user tool model is determined.

In the present specification, the turning point refers to a vertex that should be used as a reference point for the incision simulation because a lot of change of direction occurs on the moving path of the user tool model.

2 is a diagram illustrating an entry point, a turning point, and an exit point on a moving path of a user tool model.

Referring to FIG. 2, FIG. 2 illustrates a division process for a 3D model having a triangle as a basic unit. An entry point is defined as an entry point where a moving path starts or enters into a current triangle from an adjacent triangle, and an exit point where a moving path ends or exits to an adjacent triangle. If the direction of movement changes within the current triangle, this point is defined as the turning point. The three-dimensional model in the virtual space is expressed by cutting individual triangles along a straight path connecting the entry point and the entry point defined above or connecting the entry point via the intermediate point at the entry point.

In order to express the incision path more accurately in the incision simulation process, it is important to select the positions of the entry point, the exit point and the turning point. Of these, the positions of the entry and exit points are clearly determined, but the presence and position of the turning point may vary depending on the determination method. It can be seen that the result of the incision along the B arrow is performed much more precisely than the result of the incision along the arrow A in FIG. 2. In the present invention, the problem of determining the turning point is solved through the method of estimating the curvature of each section.

That is, in order to determine the turning point on the incision path, the moving path in the polygon of the user tool model is divided into a predetermined number of sections, and the curvature of the corresponding section is calculated using the start point, the middle point and the end point of each divided section ( S160).

Figure 3a shows the definition of curvature used in the present invention.

When any three vertices exist on the movement path, the closer the line segment formed by the three vertices is to a straight line, the closer to 1 the curvature becomes. On the contrary, as the change in the direction increases, the curvature increases. Using this, the moving path is sampled at a predetermined distance, the curvature of each sampled section is estimated, and the vertex located at the section showing the maximum curvature is set as the switching point (S170).

In FIG. 3A, the curvature is defined as in Equation 1 below.

In Equation 1, V _start represents a start point, V _end represents an end point, and V _center represents an intermediate point. d _start is V _start and V _center The straight line between the d _end is the V _center And a straight line between V _end and d _shortest indicates the shortest distance between V _start and V _end .

As can be seen from Equation 1, the closer the movement path is to the shortest distance, the smaller the curvature.

3B is a diagram for explaining a method of determining a turning point according to the present invention.

Referring to FIG. 3B, after dividing the moving path into a predetermined number of sections, the curvature of each section is calculated by referring to Equation 2 below.

In Equation 2, v _k represents a k-th vertex on a moving path, and d _k represents a straight line connecting v _k and v _{k +1} . In addition, s _k represents a straight line connecting v _k and v _{k +2} , and Curvature _k represents the curvature of the k-th section.

Various methods may be used to determine the turning point in FIG. 3B. For example, one of the predetermined number of intermediate points may be determined as the turning point. That is, one for every two vertices can be determined as a turning point. Using this method, the turning point can always be determined regardless of the trajectory of the moving path, which has the advantage of determining an incision path close to the moving path. In the present specification, the midpoint refers to vertices on a moving path, and the turning point refers to vertices selected to form an incision path among the midpoints.

Alternatively, if the maximum value of the curvature measured in a particular polygon is lower than a preset threshold, the incision path may be regarded as a straight line without considering the vertex as a turning point. In this way, the moving path is regarded as the shortest distance unless the moving path is significantly separated from the shortest distance. Therefore, this increases the overall speed of the simulation process by reducing unnecessary polygon splitting, and enables more accurate representation of the incision path than the conventional method.

4 is a diagram for describing a process of determining an incision path according to the presence of a turning point.

In FIG. 4, the dotted line represents the movement path of the user tool model, and the straight line represents the incision path. In the present specification, a moving path refers to an actual path in which a user moves a tool while a cutting state is maintained, and an decision path refers to a straight line connecting vertices determined as turning points in the moving path. When the moving path is a straight line, the moving path is the same as the incision path. Referring to FIG. 4, it can be seen that the turning point exists, and the more the cut-off point determination method reflects the geometric characteristics of the actual moving path, the closer the cutting path is to the moving path.

That is, the left figure of FIG. 4 shows that the incision path is determined without a turning point. Here, it can be seen that the travel path is completely ignored. In addition, the center diagram of FIG. 4 shows that one turning point is determined using the average of the coordinates of all intermediate points existing on the moving path, and the cut path is selected using the determined turning point. Finally, the right figure of FIG. 4 shows that the vertex having the maximum curvature is selected as the turning point, and the cutting path is determined using the vertex. It can be seen that the incision path approximates the movement path toward the right side of FIG. 4. If we define more turning points on the moving path, it can be seen that the incision path determined using this turning point will be closer to the moving path.

As such, when the turning point is determined, the straight line connecting the entry point and the exit point via the turning point is regarded as a cutting path, and all polygons included in the 3D wound model are divided along the cutting path (S180).

When the polygon is divided along the determined incision path, a finite-state machine or similar decision making method is used. The polygon splitting pattern is determined depending on whether the entry point and the exit point are located among the vertex, the edge, and the face of the polygon, respectively. At this time, depending on the type of polygon (triangle, rectangle, or other polygon) and the partitioning method, abnormal elements such as T-point intersections may occur. Therefore, a finite state machine that excludes them is built in the database and referred to when splitting.

As the mock incision proceeds, various feedbacks associated with the incision are calculated every frame and transmitted to the user through the output device while the incision state is maintained. First, visual feedback according to the incision includes grooves and incision planes generated along the incision path. To create gaps and incisions, the vertices are duplicated in the left and right directions based on the current vertices, and the associated geometry information is updated. At this time, the cross product of the incision direction vector and the normal vector of the current vertex is used to divide the model by dividing the left and right sides based on the incision path.

5 is a diagram for explaining a method of determining a split direction based on a cutting path.

Referring to FIG. 5, first, an incision direction vector a constituting an incision path and a normal vector b at a current vertex are determined. Then, the division direction is determined by calculating the cross product of two vectors in different order. In other words, the split direction coincides with the directions of the vectors a * b and b * a , respectively.

During segmentation, a physically-based deformation formula may optionally be applied in defining the geometry information of the affected model for more realistic incision surface representation. In addition, visualization elements such as textures, particles, and shaders may be additionally used to express effects such as bleeding.

In addition, visual, tactile, and auditory feedback is provided to the user while the segmentation is performed to increase the sensation of the cutting training in the virtual space (S190).

Among them, the tactile feedback is a vertical reaction force that prevents penetration to the affected area and allows a feeling of boundary, a resistance force acting in a direction opposite to the progression direction during incision, and a torque for reproducing a jam between the tool and the incision surface ( torque). In addition, auditory and other feedback that may occur in the actual incision situation is provided in accordance with the training progress. This feedback element can be added or subtracted depending on the purpose of the training and the configuration of the system.

Also, the auditory feedback may include a sound generated when the 3D affected model is cut. The present invention encompasses a technique of cutting a three-dimensional affected model implemented on a virtual medical training environment based on input from a user as described above, and generating various feedbacks accompanying the user.

According to the present invention, the cutting process performed in the virtual space can be reproduced close to the actual one, and this advantage is maximized, especially in a model expressed in low resolution. In addition, according to the present invention, the training effect of the user can be increased through the multi-modal feedback accompanying the incision.

If the feedback is provided, it is determined whether the incision is terminated (S195). If it is determined that the incision is not finished, the process returns to step S120 of manipulating the user tool model according to the user input.

6 is a block diagram conceptually illustrating one embodiment of an apparatus for simulating an incision treatment in a computer-based virtual medical training environment in accordance with another aspect of the present invention.

The incision treatment simulation apparatus 600 shown in FIG. 6 includes an input unit 610, an output unit 690, and a central processing unit 650. Also, the central processing unit 650 may include an input information converter 620, a collision detector 630, an incision processor 640, a virtual space controller 660, a feedback generator 670, and a training scenario database 680. It includes.

The simulation technique presented in the present invention presupposes operation in a system consisting of a computer-based processing device and one or more input / output devices. The dissection treatment simulation apparatus 600 manages the medical training scenario, the 3D model and other resources required through the training scenario database 680, and generates a virtual reality environment based on the training scenario database 680. The user controls the virtual reality environment using the input unit 610 and performs an incision training.

The collision detector 630 checks whether a collision exists between the user tool model manipulated by the user and the 3D affected model, and provides the determination result to the cutting processor 640. The incision processor 640 determines whether the incision operation is started by combining the determination result of the collision detection unit 630 and the presence or absence of the incision initiation operation, and when initiation, the incision processing for the affected model is performed.

While the incision is being performed, the output unit 690 provides the user with various information according to the training. This information reproduces visual, auditory, and tactile feedback that may occur in the actual incision process, and the feedback is generated by a series of mathematical processes by the feedback generator 670.

Referring to the operation of the input unit 610 in detail, the user's motion can be detected from any type of input device capable of sensing and digitizing the motion in the three-dimensional space. The input device is more suited for training, but is not form dependent, designed to have a similar appearance and operating structure as the surgical instrument. The information obtained from the input device basically includes three degrees of freedom or six degrees of freedom motion information depending on the type of the device, and additional motion information may be included depending on the type of surgical tool to be trained. For example, in the case of a scalpel, the 6 degree of freedom exercise information can express the current state, but in the case of scissors, the degree of freedom of the 1 degree of freedom exercise information expressing the degree of the blade is additionally necessary for basic 6 degree of freedom exercise information.

The input information converting unit 620 converts the position and state of the surgical tool obtained through the input device according to the coordinate system of the virtual environment in which the training is in progress. Then, the virtual space control unit 660 updates the position and state of the tool displayed in the virtual environment every frame based on this. Then, the collision inspecting unit 630 inspects the updated tool region and the region of the three-dimensional model to be trained every frame and considers that the contact between the tool and the affected part occurs when a collision occurs between the two regions. The incision state remains until the contact of the tool with the affected area is resolved. Some surgical tools require not only contact with the affected area, but also incisions that involve the operation of certain triggers. When simulating these tools, only while both the contact state and the trigger return true Keep incision.

The finite state machine (FSM) referenced by the dissection processing unit 640 has generally been used as a selection tool for imparting intelligence to a controlled object. In other words, a finite state machine is a model that represents a device or a device that transitions from one state to another, or causes an output or action to occur, depending on the input given, with a finite number of states that can be encountered at any given time. FSM has the advantages of being fast, easy to code, easy to correct errors, and intuitive and flexible. For example, a simple example of a finite state machine might be a light switch in a house, which is lit when the switch is turned on and is maintained. If someone comes off the switch, the light goes out. Likewise, the light will remain off until someone switches it on. In this way, the finite state machine maintains a state, and when a specific event occurs, the ecology changes to a state that meets a condition. In the present invention, various cutting situations are pre-stored using the finite state machine, and extracted from the training scenario database 680 to fit the current state and processed.

In addition, the feedback generator 670 may additionally use visualization elements such as textures, particles, and shaders to express effects such as bleeding. In the present specification, the texture is a visual tool used to more accurately express the deformation light, such as the color and shape of the incision surface of the three-dimensional affected model, the particle is a realistic representation of the particles that can be generated during the incision Is a visual tool. In addition, a visual tool called a shader may be used to give a realistic three-dimensional effect or shading of the folded portion or the like while the incision is performed.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art.

Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

As such, the present invention can be widely applied to a safe and economical medical training method. In addition, the simulation technique presented in the present invention is not limited to the medical field, but may be applied to virtual reality in general, and may be applied for interaction with the virtual environment, particularly in the fields of virtual assembly and manufacturing training, science education, and entertainment.

Claims (23)

A method for simulating incision treatment in a computer based virtual medical training environment,
A three-dimensional affected model display step of displaying a three-dimensional affected model composed of a plurality of polygons in a virtual space;
A user tool manipulation step of displaying a user tool model manipulated by a user in the virtual space and manipulating the user tool model in the virtual space based on a user input received through an input unit;
An incision step of dividing the three-dimensional affected model according to an incision pattern determined using a predetermined decision technique when the user tool model contacts the three-dimensional affected model in the virtual space; And
A feedback step of generating at least one feedback of sight, tactile, and hearing while providing the user through an output unit while the cutting step is performed;
The cutting step,
Determining whether the user tool model contacts the three-dimensional affected model in the virtual space;
If contacted, determining whether to initiate the cutting operation;
When the cutting operation is started, an entry point, which is a point where each polygon included in the 3D affected model and the user tool model meet, and an exit point where the user tool model leaves the polygon are determined. Making;
A turning point determining step of determining a turning point located between the entry point and the exit point on the movement path of the user tool model; And
And a dividing step that considers the straight line connecting the entry point and the advance point via the turning point as the cutting path and divides all the polygons included in the three-dimensional wound model along the cutting path. Method for simulating incision treatment in a computer based virtual medical training environment. The method of claim 1, wherein the displaying of the 3D affected model is performed by:
Displaying candidate affected models stored in a training scenario database;
Allowing a user to select a wound model suitable for a desired medical training environment among the candidate affected models through the input unit; And
A method for simulating an incision procedure in a computer-based virtual medical training environment, comprising: reading a user-selected lesion model from the training scenario database and displaying it in the virtual space. The method of claim 1, wherein the user tool manipulation step,
Extracting, from the user input, motion information based on at least one degree of freedom (1DOF) of the user tool model;
Converting the extracted exercise information into position and state information of the user tool model in the virtual space; And
And updating the location and state of the user tool model on the virtual space in accordance with the position and state information. The method of claim 3,
The movement information is movement information based on at least one of three degrees of freedom (3DOF) and six degrees of freedom (6DOF) to simulate the incision treatment in a computer-based virtual medical training environment Way. delete The method of claim 1, wherein the determining of the turning point,
Dividing a moving path in the polygon of the user tool model into a predetermined number of sections;
Calculating curvature of the corresponding section using the divided starting point, midpoint, and end point of each section; And
And determining the midpoint of the section in which the calculated curvature becomes greater than or equal to the threshold using the predetermined criterion as the turning point. The method of claim 6, wherein the dividing step,
Duplicating a current vertex on the incision path and updating geometry information of the current vertex with reference to a finite state machine (FSM) in which an abnormal element according to the type of the polygon is excluded;
Determining a division direction by calculating two vector cross products according to an order of an incision direction vector, which is a direction of the incision path, and a normal vector of a current vertex;
Updating geometry information of both sides of the incision path of the three-dimensional affected model according to the division directions; And
And a segmentation simulation step of considering the incision path as an incision line and expressing an incision surface obtained by dividing the three-dimensional affected model according to the updated geometry information according to the incision line. Method for simulating incision treatment in a patient. The method of claim 7, wherein the segmentation simulation step,
Simulating the incision treatment in a computer-based virtual medical training environment further comprising the step of further modifying the geometry information to reflect the physical behavior of the actual human tissue in the three-dimensional affected model for realistic incision surface representation. Way. The method of claim 7, wherein the segmentation simulation step,
And further modifying the geometry information by reflecting a visualization element including at least one of texture, particle, and shader in order to realize a physical effect occurring at the incision surface. A method for simulating incisional treatment in a computer-based virtual medical training environment. The method of claim 1, wherein the feedback step comprises:
A tactile feedback step of providing as a tactile feedback at least one of a vertical reaction force which makes a boundary to the affected area, a resistive force acting in the opposite direction to the incision direction, and a torque indicative of a grip between the user tool and the incision surface; And a method for simulating an incision treatment in a computer-based virtual medical training environment. The method of claim 1, wherein the feedback step comprises:
A method for simulating incisional treatment in a computer-based virtual medical training environment, comprising an auditory feedback step that provides as auditory feedback an auditory effect that may occur in an incision situation. 12. A computer readable recording medium for recording a program executable by a computer for executing the method according to any one of claims 1 to 4. An apparatus for simulating incision treatment in a computer based virtual medical training environment,
A central processing unit for implementing a virtual medical training environment in a virtual space;
A display configured to be implemented in the virtual space and to display a three-dimensional affected model composed of a plurality of polygons and a user tool model manipulated by a user; And
An input unit configured to receive a user input for manipulating the user tool model in the virtual space and to transmit the user input to the central processing unit;
The central processing unit,
A collision detector configured to detect whether the user tool model contacts the 3D affected model in the virtual space;
An incision processor for dividing the three-dimensional affected model according to an incision pattern determined using a predetermined decision technique when a contact is detected; And
And a feedback generator configured to generate at least one feedback among visual, tactile, and auditory while the cutting operation is performed and provide the feedback to the user through an output unit.
The cutting processing unit,
Determining whether to initiate a cutting operation when the user tool model contacts the 3D affected model in the virtual space;
When the cutting operation is started, an entry point, which is a point where each polygon included in the 3D affected model and the user tool model meet, and an exit point where the user tool model leaves the polygon are determined. Action,
A turning point determining operation of determining a turning point located between the entry point and the exit point on the movement path of the user tool model, and
A straight line connecting the entry point and the entry point via the turning point is regarded as the cutting path, and is adapted to perform a splitting operation for dividing all polygons included in the three-dimensional affected model along the cutting path. Apparatus for simulating an incision treatment in a computer-based virtual medical training environment comprising a. The method of claim 13, wherein the central processing unit,
And display the candidate affected models stored in the training scenario database, and control the display to read out the affected model selected by the user among the displayed candidate affected models from the training scenario database and display it as the three-dimensional affected model. Apparatus for simulating incisional treatment in an on-demand virtual medical training environment. In the thirteenth aspect,
The input unit extracts, from the user input, one piece of motion information based on one or more degrees of freedom of the user tool model to the central processing unit.
The central processing unit converts the extracted exercise information into position and state information of the user tool model in the virtual space, and updates the position and state of the user tool model in the virtual space according to the converted position and state information. And a device for simulating incisional treatment in a computer-based virtual medical training environment. delete The method of claim 13, wherein the cutting processor is further configured to perform the turning point determination operation.
Dividing a moving path in the polygon of the user tool model into a predetermined number of sections,
Calculating the curvature of the corresponding section using the divided starting point, midpoint, and end point of each section, and
The apparatus for simulating an incision treatment in a computer-based virtual medical training environment, characterized in that it is adapted to perform an operation of determining a midpoint of a section in which a calculated curvature is greater than or equal to a threshold as the turning point. The method of claim 17, wherein the cutting processing unit, in order to perform the division operation,
Duplicating the current vertex on the incision path and updating the geometry information of the current vertex with reference to a finite state machine (FSM) in which an abnormal element according to the type of the polygon is excluded,
Determining a division direction by calculating two vector cross products according to an order of an incision direction vector, which is a direction of the incision path, and a normal vector of a current vertex,
Updating geometry information of both sides of the incision path of the three-dimensional affected model according to the division directions, and
Considering the incision path as an incision line, the computer-based virtual medical care is adapted to perform a segmentation simulation operation that represents the incision plane by dividing the three-dimensional affected model according to the updated geometry information according to the incision line. Device for simulating incision treatment in a training environment. The method of claim 18, wherein the cutting processor is further configured to perform the division simulation operation.
Incision treatment in a computer-based virtual medical training environment is adapted to further modify the geometry information by reflecting the physical behavior of real human tissue to the 3D affected model for realistic incision surface representation. Device for simulating. The method of claim 17, wherein the cutting processing unit, in order to perform the division simulation operation,
Computer-based virtual medical care further adapted to further modify the geometry information by reflecting visualization elements including at least one of texture, particle, and shader to realize the physical effects occurring at the incision surface. Device for simulating incision treatment in a training environment. The method of claim 13, wherein the feedback generator,
Providing as tactile feedback at least one of a vertical reaction force which feels a boundary to the affected area, a resistance force acting in a direction opposite to the incision direction, and a torque indicative of a jamming phenomenon between the user tool and the incision surface. Apparatus for simulating incision treatment in a computer-based virtual medical training environment. The method of claim 13, wherein the feedback generator,
An apparatus for simulating incisional treatment in a computer-based virtual medical training environment, characterized by providing auditory feedback that may occur in an incisional situation. A computer-readable recording medium for recording a program executable by a computer for driving the apparatus according to any one of claims 13 to 15 and 17 to 22.

Images