JP5131790B2 - Surgery simulation apparatus, surgery simulation method, and program - Google Patents

Abstract

A device for simulating surgery after acquiring information about a three-dimensional object to which elasticity information is added by using two-dimensional image created by a medical apparatus such as a CT, an MRI, or a PET.  For the surgery simulation, from a first mesh information set and a surgery information set, a second mesh information set corresponding to a deformed shape is acquired.

Description

  The present invention relates to a surgical simulation apparatus and the like that can output an image showing a three-dimensional object or operate an image showing a three-dimensional object.

  It is considered that a surgical process is expressed using a set of medical images (three-dimensional images) of a patient measured by CT, MRI, etc., and used for surgical planning, intraoperative support, clinical education, explanation to a patient, and the like. On the other hand, in a surgical simulation, it is necessary to handle information on the shape (structure), physical characteristics, and surgical content of a human body that is not included in a medical image set, and express it in a form desired by a doctor who is a user of the surgical simulation.

  In response to such a request, conventionally, second mesh information acquisition is performed in which first mesh information of a stored three-dimensional object is deformed and second mesh information is acquired based on a deformation instruction that instructs deformation of the three-dimensional object. A first slice information group that is a plurality of slice information based on the second mesh information and the color information of each point after deformation is determined from the stored 3D voxel information. There has been an information processing apparatus that sets new color information for each point of the first slice information group based on the color information of the point, acquires the second slice information group, and displays the second slice information group. With this information processing apparatus, the change in the shape of the three-dimensional object can be depicted in real time with the color information on the surface and inside (see Patent Document 1).

  In addition, there has been a technique that enables the flexibility and robustness of a living tissue to be expressed by hardness from the CT value in the X-ray CT apparatus (see Patent Document 2). This technique includes input means for inputting measured X-ray absorption values of a substance to be measured, correspondence storage means for storing correspondence relations between X-ray absorption value data and hardness data for a plurality of reference substances, and the input A calculating means for calculating the hardness of the measured substance from the X-ray absorption value of the measured substance using the correspondence relationship, and an output means for outputting the converted hardness of the measured substance It is a material hardness measuring device.

WO 2006/013813 (first page, FIG. 1 etc.) JP 2002-310945 (first page, FIG. 1 etc.)

  However, in the conventional apparatus, it is not possible to appropriately simulate the operation using the operation information that is information specific to the operation.

  Further, in the conventional apparatus, it is necessary to define the shape clearly in a mesh format or the like through extraction of a target region, surface definition, and the like for simulation. It is also necessary to set physical conditions (such as elastic values and boundary conditions), so that the information processing device can manually define the elastic distribution of the living body, express the surgical process, etc. It took time.

  Also, in the medical field, due to the frequency of surgery, etc., it is not possible to prepare data (for example, elasticity information) by manpower and time as in movies, etc., and surgery simulation can be performed quickly. It was strongly desired. Further, it has been desired that doctors can carry out without requiring engineering expertise such as mesh.

  The surgery simulation apparatus according to the first aspect of the present invention is slice information that is information configured based on two-dimensional image data obtained by cutting out 3D voxel information, which is a volume texture of a three-dimensional object, in a plurality of planes. A slice information group storage unit that can store a slice information group having a plurality of slice information composed of information on a plurality of points having position information that is information indicating a position and color information that is information about a color; A first mesh information storage unit that can store first mesh information that is information of a three-dimensional mesh of the original object, a surgery information storage unit that can store surgery information that is information about surgery, and a slice information group storage unit A slice information group output unit for outputting a slice information group, and an instruction for a predetermined point or region of the slice information group output by the slice information group output unit Yes, an instruction reception unit that receives an operation instruction that is an instruction related to an operation, and based on the operation instruction and operation information received by the instruction reception unit, the first mesh information is deformed, and the second mesh information that configures the deformed shape is First slice information group acquisition for acquiring a second slice information group that is a plurality of slice information that does not have color information from the slice information group storage unit based on the second mesh information acquisition unit and the second mesh information to be acquired And a color information determination unit that determines color information of each point corresponding to a point of 3D voxel information, which is each point of a plurality of slice information constituting the first slice information group, and a color information determination unit Based on the color information of each point, second slice information that sets new color information for each point of the first slice information group acquired by the first slice information group acquisition unit and acquires the second slice information group An acquisition unit, a surgical simulation apparatus for and a deformed object output portion that outputs the second slice information group.

  With this configuration, surgery simulation can be performed using surgery information.

  The surgical simulation apparatus according to the second aspect of the present invention is an anatomical structure having human body feature point position information, which is information indicating the structure of the human body and information indicating the position of the human body part. An anatomical information storage unit that can store information; and a mesh information correction unit that corrects the first mesh information using the anatomical information and forms new first mesh information; The mesh information acquisition unit deforms the new first mesh information configured by the mesh information correction unit based on the operation instruction and the operation information received by the instruction reception unit, and acquires the second mesh information constituting the deformed shape. This is a surgical simulation device.

  With this configuration, it is possible to perform highly accurate surgery simulation using surgery information.

  Further, in the surgery simulation apparatus of the third invention, in contrast to the first invention, the surgery information has incision position information which is information indicating the position of the incision, and the second mesh information acquisition unit Based on the surgical instruction received by the instruction receiving unit and the incision position information included in the surgical information, the point in the first mesh information, and the point on the left side of the incision wound among the points corresponding to the incision wound position information is A surgical simulation device that moves to the left side, moves the point on the right side of the incision to the right side, deforms the second mesh information so that the incision opens, and acquires the second mesh information constituting the deformed shape is there.

  With this configuration, it is possible to perform an incision surgery simulation using surgery information.

  Further, in the surgery simulation apparatus of the fourth invention, compared to the first invention, the surgery information has twist position information which is information of a point or a region where a twisting operation is performed, and a second mesh information acquisition unit Is a point in the first mesh information based on the operation instruction received by the instruction receiving unit and the torsion position information included in the operation information, and the second mesh is moved so that the point corresponding to the torsion position information is moved and twisted It is a surgery simulation apparatus which deforms information and acquires the second mesh information which constitutes the deformed shape.

  With this configuration, a surgical simulation of a twisting operation can be performed using the surgical information.

  Further, in the surgery simulation apparatus of the fifth invention, the surgery information has pick position information which is information of a point or an area where a pinch operation is performed, and the second mesh information acquisition unit, relative to the first invention Is a point in the first mesh information based on the operation instruction received by the instruction receiving unit and the operation position information included in the operation information, and the second mesh information is moved to move the point corresponding to the operation position information. It is a surgery simulation device which acquires the 2nd mesh information which changes shape and constitutes the shape which changed.

  With this configuration, it is possible to perform a surgical simulation of a pinch operation using surgical information.

  Further, the surgery simulation apparatus of the sixth aspect of the invention is human body feature point position information which is information indicating the structure of the human body and information indicating the position of the part of the human body with respect to any one of the first to fifth aspects of the invention. An anatomical information storage unit that can store anatomical information, a mesh information correction unit that corrects the first mesh information using the anatomical information, and constitutes new first mesh information; The surgical simulation apparatus further comprising:

  With this configuration, it is possible to appropriately perform a surgery simulation using the first mesh information that matches the subject.

  In addition, the surgery simulation apparatus according to the seventh aspect of the invention adds elasticity information, which is information about elasticity, to each point constituting each piece of slice information with respect to any one of the first to sixth aspects. And an elastic information adding unit that configures a plurality of slice information composed of information of a plurality of points having at least position information and elasticity information, and the slice information group output unit is configured by the elastic information adding unit The slice information group is output, and the instruction reception unit receives an instruction for a predetermined point or region of the slice information group output by the slice information group output unit, and includes one or more points constituting the point or region corresponding to the instruction A position information acquisition unit that acquires position information, an elastic information acquisition unit that acquires one or more pieces of elasticity information paired with one or more pieces of position information acquired by the position information acquisition unit, and an elasticity information acquisition unit A surgical simulation apparatus further comprising an elasticity information output unit for outputting one or more elastic information.

  With this configuration, elasticity information can be automatically added to an image of a three-dimensional object. For this reason, for example, it is possible to save a great deal of time and effort to manually input elasticity information before a surgical simulation.

  Further, the surgical simulation apparatus of the eighth invention is a sampling position for acquiring a plurality of sampling position information having position information constituting a point for sampling a three-dimensional space of a three-dimensional object, relative to the first invention. The first mesh information further comprising an information acquisition unit and stored in the first mesh information storage unit is a surgical simulation device configured by a plurality of sampling position information.

  With this configuration, mesh information necessary for deformation can be automatically acquired. Therefore, for example, it is not necessary to clearly define a three-dimensional shape before the surgery simulation. For example, the surgery simulation can be realized only by providing a medical image set.

  Further, in the surgery simulation apparatus of the ninth aspect of the invention, in contrast to the seventh or eighth aspect of the invention, the elasticity information adding unit uses the CT value of each point constituting a plurality of pieces of slice information to obtain elasticity information. An operation simulation apparatus comprising: elasticity information acquisition means to acquire; and elasticity information addition means for adding the elasticity information acquired by the elasticity information acquisition means to each point constituting each of a plurality of pieces of slice information.

  With this configuration, elasticity information can be automatically acquired using color information (for example, gray value) of each point constituting a plurality of pieces of slice information. Therefore, it is possible to automatically generate elasticity information having a considerable degree of accuracy.

  Further, the surgery simulation apparatus of the tenth aspect of the invention further includes a singular point information receiving unit that receives singular point information that is information indicating one or more singular points, as compared with the seventh aspect, and an elastic information adding unit Further includes a distance acquisition unit that acquires a distance from one or more singular points, and the elasticity information acquisition unit calculates the color information of each point constituting each of the plurality of slice information and the distance acquired by the distance acquisition unit. It is a surgery simulation device that uses it to acquire elasticity information.

  With this configuration, elasticity information can be automatically determined using the distance from a singular point (for example, a point with a tumor). Therefore, for example, a surgical simulation of a patient with a tumor can be performed with high accuracy.

  According to the surgery simulation apparatus of the present invention, appropriate surgery simulation can be performed.

  Hereinafter, embodiments of an information processing apparatus and the like will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.

  (Embodiment 1)

  In the present embodiment, an information processing apparatus capable of automatically adding elasticity information and performing simulation using the elasticity information will be described. In the present embodiment, an information processing apparatus that automatically acquires mesh information will be described. In other words, in this embodiment, before the simulation, the physical phenomenon is expressed on the three-dimensional image by the deformation and distortion of the elastic map and the local space without defining the three-dimensional shape. An information processing apparatus capable of performing surgical simulation will be described. In this embodiment, for example, an information processing apparatus that determines elasticity information using biological information such as a CT value will be described.

  FIG. 1 is a block diagram of an information processing apparatus 1 in the present embodiment.

  The information processing apparatus 1 includes an input / output device 100, a slice information group storage unit 101, a first mesh information storage unit 102, an elastic information addition unit 103, a sampling position information acquisition unit 104, a first mesh information storage unit 105, and an instruction reception unit. 106, slice information group output unit 107, position information acquisition unit 108, elasticity information acquisition unit 109, second mesh information acquisition unit 110, first slice information group acquisition unit 111, color information determination unit 112, second slice information group acquisition Unit 113, elasticity information output unit 114, deformed object output unit 115, object information storage unit 116, and slice information group acquisition unit 117.

  The elasticity information addition unit 103 includes elasticity information acquisition means 1031 and elasticity information addition means 1032.

  The color information determination unit 112 includes corresponding point determination means 1121 and color information determination means 1122.

  The input / output device 100 inputs an instruction for a predetermined point or region with respect to the output slice information group, receives the output of the elasticity information output unit 114, and outputs a force vector corresponding to the output. “Output the corresponding force vector” is realized by motor drive, for example. The input / output device 100 is, for example, a PHANToM (phantom) or a vibrating input device. For example, when the output is a screen output, the input / output device 100 is, for example, a mouse and a display. The input / output device 100 may be composed of two or more devices. Further, the input / output device 100 may be considered to be included or not included in the information processing apparatus 1. In the block diagram of FIG. 1, the information processing apparatus 1 includes an input / output device 100. The instruction for the region is, for example, an instruction for the region including a plurality of points (nodes) using a surgical instrument metaphor indicating an image of a surgical instrument that is a surgical instrument. The surgical instrument metaphor may be image data having scissors or tweezers patterns used in surgery, graphic data having a shape such as a rectangular parallelepiped or a sphere, or three-dimensional image data. The surgical instrument metaphor is not shown. For the surgical instrument metaphor, see, for example, Patent Document 1.

  The object information storage unit 116 can store 3D voxel information that is a volume texture of a three-dimensional object. The 3D voxel information is a set of two-dimensional images acquired by a medical device such as CT, MRI, or PET. The 3D voxel information is, for example, a set of two-dimensional images obtained by photographing the human brain or the inside of the body by CT or MRI. The 3D voxel information is, for example, information on a point constituted by (x, y, z, col, elastic modulus). (X, y, z) of (x, y, z, col, elastic modulus) is coordinate information in a three-dimensional space. “Col” is color information of the point. “Elastic modulus” is a value indicating the elasticity of the point, and is an example of elasticity information. The elastic modulus is, for example, Young's modulus, but may be a rigidity modulus, a volume elastic modulus, or the like. Examples of elasticity information include, for example, Young's modulus, Poisson's ratio, rupture value, friction coefficient, and the like. The point information may include transparency information that is information about transparency such as an alpha value. Here, the 3D voxel information is preferably information on points that are clogged with no interval between points, but may be information on discrete points. The 3D voxel information is (x, y, z, col), and the elasticity information may be held as additional information to the 3D voxel information. In such a case, it may be considered that the 3D voxel information has elasticity information. The object information storage unit 116 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium. Further, CT (Computed Tomography) is a technique that scans an object using radiation or the like (X-rays) and processes it using a computer to construct an internal image of the object. CT includes various examination methods for obtaining cross-sectional images using a computer, such as positron tomography (PET), single photon emission tomography (SPECT), and nuclear magnetic resonance imaging (MRI).

  The slice information group acquisition unit 117 extracts a plurality of slice information that is perpendicular to the line of sight and has a constant interval from the 3D voxel information stored in the object information storage unit 116, acquires a slice information group, At least temporarily stored in the information group storage unit 101. “Perpendicular to the line of sight” means being perpendicular to the line of sight vector, which is a vector perpendicular to the screen on which the three-dimensional object is displayed. Note that the slice information is a set of information of points constituting the plane, and is usually packed with no space between the points. The slice information group acquisition unit 117 can usually be realized by an MPU, a memory, or the like. The processing procedure of the slice information group acquisition unit 117 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The slice information group storage unit 101 can store a slice information group having a plurality of slice information. The slice information is information configured based on two-dimensional image data obtained by cutting out 3D voxel information, which is a volume texture of a three-dimensional object, on a plurality of planes. The slice information includes position information that is information indicating a position and information on a plurality of points that have color information that is information about a color. The slice information preferably includes position information, color information, and elasticity information. Note that the slice information group storage unit 101 may store a slice information group acquired from 3D voxel information stored in the object information storage unit 116. Furthermore, the slice information group acquisition unit 117 may acquire the slice information group.

  The slice information group storage unit 101 is preferably a nonvolatile recording medium, but can also be realized by a volatile recording medium. The process in which slice information is stored in the slice information group storage unit 101 does not matter. For example, slice information may be stored in the slice information group storage unit 101 via a recording medium, and slice information transmitted via a communication line or the like may be stored in the slice information group storage unit 101. Alternatively, the slice information input via the input device may be stored in the slice information group storage unit 101.

  The first mesh information storage unit 102 can store first mesh information that is information about a three-dimensional mesh of a three-dimensional object. The three-dimensional mesh information is a set of information on points constituting the three-dimensional object. The information of the three-dimensional mesh is a set of information on points that are spaced apart. The point information here is usually information having a data structure of (x, y, z, col, elasticity information). However, the point information may be only coordinate information (x, y, z). The first mesh information is preferably composed of a plurality of sampling position information described later. The first mesh information storage unit 102 is preferably a nonvolatile recording medium, but can also be realized by a volatile recording medium. The process in which the first mesh information is stored in the first mesh information storage unit 102 does not matter. For example, the first mesh information may be stored in the first mesh information storage unit 102 via the recording medium, and the first mesh information transmitted via the communication line or the like is stored in the first mesh information storage unit. 102 may be stored, or the first mesh information input through the input device may be stored in the first mesh information storage unit 102.

  The elasticity information adding unit 103 automatically adds elasticity information, which is information about elasticity, to each point constituting each of a plurality of pieces of slice information. The elasticity information adding unit 103 configures a plurality of pieces of slice information including information on a plurality of points having position information, color information, and elasticity information. The elasticity information adding unit 103 may perform processing for accumulating a plurality of configured slice information in the slice information group storage unit 101. The elasticity information adding unit 103 may add elasticity information to each point of the 3D voxel information stored in the object information storage unit 116. Even when the elasticity information adding unit 103 adds elasticity information to each point of the 3D voxel information, it is the same as adding elasticity information, which is information about elasticity, to each point constituting each piece of slice information. I think it is significant. This is because each point constituting each of the plurality of pieces of slice information is also each point of 3D voxel information. The elastic information adding unit 103 can be usually realized by an MPU, a memory, or the like. The processing procedure of the elasticity information adding unit 103 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The elasticity information acquisition unit 1031 acquires elasticity information using color information of each point (may be each point of 3D voxel information) constituting a plurality of pieces of slice information. The color information is, for example, a gray value. For example, according to the CT value (an example of the gray value) that each point of each slice information acquired from the set of two-dimensional images acquired by the CT medical device has, for example, the darker the color, the elastic information acquisition unit 1031 has elasticity information for the darker color The elasticity information is determined so that the elasticity information becomes smaller as the color becomes lighter. The elastic information acquisition unit 1031 can be usually realized by an MPU, a memory, or the like. The processing procedure of the elasticity information acquisition unit 1031 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The elasticity information adding unit 1032 adds the elasticity information acquired by the elasticity information acquiring unit 1031 to each point (may be each point of 3D voxel information) constituting each piece of slice information. For example, the elasticity information adding unit 1032 performs a process of filling “elasticity information” in the structure (x, y, z, col, elasticity information) of each point constituting the slice information. The elastic information adding means 1032 can be usually realized by an MPU, a memory or the like. The processing procedure of the elasticity information adding means 1032 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The sampling position information acquisition unit 104 acquires a plurality of sampling position information having position information that constitutes a point at which a three-dimensional space of a three-dimensional object is sampled. For example, the sampling position information acquisition unit 104 divides the three-dimensional space of the three-dimensional object into 10 equal parts on the X-axis, Y-axis, and Z-axis, configures points in the three-dimensional space, and stores position information indicating the points. Sampling position information may be used. This point is preferably a point in or on a three-dimensional object. Further, the sampling position information acquisition unit 104 may, for example, divide the three-dimensional space of the three-dimensional object by dividing each axis of X, Y, and Z by 10 widths and use the position information of the divided intersection as the sampling position information. . In addition, the algorithm by which the sampling position information acquisition unit 104 acquires the sampling position information does not matter. The sampling position information acquisition unit 104 can usually be realized by an MPU, a memory, or the like. The processing procedure of the sampling position information acquisition unit 104 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The first mesh information accumulation unit 105 accumulates the first mesh information in the first mesh information storage unit 102 using the plurality of sampling position information acquired by the sampling position information acquisition unit 104. For example, the sampling position information is each point constituting the first mesh information. The first mesh information storage unit 105 can usually be realized by an MPU, a memory, or the like. The processing procedure of the first mesh information storage unit 105 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The instruction receiving unit 106 receives an instruction. For example, the instruction receiving unit 106 receives an instruction for a predetermined point or region of the slice information group output by the slice information group output unit 107. The instruction receiving unit 106 may receive a surgical instruction to be described later. The operation instruction is an instruction related to an operation, for example, an instruction for incision, twisting, pinching and the like. Alternatively, the surgical instruction received by the instruction receiving unit 106 may be a single instruction. The instruction input means may be anything such as the input / output device 100, a keyboard, a mouse, a numeric keypad, or a menu screen. The instruction receiving unit 106 can be realized by a device driver for input means such as a keyboard, control software for a menu screen, or the like.

  The slice information group output unit 107 outputs the slice information group configured by the elasticity information adding unit 103. Here, output refers to display on a display, projection using a projector, transmission to an external device (display device), sound output, storage in a recording medium, processing to other processing devices or other programs, etc. It is a concept that includes delivery of results. The slice information group output unit 107 may or may not include an output device such as a display or a speaker. The slice information group output unit 107 can be implemented by output device driver software, or output device driver software and an output device.

  The position information acquisition unit 108 acquires one or more pieces of position information of a point corresponding to the instruction received by the instruction receiving unit 106 or a point constituting an area. The position information acquisition unit 108 can be usually realized by an MPU, a memory, or the like. The processing procedure of the position information acquisition unit 108 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The elasticity information acquisition unit 109 acquires one or more elasticity information pairs with the one or more position information acquired by the position information acquisition unit 108. The one or more pieces of elasticity information are elasticity information included in the information of points constituting the slice information group stored in the slice information group storage unit 101, and the one or more pieces of position information acquired by the position information acquisition unit 108 One or more pieces of elasticity information to be paired. The position information corresponding to the elasticity information acquired by the elasticity information acquisition unit 109 may not be all the position information acquired by the position information acquisition unit 108. The elasticity information acquisition unit 109 can be usually realized by an MPU, a memory, or the like. The processing procedure of the elasticity information acquisition unit 109 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The second mesh information acquisition unit 110 deforms the first mesh information based on the instruction received by the instruction reception unit 106, and acquires the second mesh information constituting the deformed shape. The data structure of the second mesh information is usually the same as the first mesh information. The process in which the second mesh information acquisition unit 110 acquires the second mesh information that deforms the first mesh information and configures the deformed shape is usually a process by a finite element method, and is a known technique. The detailed explanation is omitted. The second mesh information acquisition unit 110 can usually be realized by an MPU, a memory, or the like. The processing procedure of the second mesh information acquisition unit 110 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The first slice information group acquisition unit 111 acquires a first slice information group, which is a plurality of slice information having no color information, from the slice information group storage unit 101 based on the second mesh information. The first slice information group acquisition unit 111 acquires a plurality of slice information that is information that can be acquired by slicing a three-dimensional object composed of second mesh information that is mesh information deformed with respect to the first mesh information. It is preferable that the interval between the slice information is constant. The plurality of slice information is preferably perpendicular to the line-of-sight vector. Further, the fact that the slice information constituting the first slice information group does not have color information includes having dummy color information that is a color that is not finally displayed. The first slice information group acquisition unit 111 can usually be realized by an MPU, a memory, or the like. The processing procedure of the first slice information group acquisition unit 111 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The color information determination unit 112 determines the color information of each point that corresponds to each point of the 3D voxel information that is each point of the plurality of slice information constituting the first slice information group. Each point corresponding to a point of 3D voxel information is a point in the 3D voxel information and is a point before deformation. The color information determination unit 112 can usually be realized by an MPU, a memory, or the like. The processing procedure of the color information determination unit 112 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  Corresponding point determination means 1121 determines each point in the 3D voxel information corresponding to each point of the plurality of slice information constituting the first slice information group. In addition, each point in this 3D voxel information is each point before a deformation | transformation. Corresponding point determination means 1121 can usually be realized by an MPU, a memory, or the like. The processing procedure of the corresponding point determination unit 1121 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The color information determination unit 1122 acquires color information of each point in the 3D voxel information determined by the corresponding point determination unit 1121. The color information determination unit 1122 can be usually realized by an MPU, a memory, or the like. The processing procedure of the color information determination unit 1122 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The second slice information group acquisition unit 113 adds a new color to each point of the first slice information group acquired by the first slice information group acquisition unit 111 based on the color information of each point determined by the color information determination unit 112. Information is set and the second slice information group is acquired. That is, the second slice information group acquisition unit 113 uses the color of each point determined by the color information determination unit 112 as new color information for each point of the first slice information group acquired by the first slice information group acquisition unit 111. Set the information. A plurality of slice information set with such color information is a second slice information group. The second slice information group acquisition unit 113 can be usually realized by an MPU, a memory, or the like. The processing procedure of the second slice information group acquisition unit 113 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The elasticity information output unit 114 outputs one or more elasticity information acquired by the elasticity information acquisition unit 109. The output of elasticity information may be output of information about force (for example, force vector) based on elasticity information, may display elasticity information, or is displayed in a color corresponding to elasticity information. It may be good. That is, the elasticity information output unit 114 outputs information about force (for example, a force vector) based on one or more elasticity information acquired by the elasticity information acquisition unit 109, for example. The output is a concept including sending a signal to the input / output device 100 such as a phantom, displaying on a display, printing on a printer, outputting sound, sending to an external device, and the like. The sound output here is, for example, an output in which force is expressed by the strength of sound. The elastic information output unit 114 can be realized by software, driver software of an output device such as a display or a speaker, or driver software of an output device and an output device.

  The deformed object output unit 115 outputs the second slice information group. The output is a concept including display on a display, printing on a printer, transmission to an external device (for example, a device having a display device), storage on a recording medium, and the like. Display refers to output to a display or projector. The display mode of the second slice information group is not limited. It is preferable that the deformed object output unit 115 sequentially outputs the slices having the deepest display depth among the plurality of slices constituting the second slice information group. The deformed object output unit 115 may or may not include an output device such as a display or a speaker. The deformed object output unit 115 can be implemented by output device driver software, or output device driver software and an output device.

  Next, the operation of the information processing apparatus 1 will be described using the flowchart of FIG.

  (Step S201) The instruction receiving unit 106 determines whether an instruction from the input / output device 100 (a keyboard or a mouse may be used) is received. If an instruction is accepted, the process goes to step S202, and if no instruction is accepted, the process returns to step S201.

  (Step S202) The instruction receiving unit 106 determines whether or not the instruction received in step S201 is a start instruction for the information processing apparatus 1 (for example, surgery simulation). If it is a start instruction, go to step S203, and if it is not a start instruction, go to step S205.

  (Step S203) The sampling position information acquisition unit 104 and the first mesh information accumulation unit 105 perform a mesh accumulation process. The mesh accumulation process will be described with reference to the flowchart of FIG.

  (Step S204) The elasticity information adding unit 103 performs elasticity information adding processing. The elasticity information adding process will be described with reference to the flowchart of FIG.

  (Step S205) The instruction receiving unit 106 determines whether or not the instruction received in step S201 is an instruction to output a slice information group. If it is an output instruction, go to step S206, and if it is not an output instruction, go to step S208.

  (Step S206) The slice information group output unit 107 reads the slice information group from the slice information group storage unit 101.

  (Step S207) The slice information group output unit 107 outputs the slice information group read out in step S206. The process returns to step S201. A three-dimensional object is displayed by outputting the slice information group.

  (Step S208) The instruction receiving unit 106 determines whether or not the instruction received in step S201 is a rotation instruction for a three-dimensional object. If it is a rotation instruction, the process proceeds to step S209, and if it is not a rotation instruction, the process proceeds to step S212.

  (Step S209) The slice information group acquisition unit 117 acquires a line-of-sight vector based on the rotation instruction received in step S201. The line-of-sight vector is a vector perpendicular to the display surface of the display. The rotation instruction is input by the input / output device 100 such as a mouse. Since the process of rotating the three-dimensional object with the mouse is a process according to a known technique, a detailed description thereof is omitted.

  (Step S210) The slice information group acquisition unit 117 cuts out a plurality of slice information that is perpendicular to the line-of-sight vector and has a constant interval from the 3D voxel information stored in the object information storage unit 116. To get. Assume that the interval value of slice information is stored in advance.

  (Step S211) The slice information group acquisition unit 117 at least temporarily accumulates the slice information group acquired in step S207 in the slice information group storage unit 101. Go to step S207.

  (Step S212) The instruction receiving unit 106 determines whether or not the instruction received in step S201 is a point or region instruction. If it is a point or area instruction, the process goes to step S213, and if it is not a point or area instruction, the process returns to step S201. Note that the point or region indication method may be any method using a mouse.

  (Step S213) The position information acquisition unit 108 acquires one or more pieces of position information of a point corresponding to the instruction received in Step S201 or a point constituting an area.

  (Step S214) The elasticity information acquisition unit 109 stores one or more pieces of elasticity information paired with one or more pieces of position information acquired by the position information acquisition unit 108, and stores each piece of slice information or object information in the slice information group storage unit 101. Obtained from the 3D voxel information of the unit 116. First, the elasticity information acquisition unit 109 searches for points having one or more pieces of position information from each slice information or 3D voxel information. Next, the elasticity information of the point obtained by the search is acquired.

(Step S215) The elasticity information output unit 114 calculates and calculates elasticity information output from one or more elasticity information acquired by the elasticity information acquisition unit 109. The elasticity information acquired by the elasticity information output unit 114 may be different from the elasticity information to be output. For example, the elastic information to be acquired may be an elastic modulus, and the elastic information to be output may be a force. In this case, the elasticity information output unit 114 may calculate an average value of one or more elasticity information and output the elasticity information. For example, the elasticity information output unit 114 may be elasticity information that outputs a maximum value of one or more elasticity information. Further, for example, the elasticity information output unit 114 may be elasticity information that outputs a minimum value of one or more elasticity information. In addition, the elasticity information output unit 114 has a predetermined function f (f (e ₁ , e ₂ ..., E _n ) using one or more pieces of elasticity information (e ₁ , e ₂ ..., E _n ) as parameters. )), The elastic information to be output may be calculated.

  (Step S216) The elasticity information output unit 114 outputs the one or more elasticity information acquired in Step S212 to the input / output device 100 (usually PHANToM).

  (Step S217) The input / output device 100 vibrates by driving the motor. By such processing, elasticity is transmitted to the user.

  (Step S218) The deformed object output unit 115 outputs a deformed slice information group (three-dimensional object). Such processing is called deformation processing and will be described with reference to the flowcharts of FIGS. The deformation process of the three-dimensional object corresponds to the instruction in step S201. That is, for example, when the user presses a certain point or region with PHANToM, the three-dimensional object is deformed depending on the pressed strength (or time). Go to step S207.

  In the flowchart of FIG. 2, the deformation process in step S215 may be performed before step S213.

  Further, in the flowchart of FIG. 2, only one of the mesh accumulation process in step S203 and the elasticity information addition process in step S204 may be executed. In such a case, it is usually necessary to perform manual work (data creation) corresponding to the other process.

  Further, in the flowchart of FIG. 2, the processing is ended by powering off or interruption for aborting the processing.

  Next, the mesh accumulation process of step S203 will be described using the flowchart of FIG.

  (Step S301) The sampling position information acquisition unit 104 acquires the division widths x, y, and z in the three-dimensional space. The division widths x, y, and z are values indicating intervals when dividing the X axis, the Y axis, and the Z axis, respectively. For example, the sampling position information acquisition unit 104 acquires the maximum values of the X axis, the Y axis, and the Z axis in the three-dimensional space, reads a predetermined division number (n), and reads the X axis, the Y axis, and the Z axis. A division width x, y, z that divides each maximum value into n equal parts is acquired.

  (Step S302) The sampling position information acquisition unit 104 substitutes 1 for a counter i.

  (Step S303) The sampling position information acquisition unit 104 determines whether i is n or less. If “i <= n”, the process proceeds to step S304, and if “i <= n”, the process returns to the upper process.

  (Step S304) The sampling position information acquisition unit 104 substitutes 1 for a counter j.

  (Step S305) The sampling position information acquisition unit 104 determines whether j is n or less. If “j <= n”, go to step S306, and if “j <= n”, go to step S313.

  (Step S306) The sampling position information acquisition unit 104 assigns 1 to the counter k.

  (Step S307) The sampling position information acquisition unit 104 determines whether k is n or less. If “k <= n”, go to step S308, and if “k <= n”, go to step S312.

  (Step S308) The sampling position information acquisition unit 104 calculates sampling position information having position information constituting points for sampling the three-dimensional space of the three-dimensional object. The sampling position information acquisition unit 104 calculates sampling position information ((i−1) × x, (i−1) × y, (i−1) × z).

  (Step S309) The sampling position information acquisition unit 104 determines that the point indicated by the sampling position information ((i−1) × x, (i−1) × y, (i−1) × z) is within a three-dimensional object. It is determined whether or not it exists (including on its surface). If it exists in the three-dimensional object, the process goes to step S310, and if it does not exist in the three-dimensional object, the process goes to step S311.

  (Step S310) The first mesh information storage unit 105 configures the first mesh information using the sampling position information acquired by the sampling position information acquisition unit 104, and stores the first mesh information in the first mesh information storage unit 102. accumulate. When the first mesh information has a structure of (x, y, z), for example, the first mesh information accumulation unit 105 performs sampling position information ((i−1) × x, (i−1) × y, (I-1) × z) is stored in the first mesh information storage unit 102 as it is.

  (Step S311) The sampling position information acquisition unit 104 increments the counter k by 1. The process returns to step S307.

  (Step S312) The sampling position information acquisition unit 104 increments the counter j by one. The process returns to step S305.

  (Step S313) The sampling position information acquisition unit 104 increments the counter i by one. The process returns to step S303.

  Note that the flowchart of FIG. 3 is merely an example of a mesh accumulation process. It does not matter what algorithm is used to acquire the sampling position information constituting the first mesh information. Information indicating external points constituting the tetrahedron as described later may be used as the sampling position information.

  In the flowchart of FIG. 3, the number of divisions of the X axis, the Y axis, and the Z axis are all the same, but may be different.

  Furthermore, in the flowchart of FIG. 3, the determination process in step S309 is not essential.

  Next, the elasticity information addition process of step S204 is demonstrated using the flowchart of FIG.

  (Step S401) The elasticity information acquisition means 1031 constituting the elasticity information adding unit 103 substitutes 1 for the counter i.

  (Step S402) The elasticity information acquisition means 1031 determines whether or not the i-th point exists among the points constituting each of the plurality of pieces of slice information. If the i-th point exists, the process proceeds to step S403, and if the i-th point does not exist, the process returns to the upper process. The plurality of pieces of slice information are slice information stored in the slice information group storage unit 101. Further, the elasticity information acquisition unit 1031 may determine whether or not the i-th point exists among the points constituting the 3D voxel information in the object information storage unit 116.

  (Step S403) The elasticity information acquisition means 1031 acquires the color information of the i-th point. Here, the color information acquired by the elasticity information acquisition unit 1031 may be only the shade value of the color information.

  (Step S404) The elasticity information acquisition means 1031 reads an arithmetic expression “f (color information)” for calculating elasticity information from the color information, substitutes the color information for the arithmetic expression, and acquires elasticity information. Here, the color information to be substituted into the arithmetic expression is the color information acquired in step S403.

  (Step S405) The elasticity information adding means 1032 writes the elasticity information acquired in step S404. That is, each point constituting the slice information in the slice information group storage unit 101 has a structure of (x, y, z, color information, elasticity information), for example. The elasticity information acquisition unit 1031 writes, for example, elasticity information of (x, y, z, color information, elasticity information).

  (Step S406) The elasticity information acquisition means 1031 increments the counter i by 1. The process returns to step S402.

  In the flowchart of FIG. 4, the data structure of each point constituting the slice information is not limited.

  Next, the deformation process in step S218 will be described with reference to the flowcharts of FIGS.

  (Step S <b> 501) The second mesh information acquisition unit 110 reads the first mesh information from the first mesh information storage unit 102.

  (Step S502) The second mesh information acquisition unit 110 deforms the first mesh information read in Step S501 based on the instruction received in Step S201, and acquires second mesh information constituting the deformed shape. Since the process of deforming the mesh information based on the deformation instruction is a known technique (technique of the finite element method), detailed description thereof is omitted.

  (Step S503) The first slice information group acquisition unit 111 acquires a first slice information group, which is a plurality of slice information, based on the second mesh information acquired in step S502. A three-dimensional object formed by the second mesh information is sliced to obtain information on a plurality of planes. Such plane information is slice information. The slice information is a set of points indicated by coordinate information (x, y, z), and has no color information. In such a case, the first slice information group acquisition unit 111 acquires a first slice information group that is a plurality of slice information at a predetermined interval that is perpendicular to the line-of-sight vector.

  (Step S504) The second slice information group acquisition unit 113 acquires a second slice information group. Details of such processing will be described with reference to the flowchart of FIG.

  (Step S505) The deformed object output unit 115 outputs the second slice information group acquired in Step S504. Return to upper function.

  Next, the process of acquiring the second slice information group in step S504 will be described using the flowchart of FIG.

  (Step S601) The second slice information group acquisition unit 113 substitutes 1 for a counter i.

  (Step S602) The second slice information group acquisition unit 113 determines whether or not the i-th slice information (unprocessed slice information) exists in the first slice information group. If the i-th slice information exists, the process goes to step S603, and if the i-th slice information does not exist, the process returns to the upper function.

  (Step S603) The second slice information group acquisition unit 113 substitutes 1 for a counter j.

  (Step S604) The second slice information group acquisition unit 113 determines whether there is an unprocessed j-th point in the i-th slice information. If the jth point exists, the process proceeds to step S605, and if the jth point does not exist, the process jumps to step S609. “Unprocessed” means that no color information is set.

  (Step S605) Corresponding point determination means 1121 is a point corresponding to the j-th point in the i-th slice information, and determines a point in the 3D voxel information. The point in 3D voxel information is a point before a deformation | transformation. Note that the jth point is a point after deformation. Details of an example algorithm for determining points in 3D voxel information will be described later.

  (Step S606) The color information determination unit 1122 acquires the color information of the point in the 3D voxel information determined in step S605.

  (Step S607) The color information determination unit 1122 sets the color information acquired in step S606 as the color information of the j-th point in the i-th slice information.

  (Step S608) The second slice information group acquisition unit 113 increments the counter j by 1. The process returns to step S604.

  (Step S609) The second slice information group acquisition unit 113 increments the counter i by 1. The process returns to step S602.

  Hereinafter, a specific operation of the information processing apparatus 1 in the present embodiment will be described. In the information processing apparatus 1, for example, the three-dimensional object is an organ such as a heart or a lung. For example, a set of medical images (three-dimensional images) illustrated in FIG. 7 is stored in the object information storage unit 116. This set of medical images is a set of medical images of a patient measured by CT. The data structure of each point constituting the 3D voxel information here is (x, y, z, col). Here, “x” is an x-coordinate value in the three-dimensional space, “y” is a y-coordinate value in the three-dimensional space, “z” is a z-coordinate value in the three-dimensional space, and col is information indicating the color of the point.

  Next, it is assumed that the user inputs an activation instruction to the information processing apparatus 1. Then, the instruction receiving unit 106 receives an activation instruction.

  Next, the sampling position information acquisition unit 104 and the first mesh information accumulation unit 105 perform mesh accumulation processing as follows. The first mesh information is accumulated in the first mesh information storage unit 102 by the mesh accumulation process.

  For example, the sampling position information acquisition unit 104 is a point in a three-dimensional space and automatically acquires position information of a point outside the tetrahedron. That is, organs such as the heart and lungs are three-dimensional objects and can be approximated by a set of tetrahedrons (regular tetrahedrons). Therefore, here, in order to simplify the description, a case where the tetrahedron shown in FIG. 8A is deformed will be described. In FIG. 8A, there are four points A, B, C, and O. Point P is a point inside the tetrahedron. That is, the sampling position information acquisition unit 104 divides the inside of the three-dimensional space into a set of tetrahedrons, and automatically acquires the position information of the outer points constituting the tetrahedron. Note that the sampling position information acquisition unit 104 holds the length of each side of the regular tetrahedron in advance. Since the process of dividing the three-dimensional space into a set of regular tetrahedrons and automatically acquiring the position information of the outer points constituting the tetrahedron is a known technique, detailed description thereof is omitted. An example of the position information acquired by the sampling position information acquisition unit 104 is shown in FIG. Then, the first mesh information accumulation unit 105 accumulates the sampling position information of FIG. 9 acquired by the sampling position information acquisition unit 104 in the first mesh information storage unit 102. Note that the first mesh information may be, for example, a set of information on the points outside and inside the tetrahedron (the points are spaced apart).

  Next, the elasticity information addition part 103 performs the elasticity information addition process shown below. That is, the elasticity information acquisition unit 1031 acquires color information for each point of a set of two or more medical images (stored in the object information storage unit 116) shown in FIG.

  Next, the elasticity information acquisition unit 1031 reads an arithmetic expression “f (color information)” for calculating elasticity information from the acquired color information, substitutes the color information for the arithmetic expression, and acquires elasticity information.

  Next, the elasticity information adding means 1032 writes the acquired elasticity information to each point of the 3D voxel information. Each point of the 3D voxel information has a structure of (x, y, z, color information, elasticity information).

  The elasticity information adding unit 103 performs acquisition of color information, acquisition of elasticity information, and writing of elasticity information for all points. By the above processing, elasticity information is automatically added.

  Next, it is assumed that the user has input a slice information group output instruction. Then, the instruction receiving unit 106 receives an instruction to output a slice information group.

Next, as shown in FIG. 10, the slice information group acquisition unit 117 is perpendicular to the line-of-sight vector and has a predetermined interval (the slice information group acquisition unit 117 holds the interval in advance). A slice information group which is a plurality of slice information is acquired from the 3D voxel information. The slice information group acquisition unit 117 obtains the positions “minD” and “maxD” of the three-dimensional object to be displayed, slices at a predetermined interval “D”, and acquires a plurality of slice information. The slice information is a set of point information. Further, there is no interval between points constituting the slice information. That is, the plane indicated by the slice information is filled with point information. The point information here has position information (x, y, z) and no color information. As a result, the slice information group acquisition unit 117 acquires the slice information group shown in FIG. The slice information group includes slice information S ₁ , slice information S ₂ , slice information S _{3 and the} like. Note that the slice information is acquired perpendicularly to the line-of-sight vector so that when the user looks at the set of slice information, even the thinned slice information can be seen stereoscopically. Moreover, the thinned slice information is acquired at a predetermined interval in order to speed up display processing. The reason why the slice information is acquired at regular intervals is to display a high-quality three-dimensional object. The line-of-sight vector is a vector perpendicular to the screen, and changes in accordance with the rotation instruction when the instruction receiving unit 106 receives the rotation instruction.

  Then, the slice information group acquisition unit 117 stores the acquired slice information group of FIG. 11 in the slice information group storage unit 101 at least temporarily.

  Next, the slice information group output unit 107 outputs the slice information group in the slice information group storage unit 101. Here, the user can recognize a three-dimensional tetrahedron from the output slice information group. Note that the trigger for the slice information group output unit 107 to output the slice information group may be a user instruction, reception of a command from an external device, or the like. The trigger does not matter.

  Next, it is assumed that the user inputs an instruction for a predetermined point or region with respect to the output slice information group and tries to deform the displayed three-dimensional tetrahedron. Such an instruction is called a deformation instruction. Here, the deformation instruction is input by, for example, a phantom included in the information processing apparatus 1. The input with the phantom is, for example, an input of pressing a tetrahedron point O shown in FIG. 8A to the left with a predetermined force. The instruction receiving unit 106 receives a deformation instruction. By such input, the tetrahedron shown in FIG. 8A becomes a tetrahedron as shown in FIG. And based on this deformation | transformation instruction | indication, the 2nd mesh information acquisition part 110 deform | transforms the 1st mesh information of FIG. 9, and acquires the 2nd mesh information which comprises the deformed shape. The second mesh information is shown in FIG. That is, the second mesh information is information indicating the tetrahedron in FIG. In addition, since the process which deform | transforms 1st mesh information and acquires 2nd mesh information is a well-known technique by a finite element method, detailed description is abbreviate | omitted.

  Next, color information acquisition processing will be described. As shown in FIGS. 8A and 8B, when a point P inside the mesh element is displaced to P ′ by the input of the deformation instruction, the position of P in the 3D voxel information is included in the color information of P ′. Needs to be assigned color information. If the relative position from any vertex of the arbitrary point inside the mesh does not change before and after the deformation, the positions of the internal points P and P ′ before and after the deformation use the common parameters s, t and u as follows: It can be expressed as a linear combination of each edge.

    OP = sOA + tOB + uOC Formula (1)

    O′P ′ = sO′A ′ + tO′B ′ + uO′C ′ Formula (2)

  Here, the corresponding point determination unit 1121 solves the equation (2) to obtain the parameters s, t, u defining the internal point P ′ from the deformed mesh, and the position P before the deformation from the equation (1). Get. Then, the color information corresponding to the position P is acquired from the 3D voxel information. Then, the color information determination unit 1122 acquires and sets color information of each point in the plurality of slice information constituting the first slice information group determined by the corresponding point determination unit 1121. As a result, each point constituting each piece of slice information after deformation in FIG. 13 has color information.

  In the above processing, even when a node (point) constituting the mesh information is displaced or the mesh information is reconfigured, the relative position of the point in the element from each node is obtained before and after the simulation. As long as it is possible, color information at an arbitrary point inside can be reproduced.

  Next, the deformed object output unit 115 outputs the second slice information group. With this process, the three-dimensional object after receiving the deformation instruction is output in real time. By repeatedly receiving the deformation instruction and outputting the deformed three-dimensional object, for example, biological function analysis in the medical field, real-time surgery simulation, and the like are possible.

  Next, the position information acquisition unit 108 acquires one or more pieces of position information of a point corresponding to the received instruction (deformation instruction) or a point constituting the region. Here, for example, the position information of the point O is acquired. Next, the elasticity information acquisition unit 109 converts each piece of slice information in the slice information group storage unit 101 into one or more pieces of elasticity information (e) paired with the position information of the point O acquired by the position information acquisition unit 108. Alternatively, it is obtained from 3D voxel information in the object information storage unit 116.

  Next, the elasticity information output unit 114 calculates and calculates a force vector output from one or more elasticity information acquired by the elasticity information acquisition unit 109. Note that when the elasticity information acquisition unit 109 has one piece of elasticity information, the elasticity information output unit 114 normally outputs the elasticity information as it is (for example, output to a phantom described later).

  Next, the elasticity information output unit 114 outputs the acquired force vector to the phantom. Then, the phantom drives the motor in response to the force vector. And a user can feel the reaction force obtained with respect to the given pushing using a phantom.

  As described above, according to the present embodiment, elasticity information is automatically added only by giving 3D voxel information, and elasticity information is output when a user operates a three-dimensional object being output. When such an information processing apparatus 1 is used for a surgical simulation, for example, a surgical simulation can be performed simply by giving the information processing apparatus 1 a set of two-dimensional images acquired by medical equipment such as CT, MRI, and PET. become.

  Moreover, according to this Embodiment, the deformation | transformation and destruction which arose in the mesh can be drawn in real time with the surface and internal color information. In addition, information on the elasticity of a three-dimensional object can be handled. Specifically, when an input / output device such as a phantom is used to push or grab the output 3D object, the 3D object is felt while feeling the hardness of the pushed / gripped part. The shape can be changed. In addition, according to the present embodiment, it is possible to simulate real-time deformation of a three-dimensional object while feeling hardness using the slice information group acquired from 3D voxel information and mesh information. In other words, by using both the original image voxel data (3D voxel information) and the mesh data (mesh information) of the target region, the deformed mesh elements are represented by overlapping texture-mapped cross sections. The user can feel the hardness of the three-dimensional object because each point constituting the three-dimensional object and / or each point constituting the slice information group has elasticity information and can draw the internal structure with high definition. However, the deformation of the three-dimensional object can be simulated. Further, according to the present embodiment, a volume composed of, for example, 256 × 256 × 256 voxel on a general-purpose PC without using a dedicated graphics card, corresponding to a dynamic calculation algorithm typified by the finite element method. Smooth deformation animation can be generated for data.

  Note that according to the present embodiment, the information processing apparatus 1 automatically adds elasticity information and also automatically acquires mesh information. However, the information processing apparatus 1 may automatically acquire only the elasticity information, and may construct the mesh information manually. Further, the information processing apparatus 1 may automatically acquire only mesh information, and elastic information may be manually constructed.

  Furthermore, the processing in the present embodiment may be realized by software. Then, this software may be distributed by software download or the like. Further, this software may be recorded and distributed on a recording medium such as a CD-ROM. This also applies to other embodiments in this specification. Note that the software that implements the information processing apparatus 1 in the present embodiment is the following program. In other words, this program is slice information that is information configured based on two-dimensional image data obtained by cutting out 3D voxel information, which is a volume texture of a three-dimensional object, from a plurality of planes. Elasticity information, which is information about elasticity, is added to each point constituting a plurality of slice information composed of information of a plurality of points having position information, which is information indicating color, and color information, which is information about color. A plurality of pieces of slice information including position information, color information, and elasticity information, and a slice information group including the slice information group formed by the elasticity information addition unit. An information group output unit; an instruction reception unit that receives an instruction for a predetermined point or region of the slice information group output by the slice information group output unit; A position information acquisition unit that acquires one or more pieces of position information of a point corresponding to the instruction or a point that constitutes a region; And a program for functioning as an elastic information output unit that outputs based on one or more pieces of elastic information acquired by the elastic information acquisition unit.

  In addition, the program deforms the first mesh information, which is information about the three-dimensional mesh of the three-dimensional object, based on the instruction received by the instruction receiving unit, and obtains second mesh information constituting the deformed shape. A first mesh information acquisition unit that acquires, based on the second mesh information, a first slice information group acquisition unit that acquires a first slice information group that is a plurality of slice information without color information from the storage medium; The color information determination unit that determines the color information of each point that corresponds to each point of the 3D voxel information that is each point of the plurality of slice information constituting the first slice information group, and the color information determination unit determines A second slice information group that sets new color information for each point of the first slice information group acquired by the first slice information group acquisition unit and acquires a second slice information group based on the color information of each point And resulting unit, as a modification object output portion that outputs the second slice information group, the computer is suitably a program for further functions.

  The program further causes the computer to function as a sampling position information acquisition unit that acquires a plurality of sampling position information having position information that constitutes a point at which a three-dimensional space of a three-dimensional object is sampled, and the first The mesh information is preferably composed of the plurality of sampling position information.

  Further, in the above program, the elasticity information adding unit acquires the elasticity information acquisition means for acquiring elasticity information using the color information of each point constituting the plurality of slice information, and the elasticity information acquisition means acquires the elasticity information. It is preferable that the program is a program for causing a computer to function as having elasticity information adding means for adding elasticity information to each point constituting each of the plurality of slice information.

  (Embodiment 2)

  In the present embodiment, an information processing apparatus capable of automatically adding elasticity information and performing simulation using the elasticity information will be described. In the present embodiment, an information processing apparatus that determines elasticity information using a function having a Eugrid distance from a singular point (for example, a point having a tumor) as a parameter will be described.

  FIG. 14 is a block diagram of the information processing apparatus 2 in the present embodiment. The information processing apparatus 2 includes a slice information group storage unit 101, a first mesh information storage unit 102, an elasticity information addition unit 203, a sampling position information acquisition unit 104, a first mesh information storage unit 105, an instruction reception unit 106, and singular point information. Reception unit 206, slice information group output unit 107, position information acquisition unit 108, elasticity information acquisition unit 109, second mesh information acquisition unit 110, first slice information group acquisition unit 111, color information determination unit 112, second slice information A group acquisition unit 113, an elasticity information output unit 114, a deformed object output unit 115, an object information storage unit 116, and a slice information group acquisition unit 117 are provided.

  The elastic information adding unit 203 includes a distance acquisition unit 2031, an elastic information acquisition unit 2032, and an elastic information addition unit 2033.

  The singularity information receiving unit 206 receives singularity information that is information indicating one or more singularities. The singular point information is usually information indicating one or more points, for example, information indicating one or more points having a structure of (x, y, z). Further, the singular point is, for example, a point in a region with a tumor or a point in a region with a wound or a surgical scar. The singularity information input means may be anything such as a keyboard, mouse, numeric keypad, or menu screen. The singularity information receiving unit 206 can be realized by a device driver of input means such as a keyboard, control software for a menu screen, or the like.

  The elasticity information adding unit 203 determines the elasticity information of each point using a distance from one or more singular points as a parameter, and adds the elasticity information to each point. In this case, the elasticity information adding unit 203 stores information on an arithmetic expression “elasticity = f (distance from one or more singular points)”, and the distance from one or more singular points is stored in the arithmetic expression. Substitute and determine elasticity information. Further, the elasticity information adding unit 203 determines the elasticity information of each point using the distance from one or more singular points and the color information of each point as parameters, and adds the elasticity information to each point. Is preferred. In this case, the elasticity information adding unit 203 stores information on an arithmetic expression of “elasticity = f (distance from one or more singular points, color information)”, and the arithmetic expression includes one or more singular points. The elasticity information is determined by substituting the distance. Each point is each point constituting a plurality of pieces of slice information, or each point of 3D voxel information stored in the object information storage unit 116. The elasticity information adding unit 203 can determine elasticity information in consideration of the fact that the vicinity of a singular point such as a point in a region with a tumor is hard.

  The distance acquisition unit 2031 acquires a distance from each point constituting each of the plurality of pieces of slice information and one or more singular points. Since each point constituting each of the plurality of slice information is included in each point of the 3D voxel information, the distance acquisition unit 2031 may acquire the distance from each point of the 3D voxel information and one or more singular points. good. A technique for calculating a distance when each point constituting slice information (or each point of 3D voxel information) is (x1, y1, z1) and a singular point is (x2, y2, z2) is a known technique Therefore, explanation is omitted. When there are a plurality of singular points, the distance acquisition unit 2031 may acquire the distance between each point constituting the slice information (or each point of the 3D voxel information) and the nearest singular point, or the slice information You may acquire the average value etc. of the distance of each point (or each point of 3D voxel information) to comprise, and a plurality of singular points. This average value is also considered to be a distance from one or more singular points. The distance acquisition unit 2031 can be usually realized by an MPU, a memory, or the like. The processing procedure of the distance acquisition unit 2031 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The elasticity information acquisition unit 2032 acquires elasticity information using the distance acquired by the distance acquisition unit 2031. Further, the elasticity information acquisition unit 2032 may acquire the elasticity information by using the color information of each point constituting each piece of slice information and the distance acquired by the distance acquisition unit 2031. The elastic information acquisition unit 2032 can be realized usually by an MPU, a memory, or the like. The processing procedure of the elasticity information acquisition means 2032 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  Next, the operation of the information processing apparatus 2 will be described using the flowchart of FIG. In the flowchart of FIG. 15, only steps different from the flowchart of FIG. 2 will be described.

  (Step S1501) The singularity information reception unit 206 determines whether or not the instruction (information) received in step S201 is singularity information. If it is singular point information, go to step S1502, and if it is not singular point information, go to step S202.

  (Step S1502) The singularity information reception unit 206 temporarily stores the received singularity information at least temporarily in a storage medium (not shown). The process returns to step S201.

  (Step S1503) The elasticity information addition unit 203 performs elasticity information addition processing. The elasticity information adding process will be described with reference to the flowchart of FIG.

  In the flowchart of FIG. 15, the process is ended by powering off or interruption for aborting the process.

  Next, the elasticity information addition process of step S1503 is demonstrated using the flowchart of FIG. In the flowchart of FIG. 16, only steps different from the flowchart of FIG. 4 will be described.

  (Step S1601) The distance acquisition unit 2031 substitutes 1 for the counter j.

  (Step S1602) The distance acquisition unit 2031 determines whether or not the j-th singular point exists. If the jth singular point exists, the process proceeds to step S1603, and if the jth singular point does not exist, the process proceeds to step S1605.

  (Step S1603) The distance acquisition unit 2031 calculates the distance between the j-th singular point and the i-th point, and arranges the distance on the memory.

  (Step S1604) The distance acquisition unit 2031 increments the counter j by 1. Go to step S1602.

  (Step S1605) The distance acquisition unit 2031 acquires the closest distance (minimum distance) among the distances (calculated in step S1603) existing on the memory. This distance is the distance from the singular point.

  (Step S1606) The elasticity information acquisition means 2032 reads the information of the arithmetic expression “elastic information = f (color information, distance)”, substitutes the acquired color information and distance into the arithmetic expression, and calculates the arithmetic expression. Execute. Then, the elasticity information acquisition unit 2032 acquires the elasticity information and arranges it on the memory.

  In the flowchart of FIG. 16, f in the arithmetic expression “elastic information = f (color information, distance)” usually acquires elastic information that becomes harder as the color indicated by the color information is darker. Moreover, f acquires the elastic information which becomes hard, so that a distance is small normally. However, the content of f does not matter. Further, f may be a function having only distance as a parameter.

  In addition, in step S1605 of the flowchart of FIG. 16, when there are a plurality of singular points, the minimum distance among the distances between the i-th point and the plurality of singular points is acquired. However, you may acquire the information regarding other distances, such as acquiring the average of the distance with several singularities.

  Further, in the flowchart of FIG. 16, the distance calculation method is not limited.

  Hereinafter, a specific operation of the information processing apparatus 2 in the present embodiment will be described. In the information processing apparatus 2, for example, the three-dimensional object is an organ such as a heart or a lung. The difference from the information processing apparatus 1 described in the first embodiment is a process of automatically adding elasticity information using singularity information. The process for automatically adding elasticity information will be described.

  Now, the elasticity information acquisition means 2032 holds the information of the arithmetic expression “elastic modulus = tone value * distance coefficient * constant”. In the arithmetic expression, the gray value is a gray value included in the color information. The distance coefficient is a distance coefficient that can be acquired from the distance coefficient graph shown in FIG. The elasticity information acquisition unit 2032 holds information of the distance coefficient graph shown in FIG. 17 in advance, and acquires the distance coefficient from the distance between the singular point and the point (value on the horizontal axis). The distance coefficient graph of FIG. 17 indicates that singular points (such as tumors) do not affect the elastic information of each point (distance coefficient is 1) when the distance is a certain value or more. The constant is a constant for converting the elasticity information to the Young's modulus, and is held in advance by the elasticity information acquisition unit 2032.

Further, it is assumed that the user inputs singular point information (x _s , y _s , z _s ) that is information indicating a point where the tumor is present. Then, the singular point information receiving unit 206 receives the singular point information (x _s , y _s , z _s ) and arranges it on the memory.

  Next, it is assumed that the user inputs an activation instruction to the information processing apparatus 1. Then, the instruction receiving unit 106 receives an activation instruction.

  Next, the sampling position information acquisition unit 104 and the first mesh information accumulation unit 105 perform mesh accumulation processing as described in the first embodiment.

  Next, the elasticity information addition unit 203 performs the following elasticity information addition processing. That is, the elasticity information acquisition unit 2032 acquires a gray value for each point of a set of two or more medical images (stored in the object information storage unit 116) illustrated in FIG.

Then, the distance acquisition unit 2031 calculates the distance between the singular point (x _s , y _s , z _s ) and each point.

  Next, the elasticity information acquisition means 2032 acquires a distance coefficient using the distance with respect to the graph of FIG. Next, the elasticity information acquisition means 2032 acquires the elasticity modulus by substituting the intensity value, distance coefficient, and constant for each point into the arithmetic expression “elastic modulus = density value * distance coefficient * constant”.

  Next, the elastic information adding means 2033 writes the elastic modulus for each point of the 3D voxel information. Each point of 3D voxel information has a structure of (x, y, z, color information, elastic modulus). By the above processing, elasticity information is automatically added.

  The operation of the information processing apparatus 2 after the structure of each point of the 3D voxel information becomes (x, y, z, color information, elastic modulus) is the same as the operation described in the first embodiment. The description is omitted.

  As described above, according to the present embodiment, elasticity information is automatically added only by giving 3D voxel information, and elasticity information is output when a user operates a three-dimensional object being output. Further, according to the present embodiment, it is possible to automatically add elasticity information considering the characteristics of a living body such as a tumor. When such an information processing apparatus 2 is used for a surgical simulation, for example, a highly accurate surgical simulation can be performed simply by giving the information processing apparatus 2 a set of two-dimensional images acquired by a medical device such as CT, MRI, or PET. Is possible.

  Note that the software that implements the information processing apparatus according to the present embodiment is the following program. In other words, this program is slice information that is information configured based on two-dimensional image data obtained by cutting out 3D voxel information, which is a volume texture of a three-dimensional object, from a plurality of planes. Elasticity information, which is information about elasticity, is added to each point constituting a plurality of slice information composed of information of a plurality of points having position information, which is information indicating color, and color information, which is information about color. A plurality of pieces of slice information including position information, color information, and elasticity information, and a slice information group including the slice information group formed by the elasticity information addition unit. An information group output unit; an instruction reception unit that receives an instruction for a predetermined point or region of the slice information group output by the slice information group output unit; A position information acquisition unit that acquires one or more pieces of position information of a point corresponding to the instruction or a point that constitutes a region, and one or more elastic information that is paired with one or more pieces of position information acquired by the position information acquisition unit And a program for functioning as an elastic information output unit that outputs based on one or more pieces of elastic information acquired by the elastic information acquisition unit.

  In the above program, the computer is further caused to function as a singular point information receiving unit that receives singular point information that is information indicating one or more singular points, and the elastic information adding unit is a distance from the one or more singular points. The elasticity information acquisition means acquires the elasticity information using the color information of each point constituting each of the plurality of slice information and the distance acquisition means using the acquired distance. It is preferable to make the computer function as described above.

  (Embodiment 3)

  In the present embodiment, an information processing apparatus capable of automatically adding elasticity information and performing simulation using the elasticity information will be described. Further, in the present embodiment, an information processing apparatus that can have information related to an operation (surgery) method and can perform surgery simulation using the information will be described. The operation information is, for example, information related to the incision, information indicating “twist”, and information indicating “pinch”. Further, in the present embodiment, an information processing apparatus that can have anatomical information (positional information such as sternum and ribs) and can correct the data to match the living body of the subject using the information. explain.

  FIG. 18 is a block diagram of the information processing apparatus 3 in the present embodiment. The information processing apparatus 3 includes a slice information group storage unit 101, a first mesh information storage unit 102, an operation information storage unit 301, an anatomical information storage unit 302, an elastic information addition unit 103, a mesh information correction unit 303, and an instruction reception unit. 106, slice information group output unit 107, position information acquisition unit 108, elasticity information acquisition unit 109, second mesh information acquisition unit 310, first slice information group acquisition unit 111, color information determination unit 112, second slice information group acquisition Unit 113, elasticity information output unit 114, object information storage unit 116, and slice information group acquisition unit 117.

  The surgery information storage unit 301 can store surgery information. Surgery information is information regarding surgery. The surgery information is, for example, information related to the living body of the operated person or information indicating points or areas in the operating person's living body. The surgical information includes, for example, incision position information that is information indicating the position of the incision. That is, the surgery information may include information such as the shape and attributes of the incision. Further, the surgery information may have physical conditions such as how to apply force. In addition, the surgery information may be, for example, twist position information that is information on a point or a region where a twisting action (one of the surgical actions) is performed. The twist position information is information for specifying a point, for example, coordinate information of (x, y, z) or a point identifier. Further, the operation information may be pick position information that is information on a point or a region where a pinching operation (one of operation operations) is performed. The knob position information is also information for specifying a point, for example, coordinate information of (x, y, z) or a point identifier. The attribute of the operation information is, for example, position information indicating the position of the scapula, the lowest part of the sternum, and the like, and is anatomical information that becomes a landmark of the human body. The position information is, for example, coordinate information (x, y, z) or a point identifier. The surgical information storage unit 301 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium. The process in which the surgery information is stored in the surgery information storage unit 301 does not matter. For example, surgery information may be stored in the surgery information storage unit 301 via a recording medium, and surgery information transmitted via a communication line or the like is stored in the surgery information storage unit 301. Alternatively, the surgery information input via the input device may be stored in the surgery information storage unit 301.

  The anatomical information storage unit 302 is information indicating the structure of the human body, and can store anatomical information having human body feature point position information that is information indicating the position of the human body part. The human body feature point position information is information indicating apex positions of the sternum, ribs, and the like, for example. The human body feature point position information is information indicating the positions of six points, for example, the uppermost part of the sternum, the lowermost part of the sternum, the lowermost part of both shoulder blades, and both nipples. The anatomical information storage unit 302 stores, for example, standard anatomical information and anatomical information of individuals who undergo surgery. It is preferable that the anatomical information storage unit 302 holds, for example, standard anatomical information different for each gender or age group, or holds standard anatomical information different for each type of person. It is. The anatomical information storage unit 302 is preferably a nonvolatile recording medium, but can also be realized by a volatile recording medium. The process in which the anatomical information is stored in the anatomical information storage unit 302 does not matter. For example, the anatomical information may be stored in the anatomical information storage unit 302 via the recording medium, and the anatomical information transmitted via the communication line or the like is stored in the anatomical information storage unit. 302 may be stored, or anatomical information input via the input device may be stored in the anatomical information storage unit 302.

  The mesh information correction unit 303 corrects the first mesh information using the anatomical information, and configures new first mesh information. For example, the mesh information correction unit 303 performs processing such as correcting incision position information so as to match the living body of the patient. That is, specifically, the mesh information correction unit 303 includes first mesh information (for example, standard human body mesh information) stored in the first mesh information storage unit 102 and anatomical information (for example, The first mesh information is corrected using the standard anatomical information) and the input individual anatomical information. The mesh information correction unit 303 corrects the first mesh information (for example, standard human body mesh information) from the difference between the individual anatomical information and the standard anatomical information. When a plurality of anatomical information is stored in the anatomical information storage unit 302, the mesh information correction unit 303 selects one anatomical information and uses the selected anatomical information. The first mesh information is corrected to form new first mesh information. There is no limitation on the method by which the mesh information correction unit 303 selects one anatomical information. One anatomical information may be selected based on an instruction from the user.

More specifically, the mesh information correction unit 303 holds, for example, information on the evaluation function shown in the following formula 1, and substitutes parameters ω _i = 1.0 and λ _i = 10.0, for example. The mesh information correction unit 303 executes the evaluation function of Formula 1 and corrects the first mesh information that minimizes the evaluation function using the least square method. Since the evaluation function of Formula 1 is a quadratic function, the corrected first mesh information can be derived by solving the minimum problem. In addition, when the evaluation function is a quadratic function, it is known that a set of p ′ (vertex after movement) that minimizes the evaluation function can be obtained uniquely. Further, the “feature point to be constrained” in the evaluation function of Equation 1 is anatomical information here.

  The mesh information correction unit 303 can be usually realized by an MPU, a memory, or the like. The processing procedure of the mesh information correction unit 303 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The instruction receiving unit 106 is an instruction for a predetermined point or region of the output slice information group, and receives an operation instruction that is an instruction related to an operation. The surgical instruction is an instruction for incision, twisting, pinching and the like. Alternatively, the operation instruction may be a single instruction. That is, the user may perform the same operation regardless of whether the incision is performed, the twist is performed, or the user is pinched, and the point or area where the operation is received may have an action identifier. The action identifier is, for example, “incision (for example,“ 1 ”),“ twist (for example, “2”) ”,“ pinch (for example, “3”) ”, or the like.

  The second mesh information acquisition unit 310 deforms the first mesh information based on the instruction received by the instruction reception unit 106, and acquires the second mesh information constituting the deformed shape. In particular, the second mesh information acquisition unit 310 deforms the first mesh information based on the operation instruction and operation information received by the instruction reception unit 106, and acquires the second mesh information constituting the deformed shape.

  The second mesh information acquisition unit 310 is a point in the first mesh information based on the operation instruction received by the instruction reception unit 106 and the incision position information included in the operation information, and the point corresponding to the incision position information. Among them, the point on the left side of the incision moves to the left side, the point on the right side of the incision moves to the right side, and the second mesh information is deformed so that the incision opens. Get second mesh information.

  The second mesh information acquisition unit 310 moves a point corresponding to the twist position information, which is a point in the first mesh information, based on the operation instruction received by the instruction reception unit 106 and the twist position information included in the operation information. Then, the second mesh information is deformed so as to be twisted, and the second mesh information constituting the deformed shape is acquired. Note that a point moving algorithm for twisting is a known technique.

  The second mesh information acquisition unit 310 moves the point corresponding to the pick position information, which is a point in the first mesh information, based on the operation instruction received by the instruction reception unit 106 and the pick position information included in the operation information. Then, the second mesh information is deformed so as to be picked, and the second mesh information constituting the deformed shape is acquired. Note that a point moving algorithm for picking is a known technique.

  The second mesh information acquisition unit 310 can usually be realized by an MPU, a memory, or the like. The processing procedure of the second mesh information acquisition unit 310 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  Next, the operation of the information processing apparatus 3 will be described. The operation of the information processing apparatus 3 is different from the operation of the information processing apparatus 1 in that a mesh correction process is added and the deformation process in step S218 is different. Therefore, only the mesh correction process and the deformation process will be described.

  First, the mesh correction process will be described below. The mesh information correction unit 303 corrects the first mesh information so that the evaluation function of Formula 1 is minimized, acquires new first mesh information, and writes it in the first mesh information storage unit 102 at least temporarily. Here, “writing new first mesh information” means updating the first mesh information.

  Next, the deformation process of the information processing apparatus 3 will be described using the flowchart of FIG. In the flowchart of FIG. 19, only the steps different from the flowchart of FIG. 5 will be described.

  (Step S1901) The second mesh information acquisition unit 310 performs acquisition processing of second mesh information. The second mesh information acquisition process will be described using the flowchart of FIG.

  Next, the second mesh information acquisition process of step S1901 will be described using the flowchart of FIG.

  (Step S2001) The second mesh information acquisition unit 310 determines whether or not the operation instruction received by the instruction reception unit 106 is an incision instruction. If it is an incision instruction, the process proceeds to step S2002. If it is not an incision instruction, the process proceeds to step S2007.

  (Step S2002) The second mesh information acquisition unit 310 acquires one or more points (or regions) for which the instruction receiving unit 106 has received an instruction.

  (Step S2003) The second mesh information acquisition unit 310 acquires the incision position information of the surgery information storage unit 301.

  (Step S2004) The second mesh information acquisition unit 310 determines whether one or more points acquired in step S2002 match the incision position information acquired in step S2003. If they match, go to step S2005, otherwise go to step S2006. “Match” means satisfying a condition for the instruction to be an incision instruction. More specifically, for example, “match” means that all the one or more points acquired in step S2002 are preliminarily determined from a straight line constituted by incision position information (for example, a set of one or more points). Having a distance within a predetermined threshold. When the surgical information has physical conditions such as the magnitude of force for incision, the second mesh information acquisition unit 310 determines that “match” only when there is an input that matches the physical conditions. You may judge.

  (Step S2005) The second mesh information acquisition unit 310 reads the first mesh information from the first mesh information storage unit 102, is a point in the first mesh information, and among the points corresponding to the incision position information, The point on the left side of the incision moves to the left (moves according to a predetermined movement algorithm) and the point on the right side of the incision moves to the right (moves according to a predetermined movement algorithm) The second mesh information is deformed so that the incision is opened, and second mesh information constituting the deformed shape is acquired. Return to upper process. The predetermined moving algorithm is an algorithm for moving a point so that the center portion of the incision line is wide open, both end portions are small open, and both end portions are shaped like an ellipse.

  (Step S2006) The second mesh information acquisition unit 310 outputs an error. For example, the second mesh information acquisition unit 310 outputs that the incision position is incorrect. Return to upper process.

  (Step S2007) The second mesh information acquisition unit 310 determines whether or not the operation instruction received by the instruction reception unit 106 is a twist instruction. If the instruction is a twist, the process goes to step S2008. If the instruction is not a twist, the process goes to step S2013.

  (Step S2008) The second mesh information acquisition unit 310 acquires one or more points (or regions) for which the instruction receiving unit 106 has received an instruction.

  (Step S2009) The second mesh information acquisition unit 310 acquires the twist position information of the surgery information storage unit 301.

  (Step S2010) The second mesh information acquisition unit 310 determines whether one or more points acquired in step S2008 match the twist position information acquired in step S2009. If they match, go to step S2011, and if not, go to step S2012. “Match” means satisfying a condition for the instruction to be a twist instruction. More specifically, for example, “match” means that all the one or more points acquired in step S2002 are the points of the region to be twisted.

  (Step S2011) The second mesh information acquisition unit 310 reads the first mesh information from the first mesh information storage unit 102, is a point in the first mesh information, and has received an instruction (one or more points in the region). Point) is deformed as if twisted, and second mesh information constituting the deformed shape is acquired. Return to upper process.

  (Step S2012) The second mesh information acquisition unit 310 outputs an error. For example, the second mesh information acquisition unit 310 outputs a message indicating that a twist instruction has been input to an area that cannot be twisted. Return to upper process.

  (Step S2013) The second mesh information acquisition unit 310 determines whether or not the operation instruction received by the instruction reception unit 106 is a knob instruction. If it is a knob instruction, the process proceeds to step S2014, and if it is not a knob instruction, the process proceeds to step S502.

  (Step S2014) The second mesh information acquisition unit 310 acquires one or more points (or regions) for which the instruction receiving unit 106 has received an instruction.

  (Step S2015) The second mesh information acquisition unit 310 acquires the pick position information in the surgery information storage unit 301.

  (Step S2016) The second mesh information acquisition unit 310 determines whether or not one or more points acquired in step S2008 match the knob position information acquired in step S2009. If they match, go to step S2017, otherwise go to step S2018. Note that “match” means that the condition for the instruction to be a picking instruction is satisfied. More specifically, for example, “match” means that all the one or more points acquired in step S2002 are the points of the target area to be picked.

  (Step S2017) The second mesh information acquisition unit 310 reads the first mesh information from the first mesh information storage unit 102, is a point in the first mesh information, and has received an instruction (one or more points in the region). The second mesh information constituting the deformed shape is acquired. Return to upper process.

  (Step S2018) The second mesh information acquisition unit 310 outputs an error. For example, the second mesh information acquisition unit 310 outputs a message indicating that an instruction to pick a region that cannot be picked is input. Return to upper process.

  Hereinafter, a specific operation of the information processing apparatus 3 in the present embodiment will be described. In the information processing apparatus 3, for example, the three-dimensional object is a human chest. The object information storage unit 116 holds CT volume data of a certain adult male.

  Now, the surgery information storage unit 301 stores incision position information that is a set of points around the center of the human chest. The anatomical information storage unit 302 holds information indicating the positions of six points of the sternum uppermost part, the sternum lowermost part, both shoulder blade lowermost parts, and both nipples. This is shown in FIG. FIG. 21A shows that the information indicating the position of the uppermost part of the sternum and the information indicating the position of the lowermost part of the sternum are anatomical information. FIG. 21B shows that the information indicating the position of the lowest part of both shoulder blades is anatomical information. FIG. 21C shows that the information indicating the positions of both nipples is anatomical information. FIG. 21 (d) shows an incision that is a line connecting point information indicated by the incision position information. Note that the anatomical information (scapula position information, position information at the bottom of the sternum, etc.) may be input by the user using an input means such as a mouse, or may be automatically acquired by image processing. Since a technique for acquiring feature points by image processing is a known technique, detailed description thereof is omitted.

  Then, it is assumed that the information processing apparatus 3 obtains information (hereinafter referred to as an incision template) as illustrated in FIG. The incision template includes shape information composed of first mesh information, elasticity information, incision wound position information, and anatomical information (information indicating positions of six points).

  In such a case, for example, the mesh information correction unit 303 corrects the first mesh information so as to minimize the evaluation function of Formula 1, and configures new first mesh information.

  In the above state, it is assumed that the user inputs an activation instruction to the information processing apparatus 3. Then, the instruction receiving unit 106 receives an activation instruction. It is assumed that the information processing apparatus 3 automatically adds the elasticity information and automatically acquires the first mesh information from the CT volume data in the object information storage unit 116 by the process described in the first embodiment.

  Next, it is assumed that the user has input a slice information group output instruction. Then, the instruction receiving unit 106 receives an instruction to output a slice information group. Then, the information processing apparatus 3 displays the chest by the process described in the first embodiment (the same process as the information processing apparatus 1).

  Next, it is assumed that the user inputs an operation instruction for incision along the incision. Then, the second mesh information acquisition unit 310 acquires one or more points (or regions) for which the instruction receiving unit 106 has received the instruction.

  Next, the second mesh information acquisition unit 310 acquires incision position information (a set of points indicating an incision in FIG. 21D) in the surgery information storage unit 301.

  Then, it is assumed that the second mesh information acquisition unit 310 determines that one or more points for which the instruction has been received match the acquired incision position information.

  Then, the second mesh information acquisition unit 310 reads the first mesh information from the first mesh information storage unit 102, is a point in the first mesh information, and among the points corresponding to the incision position information, the incision wound information. The point on the left side of the eye will move to the left (move with a pre-determined movement algorithm), the point on the right side of the incision will move to the right (move with a pre-determined movement algorithm), and The second mesh information is deformed so that the wound opens, and second mesh information constituting the deformed shape is acquired.

  And the 1st slice information group acquisition part 111 acquires the 1st slice information group which is several slice information based on the acquired 2nd mesh information. That is, the 1st slice information group acquisition part 111 slices the three-dimensional object which 2nd mesh information comprises, and acquires the information of several planes.

  Next, the second slice information group acquisition unit 113 acquires the second slice information group by the same processing as that described in the first embodiment.

  Then, the deformed object output unit 115 outputs the acquired second slice information group by a process similar to the process described in the first embodiment.

  As a result, the output is changed from FIG. 22A to FIG. 22B, and the incision is opened. Note that the output in FIG. 22B is obtained by applying an incision template to a medical image using an evaluation function (Equation 1) with 6 feature points as a constraint, and then opening the medical image. The output shown in FIG. 22B has the effect that the image is not cut, the incision is in the correct position, and the space inside the chest wall can be confirmed from the incision.

  FIG. 23 shows an output example when the mesh information correction unit 303 does not perform correction. That is, FIG. 23 is an output example when the first standard mesh information prepared in advance is used. According to FIG. 23, the volume is shaved (see the circle in FIG. 23 (a)), the mesh and the image are greatly displaced, and the position of the incision is displaced (see FIG. 23 (b)). Can be confirmed. This is because the standard-form first mesh information prepared in advance does not fit the subject's body, so only the area corresponding to the inside of the template (first mesh information) that does not fit the body is visualized. This is a phenomenon that occurs. In more detail, in FIG. 23, the subject is thicker than the standard body shape, and the three-dimensional image of the subject (or the first mesh information that matches the body of the subject) may be the standard body shape. Since the three-dimensional image (first mesh information) is small, a part of the three-dimensional image is cut off. In addition, when the corrected 1st mesh information fitted to the test subject's body is used, as mentioned above, the one part location of a three-dimensional image is not shaved.

  FIG. 24 is a table for explaining that the average distance error between the feature point and the theoretical value of the evaluation point is calculated by adapting the template to the patient's individual body shape. The “average error of feature points” is an average of distance errors before correction of feature points (for example, nipple positions). In FIG. 24, the average of the distance errors between the plurality of feature points before correction of adult male A and the plurality of theoretical feature points was 19.85 (voxcel), but is corrected to the shape of adult male A. This indicates that the average distance error between the corrected plurality of feature points and the theoretical plurality of feature points is 0.18 (voxcel). “Average error of evaluation points” refers to the average of distance errors between multiple evaluation points before correction and multiple theoretical evaluation points, and multiple evaluation points after correction and multiple theoretical evaluation points. And the average distance error. In FIG. 24, the average of the distance error between the plurality of evaluation points before correction of adult male A and the plurality of theoretical evaluation points was 16.77 (voxcel), but is corrected to the body type of adult male A. This indicates that the average distance error between the corrected evaluation points and the theoretical evaluation points is 10.61 (voxcel). The evaluation point is not a feature point, but is another point on the human body to be evaluated, for example, the position of the umbilicus.

  In FIG. 24, when the mesh information correction unit 303 performs correction processing, the feature points of the adult males A and B substantially match the theoretical values, and the evaluation points are closer to the patient's body shape than before adaptation. I understand. Note that it has been confirmed that if the information processing device 3 is used, the feature points of the adult males A and B can be operated in a few minutes after the input means (for example, a mouse) is specified until the volume operation. .

  As described above, according to the present embodiment, elasticity information is automatically added only by giving 3D voxel information, and elasticity information is output when a user operates a three-dimensional object being output. Further, according to the present embodiment, it is possible to have surgery information, and surgery can be simulated using the surgery information. Moreover, according to this Embodiment, the surgery simulation matched with the patient's body shape is attained.

  Conventionally, in order to make an object (an object such as an organ, a head, a hand, and a leg) operable, the contour of the object is extracted and vectorized from a three-dimensional image acquired by an imaging apparatus such as MRI. This is called meshing. This meshing work required a lot of labor. On the other hand, in the information processing apparatus 3 according to the present embodiment, a mesh is held in advance, and further, feature points (e.g., a nipple or a scapula projection) generally possessed by an imaging target and surgical information (e.g. The incision location and incision direction are kept on the mesh. Therefore, in the information processing apparatus 3, the meshing operation can be largely omitted by applying the feature points, the operation information, and the first mesh information to the three-dimensional image.

  Note that the software that implements the information processing apparatus 3 in the present embodiment is the following program. In other words, this program is slice information that is information configured based on two-dimensional image data obtained by cutting out 3D voxel information, which is a volume texture of a three-dimensional object, from a plurality of planes. Elasticity information, which is information about elasticity, is added to each point constituting a plurality of slice information composed of information of a plurality of points having position information, which is information indicating color, and color information, which is information about color. A plurality of pieces of slice information including position information, color information, and elasticity information, and a slice information group including the slice information group formed by the elasticity information addition unit. An information group output unit; an instruction reception unit that receives an instruction for a predetermined point or region of the slice information group output by the slice information group output unit; A position information acquisition unit that acquires one or more pieces of position information of a point corresponding to the instruction or a point that constitutes a region; And a program for functioning as an elastic information output unit that outputs based on one or more pieces of elastic information acquired by the elastic information acquisition unit.

  In the above program, the instruction receiving unit is an instruction for a predetermined point or region of the output slice information group, receives a surgical instruction that is an instruction related to surgery, and the second mesh information acquisition unit is Based on the operation instruction received by the instruction reception unit and the operation information stored in the storage medium, the first mesh information is deformed, and the second mesh information constituting the deformed shape is acquired as a computer. It is preferable that the program is for functioning.

  In the above program, the surgical information includes incision position information that is information indicating the position of the incision, and the second mesh information acquisition unit is configured to receive the operation instruction and the operation information received by the instruction reception unit. Is a point in the first mesh information based on the incision wound position information, and among the points corresponding to the incision wound position information, the point on the left side of the incision moves to the left side and the right side of the incision wound A certain point is a program for moving the computer to the right side, deforming the second mesh information so that the incision opens, and causing the computer to function as acquiring the second mesh information constituting the deformed shape. Is preferred.

  Further, in the above program, the surgery information includes twist position information that is information on a point or a region where a twisting operation is performed, and the second mesh information acquisition unit includes the surgery instruction received by the instruction reception unit and the Based on the torsional position information included in the operation information, the points in the first mesh information are moved, the points corresponding to the torsional position information are moved, and the second mesh information is deformed so as to twist, and a deformed shape is configured. It is preferable that the second mesh information is a program for causing a computer to function.

  Further, in the above program, the operation information includes pick position information that is information on a point or region where a pinch operation is performed, and the second mesh information acquisition unit includes the operation instruction received by the instruction reception unit and the operation instruction. Based on the picking position information included in the surgery information, the point in the first mesh information is moved, the point corresponding to the picking position information is moved, the second mesh information is deformed to pick, and a deformed shape is configured. It is preferable that the second mesh information is a program for causing a computer to function.

  Further, in the above program, the first mesh information is corrected using anatomical information having human body feature point position information which is information indicating the structure of the human body and information indicating the position of the human body. It is preferable that the program further functions as a mesh information correction unit constituting new first mesh information.

  FIG. 25 shows the external appearance of a computer that executes the program described in this specification to realize the information processing apparatus or the like according to the above-described embodiment. The above-described embodiments can be realized by computer hardware and a computer program executed thereon. FIG. 25 is an overview diagram of the computer system 340, and FIG. 26 is a diagram showing an internal configuration of the computer system 340.

  25, the computer system 340 includes a computer 341 including an FD drive 3411 and a CD-ROM drive 3412, a keyboard 342, a mouse 343, and a monitor 344.

  26, in addition to the FD drive 3411 and the CD-ROM drive 3412, the computer 341 stores an MPU 3413, a bus 3414 connected to the CD-ROM drive 3412 and the FD drive 3411, and a program such as a bootup program. ROM 3415 for storing, RAM 3416 for temporarily storing application program instructions and providing a temporary storage space, and a hard disk 3417 for storing application programs, system programs, and data. Although not shown here, the computer 341 may further include a network card that provides connection to the LAN.

  A program that causes the computer system 340 to execute the functions of the information processing apparatus and the like of the above-described embodiment is stored in the CD-ROM 3501 or the FD 3502, inserted into the CD-ROM drive 3412 or the FD drive 3411, and further the hard disk 3417. May be transferred to. Alternatively, the program may be transmitted to the computer 341 via a network (not shown) and stored in the hard disk 3417. The program is loaded into the RAM 3416 at the time of execution. The program may be loaded directly from the CD-ROM 3501, the FD 3502, or the network.

  The program does not necessarily include an operating system (OS) or a third-party program that causes the computer 341 to execute the functions of the information processing apparatus according to the above-described embodiment. The program only needs to include an instruction portion that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 340 operates is well known and will not be described in detail.

  In the above program, in a transmission step for transmitting information, a reception step for receiving information, etc., processing performed by hardware, for example, processing performed by a modem or an interface card in the transmission step (only performed by hardware). Not included) is not included.

  Further, the computer that executes the program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  In each of the above embodiments, each process (each function) may be realized by centralized processing by a single device (system), or by distributed processing by a plurality of devices. May be.

  Further, the information processing apparatus described in each of the above embodiments is mainly used in surgery simulation, and may be rephrased as a surgery simulation apparatus.

  The present invention is not limited to the above-described embodiments, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, the surgical simulation apparatus according to the present invention has an effect that an appropriate surgical simulation can be performed, and is useful as, for example, a surgical simulation apparatus.

Block diagram of information processing apparatus according to Embodiment 1 A flowchart for explaining the operation of the information processing apparatus 1 A flowchart for explaining the operation of the mesh accumulation process A flowchart for explaining the operation of the elasticity information addition processing Flow chart for explaining the operation of the deformation process Flow chart for explaining the operation of the deformation process Diagram showing a set of medical images The figure which shows the tetrahedron which comprises the three-dimensional object of the deformation | transformation object The figure which shows the example of the positional information which the sampling positional information acquisition part acquired The figure which shows the image at the time of acquiring the same 1st slice information group Diagram showing the same slice information group The figure which shows the second mesh information The figure which shows the slice information group after the deformation | transformation Block diagram of an information processing apparatus according to Embodiment 2 A flowchart for explaining the operation of the information processing apparatus 2 A flowchart for explaining the operation of the elasticity information addition processing Diagram showing the same distance coefficient graph Block diagram of information processing apparatus 3 according to Embodiment 3 Flow chart for explaining the operation of the information processing apparatus The flowchart explaining the operation of the second mesh information acquisition process Diagram explaining the anatomical information The figure which shows the operation at the time of the incision Output example when correction by the mesh information correction unit is not performed The figure which shows the table explaining adapting the same template to the patient's individual figure Overview of the computer system The figure which shows the internal configuration of the computer system

Claims (7)

Position information and color, which is slice information that is information configured based on two-dimensional image data obtained by cutting out 3D voxel information that is a volume texture of a three-dimensional object on a plurality of planes, and is information indicating a position A slice information group storage unit capable of storing a slice information group having a plurality of slice information composed of information of a plurality of points having color information which is information about
A first mesh information storage unit capable of storing first mesh information which is information of a three-dimensional mesh of a three-dimensional object;
Surgery information, which is information related to the operation, including incision position information indicating the position of the incision, twist position information indicating the position to be twisted, or picking position information indicating the position to pick before the operation instruction, which is an instruction regarding the operation, is input. , A surgical information storage unit that stores in advance,
A slice information group output unit for outputting a slice information group in the slice information group storage unit;
An instruction receiving unit that receives an operation instruction that is an instruction related to surgery, which is an instruction for a predetermined point or region of the slice information group output by the slice information group output unit;
The surgery information is read from the surgery information storage unit, and the incision wound position information or the twist position information or the pick position information included in the surgery information matches the point or region indicated by the surgery instruction received by the instruction reception unit. The second mesh information acquisition unit that acquires the second mesh information that deforms the first mesh information and configures the deformed shape,
Based on the second mesh information, from the slice information group storage unit, a first slice information group acquisition unit that acquires a first slice information group that is a plurality of slice information without color information;
A color information determination unit for determining color information of each point corresponding to each point of the 3D voxel information, which is each point of the plurality of slice information constituting the first slice information group;
Based on the color information of each point determined by the color information determination unit, new color information is set for each point of the first slice information group acquired by the first slice information group acquisition unit, and the second slice information group A second slice information group acquisition unit for acquiring
A deformable object output unit for outputting the second slice information group,
The surgical information is
Incision position information that is information indicating the position of the incision, twist position information that is information of a point or region where a twisting operation is performed, or picking position information that is information of a point or region where a pinching operation is performed Have one or more pieces of information
The second mesh information acquisition unit
Based on the surgical instruction received by the instruction receiving unit and the incision wound position information included in the surgical information, the point in the first mesh information is located on the left side of the incision wound among the points corresponding to the incision wound position information. A process in which a point moves to the left side, a point on the right side of the incision moves to the right side, the second mesh information is deformed so that the incision opens, and the second mesh information constituting the deformed shape is acquired. ,
Based on the operation instruction received by the instruction reception unit and the twist position information included in the operation information, the second mesh point is a point in the first mesh information, and the point corresponding to the twist position information is moved and twisted. Processing to deform the mesh information and obtain second mesh information constituting the deformed shape;
Based on the operation instruction received by the instruction receiving unit and the pick position information included in the operation information, the second mesh is a point in the first mesh information, and the point corresponding to the pick position information is moved and picked. A surgical simulation apparatus that performs any one or more of the processes of deforming information and acquiring second mesh information constituting the deformed shape. Anatomy that can store standard anatomical information having human body feature point position information, which is information indicating the structure of the human body, and information indicating the position of the human body part, and anatomical information of the individual undergoing surgery A scientific information storage,
A mesh information correction unit configured to correct the first mesh information by using a difference between the standard anatomical information and the individual anatomical information and configure new first mesh information; ,
The second mesh information acquisition unit
The new first mesh information formed by the mesh information correcting unit is deformed based on the operation instruction received by the instruction receiving unit and the operation information, and second mesh information constituting the deformed shape is acquired. The surgery simulation apparatus according to 1. A sampling position information acquisition unit for acquiring a plurality of sampling position information having position information constituting points for sampling the three-dimensional space of the three-dimensional object;
The surgical simulation apparatus according to claim 1, wherein the first mesh information stored in the first mesh information storage unit includes the plurality of sampling position information. For each point constituting each of the plurality of slice information, elasticity information that is information on elasticity is added, and a plurality of slice information composed of information on a plurality of points having at least position information and elasticity information It further comprises an elastic information adding part to constitute,
The slice information group output unit includes:
Output the slice information group configured by the elastic information adding unit,
The instruction receiving unit
Accepting an instruction for a predetermined point or region of the slice information group output by the slice information group output unit,
A position information acquisition unit that acquires one or more pieces of position information of a point corresponding to the instruction or a point constituting an area;
An elasticity information acquisition unit that acquires one or more pieces of elasticity information paired with one or more pieces of position information acquired by the position information acquisition unit;
An elastic information output unit that outputs one or more pieces of elastic information acquired by the elastic information acquisition unit;
The elastic information adding unit is
Elastic information acquisition means for acquiring elastic information using CT values of points constituting each of the plurality of slice information;
The surgery simulation apparatus according to claim 1 or 2, further comprising: elasticity information adding means for adding the elasticity information acquired by the elasticity information acquiring means to each point constituting the plurality of pieces of slice information. For each point constituting each of the plurality of slice information, elasticity information that is information on elasticity is added, and a plurality of slice information composed of information on a plurality of points having at least position information and elasticity information It further comprises an elastic information adding part to constitute,
The slice information group output unit includes:
Output the slice information group configured by the elastic information adding unit,
The instruction receiving unit
Accepting an instruction for a predetermined point or region of the slice information group output by the slice information group output unit,
A position information acquisition unit that acquires one or more pieces of position information of a point corresponding to the instruction or a point constituting an area;
An elasticity information acquisition unit that acquires one or more pieces of elasticity information paired with one or more pieces of position information acquired by the position information acquisition unit;
An elastic information output unit that outputs one or more pieces of elastic information acquired by the elastic information acquisition unit;
A singularity information receiving unit that receives singularity information that is information indicating one or more singularities that are points of a region with a tumor or a point of a region with a wound or a surgical scar,
The elastic information adding unit is
Further comprising distance acquisition means for acquiring a distance from the one or more singular points;
The elasticity information acquisition means
The surgery simulation apparatus according to claim 1 or 2, wherein the elasticity information is acquired using the distance acquired by the distance acquisition unit so that the elasticity information becomes harder as the distance is smaller. A surgical simulation method realized by a slice information group output unit, an instruction reception unit, a second mesh information acquisition unit, a first slice information group acquisition unit, a color information determination unit, a second slice information group acquisition unit, and a deformed object output unit There,
In the storage medium,
Position information and color, which is slice information that is information configured based on two-dimensional image data obtained by cutting out 3D voxel information that is a volume texture of a three-dimensional object on a plurality of planes, and is information indicating a position A slice information group having a plurality of slice information composed of information of a plurality of points having color information which is information about
First mesh information, which is 3D mesh information of a 3D object,
Surgery instructions, which are instructions related to surgery, are information relating to surgery, and are surgical incision position information indicating the position of the incision, twisting position information indicating the position to be twisted, or surgery information having picking position information indicating the position to pick. Stored before and
A slice information group output step of outputting a slice information group of the slice information group storage unit by the slice information group output unit;
An instruction receiving step for receiving a surgical instruction that is an instruction related to a surgery, which is an instruction for a predetermined point or region of the slice information group output in the slice information group output step by the instruction receiving unit;
The surgical information is read from the storage medium by the second mesh information acquisition unit, and the surgical wound instructions received in the instruction receiving step and the incision wound position information, twist position information, or picking position information included in the surgical information indicate Or a second mesh information acquisition step of determining whether or not the region matches, and if matching, deforming the first mesh information and acquiring second mesh information constituting the deformed shape;
A first slice information group acquisition step of acquiring a first slice information group, which is a plurality of slice information having no color information, from the storage medium based on the second mesh information by the first slice information group acquisition unit. When,
A color information determination step of determining color information of each point corresponding to a point of the 3D voxel information which is each point of a plurality of slice information constituting the first slice information group by the color information determination unit;
Based on the color information of each point determined in the color information determination step, the second slice information group acquisition unit newly adds each point of the first slice information group acquired in the first slice information group acquisition step. A second slice information group acquisition step for setting the correct color information and acquiring the second slice information group;
A deformed object output step of outputting the second slice information group by the deformed object output unit;
The surgical information is
Incision position information that is information indicating the position of the incision, twist position information that is information of a point or region where a twisting operation is performed, or picking position information that is information of a point or region where a pinching operation is performed Have one or more pieces of information
The second mesh information acquisition step includes
Based on the surgical instruction received in the instruction receiving step and the incision position information included in the surgical information, the left side of the incision is a point in the first mesh information, and among the points corresponding to the incision position information The point on the left side moves to the left side, the point on the right side of the incision moves to the right side, the second mesh information is deformed so that the incision opens, and the second mesh information constituting the deformed shape is acquired. processing,
Based on the operation instruction received in the instruction reception step and the twist position information included in the operation information, the point in the first mesh information is moved and the point corresponding to the twist position information is moved and twisted. The process of deforming the two mesh information and acquiring the second mesh information constituting the deformed shape,
Based on the surgical instruction received in the instruction receiving step and the picking position information included in the surgical information, the second point is a point in the first mesh information, and the point corresponding to the picking position information is moved and picked. A surgical simulation method for performing any one or more of the processes of deforming mesh information and acquiring second mesh information constituting the deformed shape. In the storage medium,
Position information and color, which is slice information that is information configured based on two-dimensional image data obtained by cutting out 3D voxel information that is a volume texture of a three-dimensional object on a plurality of planes, and is information indicating a position A slice information group having a plurality of slice information composed of information of a plurality of points having color information which is information about
First mesh information, which is 3D mesh information of a 3D object,
Surgery instructions, which are information related to surgery, are surgical incision position information indicating the position of the incision, or twisting position information indicating the position to be twisted, or surgery information having picking position information indicating the position to be picked. Stored before and
Computer
A slice information group output unit for outputting a slice information group in the slice information group storage unit;
An instruction receiving unit that receives an operation instruction that is an instruction related to surgery, which is an instruction for a predetermined point or region of the slice information group output by the slice information group output unit;
The surgery information is read from the surgery information storage unit, and the incision wound position information or the twist position information or the pick position information included in the surgery information matches the point or region indicated by the surgery instruction received by the instruction reception unit. The second mesh information acquisition unit that acquires the second mesh information that deforms the first mesh information and configures the deformed shape,
Based on the second mesh information, from the slice information group storage unit, a first slice information group acquisition unit that acquires a first slice information group that is a plurality of slice information without color information;
A color information determination unit for determining color information of each point corresponding to each point of the 3D voxel information, which is each point of the plurality of slice information constituting the first slice information group;
Based on the color information of each point determined by the color information determination unit, new color information is set for each point of the first slice information group acquired by the first slice information group acquisition unit, and the second slice information group A second slice information group acquisition unit for acquiring
A program for functioning as a deformed object output unit that outputs the second slice information group, the surgery information,
Incision position information that is information indicating the position of the incision, twist position information that is information of a point or region where a twisting operation is performed, or picking position information that is information of a point or region where a pinching operation is performed Have one or more pieces of information
The second mesh information acquisition unit
Based on the surgical instruction received by the instruction receiving unit and the incision wound position information included in the surgical information, the point in the first mesh information is located on the left side of the incision wound among the points corresponding to the incision wound position information. A process in which a point moves to the left side, a point on the right side of the incision moves to the right side, the second mesh information is deformed so that the incision opens, and the second mesh information constituting the deformed shape is acquired. ,
Based on the operation instruction received by the instruction reception unit and the twist position information included in the operation information, the second mesh point is a point in the first mesh information, and the point corresponding to the twist position information is moved and twisted. Processing to deform the mesh information and obtain second mesh information constituting the deformed shape;
Based on the operation instruction received by the instruction receiving unit and the pick position information included in the operation information, the second mesh is a point in the first mesh information, and the point corresponding to the pick position information is moved and picked. A program for causing a computer to function so as to perform any one or more of the processes of deforming information and acquiring second mesh information constituting a deformed shape.

Images