JPH05123327A - Operation simulation system - Google Patents

Abstract

PURPOSE:To create the reality of an operation including proper image display by providing a superposing display means for superposing and displaying a pseudo operating tool including its movement on an image, and an artificial reality creating means for giving a repelling force to the pseudo operating tool. CONSTITUTION:A three-dimensional position input device 9 is connected to a pseudo operating tool such as a surgical knife, a drill or the others, and an operator can hold and three-dimensionally operate the pseudo operating tool. A virtual space forming part 13 forms a virtual operating tool image having the form and motion corresponding to the operated pseudo operating tool, and it is superposed on a synthetic image and three-dimensionally displayed by a virtual space display part 12. An artificial reality generating part 8 determines the resistance to the pseudo operation tool held by the operator by the contact between the virtual operating tool and a virtual operation and outputs it to a force feedback device 10, which then gives a resistance according to the input value to the pseudo operating tool held by the operator. The frequency and amplitude of a sound estimated at the time of an actual operation are also calculated and outputted to a voice output device 11, which then generates an operation sound according to the input value.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a surgery simulation system using medical images, and more particularly to a surgery simulation system capable of creating a sense of reality in surgery.

[0002]

2. Description of the Related Art Although a human body has a three-dimensional structure, an initial medical image is a two-dimensional projection image as typified by an X-ray photograph, and then is represented by an X-ray CT, MRI, ultrasonic wave, etc. Three-dimensional information has come to be obtained from the medical images.

However, since these medical images are photographed as tomographic images, each image is also a two-dimensional image, and a three-dimensional image can be obtained by photographing a plurality of these tomographic images and combining them. .. Therefore, in most cases, doctors currently observe multiple tomographic images and construct a three-dimensional image in their heads, but this requires skill and is difficult to quantitatively grasp. Is.

On the other hand, a three-dimensional display of a three-dimensional image is performed. In particular, displaying three-dimensionally in a virtual space is a well-known technique, and is popular in the field of CG (computer graphics). There are many documents regarding three-dimensional display, but the latest one is the Institute of Electrical Engineers of Japan 1
A special issue was published in the February 991 issue, which also mentions its application to medical imaging.

For three-dimensional display in a medical image, there is a method using binocular parallax and motion parallax, but surface display is often used. However, a problem with surface display is that the image is binarized. That is, since the binarized image is not a grayscale image, it is not suitable for diagnosis.

Therefore, many three-dimensional display of medical images is applied to a surgery simulation system. For example, 1) a tumor, blood vessel, important tissue, etc. are three-dimensionally displayed and their position and size are grasped. 2) A surgery simulation system, such as performing a surgery simulation when cutting or moving a part of bone, For example, “Medical Imaging Technology (Medical Imagin
g Technology) ”March 1989 and June 1990
It is reported in the monthly issue.

[0007]

However, the above-mentioned surgery simulation system is still at a level of image display and cannot be said to be a full-scale surgery simulation system. This is because: 1) Various images such as CT images and MRI images have distinctive features (for example, CT images show clearly projected bones, and MRI images show clearly projected organs). It is difficult to photograph all tumors, blood vessels, important tissues, etc. in a visible state during actual surgery.
For that purpose, it is necessary to combine and display a plurality of types of images while making the most of the advantages of each image (hereinafter referred to as “composite display”), but such a system does not exist.

Recently, PACS (Picture Archiving an
d Communication System; medical image storage communication system)
Therefore, although a plurality of types of images are collectively displayed, in PACS, the images are displayed independently for each type, and not the composite display here. The composite display required for the surgery simulation is to combine and display a plurality of types of images into one image.

2) The display method is fixed, and the human interface in the display is not sufficient (the operator cannot change the display method).

3) The system incorporates a surgical instrument such as a scalpel and a drill, and the simulated surgical instrument is displayed by being superimposed on a human body image, and the human body tissue touching the resistance and human body tissue transmitted through the simulated surgical instrument is touched. You cannot experience a surgical simulation with a sense of reality, such as the generation of sound when you hit. As a realistic simulation system, a simulation system for flight training of an aircraft is well known, but a simulation system applied to medical use has not yet existed.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a surgery simulation system capable of creating a sense of reality of surgery including appropriate image display.

[0012]

In order to solve the above-mentioned problems, the present invention is provided with a simulated surgical tool which can be grasped and moved by an operator, and an image including each tissue of an operation target portion from images of a plurality of modalities. Image synthesizing means for synthesizing, and a three-dimensional image displaying means for three-dimensionally displaying the synthesized image,
Superimposition display means for superimposing and displaying the simulated surgical tool on the image including its movement, and when the simulated surgical tool comes into contact with a tissue in the superimposed display means, a sound corresponding to the hardness of both is generated, and the simulated surgical tool is produced. Provided is a surgery simulation system including means for creating an artificial reality that gives repulsive force to

[0013]

The surgical simulation system of the present invention synthesizes images of a plurality of modalities at the same position, scale, and deviation angle with respect to human body tissues having high and low contrasts depending on modalities.
You can clearly see the three-dimensional human tissue image. In addition, the surgical simulation system of the present invention is equipped with an actual-sized simulated surgical tool and displays an image together with human tissue.
Then, when the operator grasps and moves it, the simulated surgical tool on the image moves and touches the human body tissue,
At that time, based on the relationship between the types of simulated surgical tools such as scalpels and drills and human tissues such as bones and soft tissues, a sound expected during actual surgery is generated, and repulsive force is applied to the simulated surgical tools grasped by the operator.

[0014]

Embodiments of the present invention will be described below with reference to the accompanying drawings.

FIG. 1 is a block diagram of a surgery simulation system 1 according to an embodiment of the present invention. That is, the image synthesizer 3, the massively parallel processing device 4 and the expert system 5 are connected to the controller 2 and the atlas database 6
Is also connected to the control unit 2 via the important tissue recognition unit 7. The artificial reality generating unit 8 is also connected to the control unit 2. The artificial reality generating unit 8 is also connected to the three-dimensional position input device 9, the force feedback device 10, and the voice output device 11. In addition, the virtual space display unit 12
Also connects to the artificial reality generating unit 8 via the virtual space creating unit 13.

The image synthesis unit 3 includes a plurality of modalities (MR
Image data of I, CT, US [ultrasound], NM [nuclear medicine], etc.) is input for synthesis, but on the other hand, an atlas that digitizes and stores an ideal three-dimensional anatomy of the human body. The anatomical chart data is output from the database 6 to the important tissue recognition unit 7. The important tissue recognizing unit 7 recognizes the important tissue of the human body taken over a plurality of types (modality) images such as tumor, blood vessel, and organ from the inputted anatomical chart data, and controls the important tissue. 2 to the image compositing unit 3.

The image synthesizing section 3 is composed of the sent important organization and an expert system (a system showing a question answering performance in which a database of knowledge about human tissue obtained by learning is stored) and an expert system 5 executed via the control section 2. Based on the response, the positional relationship between the images of a plurality of modalities is grasped, and these images are combined. When a large amount of data processing is required in the important organization recognition unit 7, the expert system 5, and the image synthesis unit 3, the mass processing is performed by the massively parallel processing unit 4 via the control unit 2 at high speed. The calculation is executed. Therefore, the surgery simulation system 1 according to the present embodiment can obtain a necessary composite image at high speed.

The composite image formed by the image composition section 3 is passed through the control section 2 and the artificial reality generating section 8 to the virtual space creating section 1
3 and the data is converted into a three-dimensional image viewed from the designated line-of-sight direction. Then, it is three-dimensionally displayed in the virtual space by the subsequent virtual space display unit 12.

On the other hand, the three-dimensional position input device 9 is connected to a simulated surgical instrument such as a scalpel or a drill, and an operator can grasp (move) the simulated surgical instrument and move (operate) it three-dimensionally. Further, the virtual space creation unit 13 creates a virtual surgical tool image having a shape and movement corresponding to the simulated surgical tool operated by the operator using the three-dimensional position input device 9, and the virtual space display unit 12 creates a composite image of the virtual surgical tool image. 3D display is superimposed on. Therefore, the operator looks at the composite image and the virtual surgical instrument image on the virtual space display unit 12, and the image of the virtual surgical instrument comes into contact with the synthetic image of the target human body tissue so as to perform an operation according to the surgical purpose. Then, the three-dimensional movement of the simulated surgical tool is reflected on the virtual surgical tool image on the virtual space display section 12 via the artificial reality generating section 8 and the virtual space creating section 13.

In actual surgery, when a surgical tool such as a scalpel or a drill touches human body tissue and operates, the operator feels resistance depending on the hardness of the human body tissue, and sometimes the contact sound, Impact noise (such as the sound of cutting a bone with a drill) is generated. Therefore, the artificial reality generation unit 8 obtains the amount of resistance (force feedback) to the simulated surgical tool grasped by the operator from the relationship between the virtual surgical tool and the human body tissue that is divided into the virtual surgical tool. Output to the force feedback device 10. The force feedback device 10 applies a resistance force corresponding to the input value from the artificial reality generating unit 8 to the simulated surgical instrument held by the operator.

Further, the artificial reality generating section 8 similarly produces a parameter (frequency) of sound that is expected to be generated during an actual operation due to the relationship between the virtual surgical tool and the human body tissue that is separated into the virtual surgical tool. , Amplitude, etc.) are also calculated and output to the audio output device 11. The voice output device 11 generates an operation sound according to the input value.

Next, FIGS. 2A and 2B and FIG.
With reference to (A) to (E), a method of image composition performed by the image composition unit 3 will be described.

Since the images of each modality input to the image synthesizing unit 3 have different scales, shooting angles, and shooting positions,
Images cannot be combined unless they match. Therefore, in the surgery simulation of the present embodiment, a scale between images, a shooting angle, a synthetic reference marker serving as a reference of a shooting position is used, and a scale between images, a shooting angle,
The images are combined after matching the shooting positions. Hereinafter, a case where a CT image and an MRI image are combined will be described as an example.

The synthetic reference marker 20 shown in the perspective view of FIG. 2A and the plan view of FIG. 2B is an L-shaped container 21 made of aluminum, and the hollow portion is filled with water 22. Aluminum has a high so-called CT number (concentration) during X-ray CT imaging, and conversely, aluminum has a low concentration and high water concentration in MRI images. Therefore, the synthetic reference marker 20 becomes a high-contrast image even in the MRI apparatus in the X-ray CT apparatus.

To the image synthesizer 3, the CT image 23 shown in FIG. 3 (A) and the MRI image 24 shown in FIG. 3 (C) obtained by photographing the synthesis reference marker 20 are input. In the figure,
Reference symbols ROI1 and ROI2 are different regions of interest at the same imaging position (slice). Therefore, in the image synthesizer 3, first, the deviation of the angle between the container 21 of the composition reference marker 20 and the water 22 contained in the container 21 in the CT image 23 is checked, and the image is rotated counterclockwise by the deviation angle θ, and then, as shown in FIG. The CT image 27 shown in (B) is obtained.

Next, the image synthesizer 3 uses C in FIG. 3 (B).
The inside dimension a of the container 21 in the T image 23 and FIG.
In the water 22 of the MRI image 24 of FIG. Since a> b here, the MRI image 24 of FIG. 3C is magnified a / b times to obtain the MRI shown in FIG.
Obtain the image 28. As a result, the inner dimension a of the container 21 in FIG. 3 (B) and the water dimension b ′ in FIG. 3 (D) become equal.

Finally, since the CT image 27 and the MRI image 28 have the same scale and photographing angle, the image synthesizer 3 superimposes them and synthesizes them to obtain a synthesized image 30 shown in FIG. 3 (E). ..

Next, an example of three-dimensional display on the virtual space display unit 12 will be described. FIG. 4A is a surface display (a type of three-dimensional display) of the entire head that is the surgical target. This screen can be rotated in any direction and viewed from any direction.

In actual surgery, the surface of the head is incised, so during simulation,
I want an image of the inside of the head with the surface incised. In this embodiment, as shown in FIG. 4B, a hole having an arbitrary size can be formed from an arbitrary direction. For the image at the back of the hole, a plurality of types of display methods are prepared below, and these can be selected by the operator as appropriate. The surface other than the hole is often displayed, but other display methods are possible.

In this embodiment, three types of display methods are possible for the image at the back of the hole. 1) Grayscale image of the tomographic plane shown in FIG. 5A: Gray level grayscale display of the tomographic plane is performed for the entire region behind the hole. The tomographic plane can be set arbitrarily in the depth direction of the hole. Since the grayscale image is excellent in spatial resolution and density resolution, it is possible to observe in detail all the tissues at the position of the predetermined depth.

2) Pseudo three-dimensional image of the tissue shown in FIG. 5B: A specific tumor, tissue or the like is displayed in a pseudo three-dimensional manner as if a person looked through a hole. If the tumor or tissue to be displayed is designated in advance, the important tissue recognition unit 7 captures the tumor or tissue and creates a three-dimensional image in advance. This 3
When creating a three-dimensional image, a medical image of a modality that allows easy recognition and observation of the tissue is used. Since the pseudo three-dimensional image is suitable for grasping the entire tissue, the positional relationship between the tumor and blood vessels can be confirmed.

3) Gray-scale image of the tissue shown in FIG. 5C: The tumor and tissue designated as in the case of the above 2) pseudo three-dimensional image are displayed in the same gray-scale as in 1). This is suitable for confirming the tumor and the like in more detail after confirming the positional relationship between the tumor and the blood vessel by the pseudo three-dimensional image of 2).

Each of the above three display methods has advantages and disadvantages. Therefore, although only one type of display method as in the past is not suitable for surgery simulation,
According to this embodiment, there is an advantage that an appropriate display method can be selected depending on the case.

Next, a semi-transparent three-dimensional display (semi-transparent display)
Will be described. One of the pseudo three-dimensional displays is semi-transparent display. The semi-transparent display is a display method in which, for example, the bone is semi-transparent and the internal tumor is displayed on the surface so that the internal tumor can be observed through the bone. In the present embodiment, the semi-transparent portion is designated not by the tissue but by an arbitrary depth from the surface.

Therefore, the previous FIG. 4 (A) can be regarded as a semi-transparent display in which the transparency of the frontal bone is 0, and FIG. 6 shows the relationship between the depth and the transparency seen by the observer in FIG. 4 (A). Is shown in the graph. That is, in this case, since the transparency is 0 at a depth shallower than the front skull, only the surface of the front skull can be seen, and the inside of the skull cannot be observed.

Further, in FIG. 5A, the upper hemisphere can be regarded as a transparent semitransparent display when the entire skull and its internal tissue are regarded as a sphere. FIG. 7 shows this FIG.
(A) is a graph showing the relationship between the depth and the transparency as seen by the observer. In this case, the transparency is 0 up to the front skull and the central tissue, and the deeper part is opaque, so it is almost in the center. The cross section can be observed.

FIG. 8 is a graph showing the relationship between depth and transparency when the transparency is changed stepwise. In this figure, since it is transparent up to the depth of P1 shallower than the front skull, the portion up to the depth P1 is not displayed. On the other hand, from the depth P1 to the depth P2 (a position which has advanced from the frontal temporal bone toward the center part to some extent), the transparency is almost half, so that the tissue in the meantime is displayed semitransparently. When the depth is deeper than P2, the transparency is 0.
The cross section is displayed at position 2.

In the surgery simulation system of this embodiment, the two depths P1 (semi-transparent) and P2 (transparency 0) can be set to arbitrary depths, and the virtual space creation unit 13 and the virtual space display unit 12 can set the depths. Create and display a semi-transparent image according to the settings. Therefore, a tissue having an arbitrary depth can be displayed with an arbitrary transparency according to the purpose of observation.

FIG. 9 is a graph showing the relationship between the depth and the transparency when the transparency between the depth P1 and the depth P2 is variable. The transparency continuously decreases from the depth P1 to the depth P2. In this case, the positions of P1 and P2 and the respective transparency levels T1 and T2 can be set arbitrarily.

By the way, in the three-dimensional display such as the surface display and the semi-transparent display described above, the image is rotated and displayed in correspondence with the various line-of-sight directions so that the lesions and important tissues (blood vessels, nerves, organs) in the body are displayed. The position, the size, and the positional relationship between the two can be grasped, and the display is useful for medical purposes.
In the rotation display, the conventional three-dimensional image data is used.
It was rotating around the center coordinates of the data.

However, in this rotation method, when the region of interest (lesion, important tissue, etc.) is distant from the center coordinates, the region moves with rotation, so the state around the region of interest is observed. It gets harder.

Therefore, in the surgery simulation system of this embodiment, as shown in FIG. 10A, when a lesion 41 in the skull 40 is set as a region of interest, this lesion 41 is set as the center point of the rotation display. In the figure, reference numeral 42
Is a blood vessel around the lesion 41, and the center of rotation is displayed by a cursor 43 surrounded by a dashed circle.

For this purpose, the operator of the surgery simulation system first designates the lesion 41 as the attention area from the three-dimensional position input device 9 in FIG. Then, the deviation (x-coordinate, y-coordinate) in the three-dimensional space between the coordinates of the attention area (lesion 41) and the center coordinates on the three-dimensional image data display.
In the z coordinate, it is assumed that they are displaced by l, m, and n, respectively).

Next, the operator inputs various line-of-sight directions (rotation angles) from the same three-dimensional position input device 9 (for example, a three-dimensional joystick. At this time, it is possible to perform enlargement or reduction with rotation. In the simulation system, each time the rotation angle is input, the following three-dimensional affine transformation formula

[0045]

[Equation 1]

(Here, x, y, z are the images 1 before the rotational movement.
Pixel x coordinate, y coordinate, z coordinate, b, c, d, f, g,
h represents a rotation angle parameter, and a, e, and i represent enlargement or reduction conversion parameters. ), Rotation is performed with the lesion 41 as the center of rotation (inside the cursor 43), and a rotated image as shown in FIG. 10B, for example, is obtained.

Therefore, according to the surgery simulation system of the present embodiment, since it is possible to rotate and display the region of interest such as a lesion, it is possible to easily carry out a surgery plan and a surgery simulation while viewing the image.

The rotation display in this embodiment is particularly applicable to stereotactic surgery (stereotactics). Open-cerebral position surgery is a surgery in which a small hole is made in the skull and a needle-shaped therapeutic device is inserted through it to inject a drug solution or suck up bleeding.

What is important in this fixed-cerebral-position surgery is not only to make sure that the above-mentioned therapeutic instrument reaches the intended position but also to minimize the influence on other tissues in the route to the arrival. is there. According to the surgery simulation system of the present embodiment, various rotation displays are used to check the path to the tissue at the center of rotation and the effect on other tissue when passing through the path, and the effect on other tissue is minimized in advance. It is possible to easily find out the therapeutic instrument insertion path.

Further, in the surgery simulation system of the present invention, in addition to visually appealing three-dimensional image display, it is possible to create sounds, responsiveness, etc. by a scalpel, a drill, etc. experienced by an operator in actual surgery. .. Therefore, in this embodiment, therefore, based on (1) means for relating the gray value of the three-dimensional image data to the hardness (bone etc.) of the tissue, (2) the hardness of the tissue to be operated. , Means for synthesizing sounds generated during actual surgery and putting them in the operator's ear, and (3) based on the hardness of the tissue to be operated,
The response to the surgical tools generated during the actual surgery is combined to incorporate the three means of making the operator feel.

FIG. 11 shows a general outline of the surgery simulation system according to this embodiment. In the figure, a data processing unit 50 is a control unit 2, an image synthesizing unit 3, a massively parallel processing unit 4, an expert system 5, an atlas database 6, an important organization recognition unit 7, and an artificial reality generating unit 8 in FIG. Also, the virtual space creating unit 13 is included, and image data of a plurality of modalities (CT, MRI, etc.) are input and various operation data related to this surgery simulation are also input.

On the other hand, the data processing unit 50 is connected to the virtual space display unit 12 and the manipulator unit 52 including the simulated surgical tool 51 which the operator OP holds. On the virtual space display unit 12, a front view (an image in which the patient P and the simulated surgical instrument 51 are superimposed) indicated by reference numeral FD is displayed. Here, the virtual space display unit 12 is actually the console 5 in front of the operator OP.
It is the display 55 that is slightly tilted in 4
The front view FD is displayed in this display 55.

FIG. 12 is a block diagram of the manipulator unit 52. That is, the manipulator unit 52 is a three-dimensional position detecting device 57 connected to the simulated surgical tool 51, and a voice generating unit 58 and a force generating unit 59 connected to the data processing unit 50 together with the three-dimensional position detecting device 57, and a voice generating unit. Part 5
The force generator 59 is connected to the simulated surgical tool 51.

Therefore, as will be described with reference to FIG. 1, the simulated surgical tool 51 and the three-dimensional position detecting device 57 are connected to the three-dimensional position input device 9, the voice generating section 58 and the speaker 60 are connected to the voice output device 11, and the force is applied. The generator 59 and the simulated surgical tool 51 correspond to the force feedback device 10.

The three-dimensional position detecting device 57 detects the position and movement of the simulated surgical tool 51 and sends it to the data processing section 50.
Therefore, the data processing unit 50 changes / displays the image of the simulated surgical tool 51 in the virtual space in accordance with the movement of the simulated surgical tool 51, and together with this, the pixels on the image in contact with the simulated surgical tool 51. Value and movement of simulated surgical tool 51
9 and the voice generator 58.

The sound generation unit 58 associates the pixel value sent from the data processing unit 50 with the movement of the simulated surgical operation tool 51 with the type of the simulated surgical operation tool 51 to generate a sound (sound quality, volume) during the operation. Etc.) through the speaker 60.

In the force generator 59, the data generator 5
The pixel values sent from 0 and the movement of the simulated surgical tool 51 are associated with the type of the simulated surgical tool 51, and the force (repulsive force, vibration) generated during the surgery is driven through the simulated surgical tool 51 to generate the force. To do.

The outputs (volume, sound quality, repulsive force, vibration, etc.) of the sound generator 58 and the force generator 59 can be increased or decreased by operating a switch (not shown).

In the sound generating unit 58 and the force generating unit 59 described above, the sound or repulsive force generated when processing a hard material is generally larger than that of a soft material. The sound and repulsive force expected during surgery are generated based on the value.

In the graph of FIG. 13A, the horizontal axis is the pixel value,
The vertical axis represents the volume. In the X-ray CT image, the distribution of the X-ray absorption value of the subject is converted into a value called the Hansfield number (air = −1000, water = 0). Human bone tissue (containing a large amount of Ca) absorbs X-rays well and exhibits a Hansfield number of about 1000, while other soft tissues have a Hansfield number of about 40. Higher tissues tend to be harder. Therefore, in this embodiment, the volume and the repulsive force corresponding to the hardness of the tissue are generated from the pixel value (-2048 to 2047) based on the Hansfield number.

That is, since the operation sound generated during the operation is mainly the sound of breaking a hard object, that is, the sound generated when processing the bone, absorption of bone tissue having a pixel value of about 500 to 1000 is absorbed. Sound is generated in the value band, and the volume output is increased as the pixel value increases. At this time, it is possible to prepare a conversion table according to the type of surgical tool and make changes such as increasing the tone color when processing a hard tissue.

On the other hand, with respect to the magnitude of the repulsive force shown in FIG. 13 (B), it is necessary to process a hard object during actual surgery, and even soft tissue has a touch if touched.
Therefore, a force is generated at the pixel value (about 0 to 1000) of the tissue to be operated, and the output is gradually increased as the pixel value increases. When processing hard bones,
It also causes vibration. This prepares a conversion table according to the type of surgical tool, adjusts the amplitude of vibration, etc., and synthesizes it into a repulsive force.

[0063]

As described above, according to the surgery simulation system of the present invention, it is possible to clearly see the same three-dimensional human tissue image as seen during actual surgery.
Moreover, a sound expected at the time of actual surgery is generated and a repulsive force is applied to the simulated surgical tool held by the operator, so that an extremely realistic and effective surgical simulation can be performed.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a surgery simulation system according to an embodiment of the present invention.

2A and 2B are a perspective view and a plan view of a synthetic reference marker, respectively.

3A is a diagram showing a CT image at a certain slice position, FIG. 3B is a diagram showing a CT image obtained by rotating the CT image of FIG. 3A by a shift angle, and FIG. 3C is a CT image of FIG. A diagram showing an MRI image at the same slice position as that of (D), a diagram showing an enlarged MRI image of the MRI image of (C),
(E) is a figure which shows the image which combined the CT image of (B) and the MRI image of (D).

FIG. 4A is a surface display image of the entire head, and FIGS.
(H) is a figure which shows the surface display image when the hole of various sizes is punched from various directions in the head, respectively.

5A shows a grayscale image of a tomographic plane of the head, FIG. 5B shows a pseudo three-dimensional image of a tissue, and FIG. 5C shows a grayscale image of a tissue of the head.

FIG. 6 is a graph showing the relationship between depth and transparency as viewed by the observer in FIG.

FIG. 7 is a graph showing the relationship between depth and transparency as viewed by the observer in FIG.

FIG. 8 is a graph showing the relationship between the depth and the transparency when the transparency is changed stepwise in the tomographic image of the head tomographic plane.

FIG. 9 is a graph showing the relationship between depth and transparency when the transparency between two depth positions is continuously changed in a tomographic image of the tomographic plane of the head.

10A is a diagram showing a three-dimensional image of a skull including a lesion, and FIG. 10B is a diagram showing an image obtained by rotating the image of FIG. 10A around the lesion.

FIG. 11 is a configuration diagram of the surgery simulation system including a partial perspective view of a console.

FIG. 12 is a configuration diagram of a manipulator unit in the surgery simulation system.

FIGS. 13A and 13B are graphs showing the relationship between pixel value, volume, and repulsive force, respectively.

[Explanation of symbols]

 3 Image Synthesis Section 6 Atlas Database 7 Important Tissue Recognition Section 9 3D Position Input Device 10 Force Feedback Device 11 Voice Output Device 12 Virtual Space Display Section 20 Synthesis Reference Marker 41 Lesion 51 Surgical Tool

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Hitoshi Sato 1385-1 Shimoishigami, Otawara-shi, Tochigi Stock company, Toshiba Nasu factory (72) Inventor Kazuhiro Tamura 1385-1, 1-5 Shimoishi, Otawara, Tochigi Cal Engineering Co., Ltd.

Claims (6)

[Claims] 1. An image synthesizing means for synthesizing an image including each tissue of an operation target portion from images of a plurality of modalities, comprising a simulated surgical tool which an operator can grasp and move, and the synthesized image three-dimensionally. A three-dimensional image display means for displaying, a superimposing display means for superimposing and displaying the simulated surgical tool on the image including its movement, and when the simulated surgical tool touches a tissue in the superimposing display means, it corresponds to the hardness of both. Surgery simulation system equipped with an artificial reality creating means for producing a sound that makes a repulsive force on a simulated surgical tool. 2. The image synthesizing method used in the surgery simulation system according to claim 1, wherein (1) preparing a synthetic reference marker including a portion displayed with sufficient contrast in images of a plurality of modalities. And (2) obtaining images of a plurality of modalities for the subject including the composite reference marker, and (3) shifting the images of the plurality of modalities in size and angle of the composite reference marker in these images. Rotation, reduction to eliminate
An image synthesizing method including a step of superimposing while enlarging. 3. The three-dimensional image display means includes means for displaying a surface of an operation target, means for making a hole of an arbitrary size in an arbitrary direction on an arbitrary surface of the operation target, and the inside of the holes. A means for displaying an image in any of three-dimensional display modes such as a grayscale image of a tomographic plane at the inner part of a hole, a pseudo three-dimensional image of a predetermined tissue in the hole, and a grayscale image of a tomographic plane of a predetermined tissue in the hole Item 1. The surgery simulation system according to item 1. 4. The surgery simulation system according to claim 1, wherein the three-dimensional image display means includes means for semi-transparently displaying an operation target from a near side to an arbitrary depth with an arbitrary transparency. 5. The surgery simulation system according to claim 1, wherein the three-dimensional image display means includes means for displaying the surgery target image by rotating it by an arbitrary angle about an arbitrary point therein and displaying it. 6. The artificial reality creating means associates the type of simulated surgical tool, the position and movement amount of the simulated surgical tool, and the pixel value of the human body tissue with which the simulated surgical tool is in contact with each other so as to correlate the simulated surgical tool and the human body tissue. The surgery simulation system according to claim 1, further comprising means for generating an appropriate sound and repulsive force corresponding to the contact.