JP2016168078A - Medical observation support system and 3-dimensional model of organ - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a medical observation support system and a 3-dimensional model of an organ in which observation support with high practicality by intuitive operation is achieved.SOLUTION: A medical observation support system includes: a 3-dimensional model 16 of an organ created by a 3-dimensional molding device based on 3-dimensional image data of the organ obtained previously; a position determination unit 62 for determining a model top position indicated by an indication unit of a finger-tip 70 or the like of a practitioner in the 3-dimensional model 16; an image display unit 50 for generating and displaying an image of the organ, based on the 3-dimensional image data of the organ; and an association control unit 64 for associating the model top position with a position in the image of the organ displayed by the image display unit 50 or related information related to the position. According to the above configuration, the operation of the indication unit for indicating the 3-dimensional model 16 can be reflected in the image of the organ displayed by the image display unit 50 by using the 3-dimensional model 16 of the organ as an input interface.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a medical observation support system and a three-dimensional model of an organ that support observation of an organ in a subject, and more particularly, to an improvement for realizing highly practical observation support by an intuitive operation.

  With the recent development of medical imaging apparatuses, medical images such as three-dimensional virtual images have been widely used in diagnosis and surgery. In such a technique, image data of an organ imaged in advance by a CT (Computed Tomography) apparatus or a magnetic resonance imaging (MRI) apparatus, or a computer graphics technique based on such image data is used. From the three-dimensional virtual image or the like reproduced on the screen, the anatomical structure of the organ in the subject can be grasped.

  In recent years, a three-dimensional molding apparatus that creates a three-dimensional object (three-dimensional object) based on three-dimensional data has rapidly developed and attracted attention. An example is a so-called three-dimensional molding machine (rapid prototyping) or a three-dimensional printer (3D printer). In particular, according to the three-dimensional molding apparatus capable of creating the three-dimensional object using a transparent material, with respect to an organ model, for example, a lesion site such as a blood vessel or a tumor inside the organ is colored and a surrounding tissue is made of a transparent material. By configuring each of them, it is possible to create a three-dimensional model in which blood vessels, tumors, and the like inside the organ can be visually recognized from the outside. According to such a technique, it is possible to create a three-dimensional model as a so-called made-to-order model reflecting the characteristics of each organ of a subject, and use it for diagnosis and surgery support (see, for example, Non-Patent Document 1).

Igami T, Nakamura Y, et al.? Application of a Three-dimensional Print of a Liver in Hepatectomy for Small Tumors Invisible by Intraoperative Ultrasonography: Praliminary Experience. ″ World Journal of Surgery, _DOI.10.1007 / s00268-014-2740-7 , 2014

  In the medical image display technique, for example, in order to view a three-dimensional image from a desired direction, the three-dimensional image is moved by instructing rotation about a plurality of axes set in advance in the three-dimensional image space. Alternatively, a relatively complicated procedure such as redrawing by designating the viewpoint position in the three-dimensional image space is required. In the three-dimensional virtual image, processing such as quantitative measurement or display of a cut surface is easily realized. For example, during surgery, it is necessary to measure the length of a blood vessel in the vicinity of the site to be excised, but in the three-dimensional virtual image, by specifying the target section on the computer screen, Calculation of the length of a section, display of a calculation result, etc. are performed. However, such designation also requires, for example, the position in the three-dimensional image to be performed using an input device such as a mouse, which is a relatively complicated procedure, or the two-dimensional plane on which the three-dimensional image is displayed. In the display, it may be difficult to accurately input a desired position.

  The three-dimensional model of the organ created by the three-dimensional molding apparatus is useful for grasping the shape of the organ and its three-dimensional feeling. However, with such a three-dimensional model of an organ, operations such as quantitative measurement or display of a cut surface are difficult. For example, when measuring the length of a blood vessel or the like in the vicinity of a site to be excised at the time of surgery, a method such as applying a length measurement wire to the three-dimensional model of the organ may be used. Work and poor practicality. On the other hand, when a three-dimensional image is used, it is considered relatively easy to calculate the distance on the three-dimensional image, but it is difficult to input which point is to be measured as in the case described above.

  The present invention has been made against the background of the above circumstances, and the object of the present invention is to provide a medical observation support system and a three-dimensional model of an organ that provide highly practical observation support through intuitive operations. There is to do.

  In order to achieve such an object, the gist of the first invention is a medical observation support system that supports observation of an organ in a subject, based on three-dimensional image data of the organ obtained in advance. A three-dimensional model of the organ created by the three-dimensional molding apparatus, a position determination unit that determines a position on the model indicated by a predetermined pointing unit in the three-dimensional model of the organ, and three-dimensional image data of the organ And a display unit for generating and displaying the organ image, and an association control for associating the position on the model with a position in the organ image displayed by the display unit or related information related to the position And a section.

  According to the first invention, in the three-dimensional model of the organ created by the three-dimensional molding apparatus based on the three-dimensional image data of the organ obtained in advance, and in the three-dimensional model of the organ, a predetermined instruction A position determination unit that determines a position on the model indicated by the unit, a display unit that generates and displays an image of the organ based on the three-dimensional image data of the organ, and the display unit displays the position on the model An associated control unit that associates the position of the organ in the image of the organ or related information related to the position with the corresponding three-dimensional model of the organ as an input interface. An operation pointing to the three-dimensional model of the organ can be reflected in the image of the organ displayed by the display unit. That is, it is possible to provide a medical observation support system that realizes highly practical observation support by intuitive operation.

  The gist of the second invention subordinate to the first invention includes a distance calculation unit that calculates a distance corresponding to the position on the model in the image of the organ. In this way, the distance corresponding to the operation of the pointing unit can be easily calculated using the three-dimensional model of the organ as an input interface.

  The gist of the third invention subordinate to the first invention or the second invention is that the display unit generates and displays a three-dimensional virtual image of the organ based on the three-dimensional image data of the organ. The incision line corresponding to the transition of the position on the model is synthesized and displayed on the three-dimensional virtual image of the organ. In this way, using the 3D model of the organ as an input interface, an incision line corresponding to the operation of the pointing unit can be simply synthesized and displayed on the 3D virtual image of the organ.

  The gist of the fourth invention according to any one of the first to third inventions is that the display unit generates a three-dimensional virtual image of the organ based on the three-dimensional image data of the organ. The position on the model is synthesized and displayed on the three-dimensional virtual image of the organ. In this way, using the 3D model of the organ as an input interface, the position pointed to by the pointing unit can be easily synthesized and displayed on the 3D virtual image of the organ.

  The gist of the fifth invention according to any one of the first to fourth inventions is that the display unit generates a tomographic image of the organ based on the three-dimensional image data of the organ. The position on the model is combined and displayed on the tomographic image of the organ. In this way, using the three-dimensional model of the organ as an input interface, the position indicated by the pointing unit can be easily synthesized and displayed on the tomographic image of the organ.

  The gist of the sixth invention according to any one of the first to fifth inventions includes an attitude sensor for detecting the attitude of the three-dimensional model of the organ with reference to a predetermined direction, and the display The unit generates and displays a three-dimensional virtual image of the organ based on the three-dimensional image data of the organ, and based on a detection result by the posture sensor, the posture of the three-dimensional virtual image of the organ Is to change. This makes it possible to realize display control in which the posture of the three-dimensional virtual image of the organ follows the posture of the three-dimensional model of the organ, using the three-dimensional model of the organ as an input interface.

  The gist of the seventh invention, which is subordinate to the sixth invention, includes a sterilized bag member, and the three-dimensional model of the organ and the posture sensor are sealed in the bag member. In this way, the three-dimensional model of the organ can be used as an input interface even during surgery.

  The gist of the present invention according to any one of the first invention to the seventh invention is that a plurality of markers attached to the three-dimensional model of the organ, a marker attached to the pointing unit, A marker detection device that detects the position of the marker, and the position determination unit determines the position on the model based on a detection result of the marker detection device. If it does in this way, the position which the said instruction | indication part points in the three-dimensional model of the said organ can be determined with a practical aspect.

  The gist of the ninth invention subordinate to the eighth invention is that the marker is a magnetic marker, and the marker detection device is a device for magnetically detecting the marker. If it does in this way, the position which the said instruction | indication part points in the three-dimensional model of the said organ can be determined with a practical aspect.

  The gist of the tenth aspect of the present invention that depends on the eighth aspect is that the marker is an optical marker, and the marker detection device is a device that optically detects the marker. If it does in this way, the position which the said instruction | indication part points in the three-dimensional model of the said organ can be determined with a practical aspect.

  The gist of the eleventh invention subordinate to the eighth invention is that the marker is a two-dimensional code, and the marker detection device is an imaging device and a two-dimensional code reader. If it does in this way, the position which the said instruction | indication part points in the three-dimensional model of the said organ can be determined with a practical aspect.

  The gist of the twelfth aspect of the present invention is a three-dimensional model of an organ used in the medical observation support system according to any of the eighth to eleventh aspects of the present invention, wherein the correlation reference by the correlation control unit is One or a plurality of reference positions are determined in advance, and the marker is attached to each reference position. In this way, the position on the model can be associated with the position in the image of the organ displayed by the display unit in a practical manner.

It is a figure which illustrates the structure of the medical observation assistance system which is one Example of this invention. It is a functional block diagram explaining the principal part of an example of the control function with which the medical observation assistance system of FIG. 1 was equipped. In the medical observation support system of FIG. 1, it is a figure which shows roughly the aspect which is equipped with a two-dimensional code as a marker and a practitioner's fingertip corresponds to an instruction | indication part. In the medical observation support system of FIG. 1, it is a figure which shows schematically the aspect which equips the magnetic position sensor as a marker and the front-end | tip part of a pointer corresponds to an instruction | indication part. In the medical observation support system of FIG. 1, it is a figure which shows schematically the aspect by which the gyro sensor as an attitude | position sensor was attached to the three-dimensional model. It is a figure which illustrates a mode that the gyro sensor and three-dimensional model shown in FIG. 5 were enclosed in the sterilized bag member. It is a figure which shows typically a mode that the medical observation assistance system of FIG. 1 is actually used in the field of the surgery of an organ. It is a flowchart explaining the principal part of an example of the observation assistance control of a present Example by CPU of the computer with which the medical observation assistance system of FIG. 1 was equipped. FIG. 2 is a diagram illustrating a specific example of a three-dimensional virtual image and a tomographic image displayed on a video display device in the medical observation support system of FIG. 1. It is the photograph which image | photographed the example of 1 structure of the three-dimensional model with which the medical observation assistance system of FIG. 1 was equipped.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating a configuration of a medical observation support system 10 (hereinafter simply referred to as a system 10) that is an embodiment of the present invention. As shown in FIG. 1, a system 10 according to the present embodiment includes a computer 12 that is a main body device (computer), a position information acquisition device 14, and a three-dimensional model 16 of an organ to be observed. . Various forms of the position information acquisition device 14 are conceivable, which will be described later with reference to FIGS. The three-dimensional model 16 will be described later together with the description of the three-dimensional image data of the organ to be observed.

  The computer 12 includes a CPU 20 as a central processing unit, a ROM 22 as a read-only memory, a RAM 24 as a write / read memory as needed, a storage device 26, an input device 28, a video display device 30, and an input interface. 32. The CPU 20 is a so-called microcomputer that processes and controls electronic information based on a predetermined program stored in advance in the ROM 22 while using the temporary storage function of the RAM 24.

  The storage device 26 is a known storage device (storage medium) such as a hard disk drive capable of storing information. It may be a removable storage medium such as a small memory card or an optical disk. The input device 28 is a known input device such as a keyboard or a mouse. The video display device 30 is a known display device (display) such as a TFT (Thin Film Transistor Liquid Crystal). The input interface 32 is a known input interface that supplies a signal input from a position information acquisition device 14 described later to the CPU 20.

  FIG. 2 is a functional block diagram illustrating a main part of an example of the control function provided in the system 10. The image display unit 50, the position determination unit 62, the association control unit 64, and the distance calculation unit 66 shown in FIG. 2 are preferably all functionally provided in the CPU 20 of the computer 12. Each control unit may be individually provided in a plurality of CPUs or computers and configured to be able to communicate with each other.

  As shown in FIG. 2, an image database 68 is provided in the storage device 26 provided in the computer 12. The image database 68 stores three-dimensional image data of a subject obtained in advance by a known X-ray CT (Computed Tomography) apparatus, a magnetic resonance imaging (MRI) apparatus, or the like. The three-dimensional image data includes at least three-dimensional image data of an organ that is the observation target of the system 10. In the case where the three-dimensional image data is obtained by X-ray CT, the subject is irradiated with X-rays from multiple directions, the X-ray transmitted through the subject is detected by a detector, and transmitted X The dose information is processed by a computer and reconstructed as a three-dimensional image. The image database 68 stores, for example, a three-dimensional grayscale image of a subject obtained by X-ray CT, a magnetic resonance apparatus, or the like, in other words, a pixel value (density value) for each voxel that is a unit volume of the subject. Is done.

  The three-dimensional model 16 is created by a known three-dimensional molding apparatus based on the three-dimensional image data of the organ stored in the image database 68. For example, based on the three-dimensional image data of the organ, a three-dimensional organ model created so as to imitate the shape of the organ by a known three-dimensional printer (3D printer) (that is, to have substantially the same shape) (Replica). The three-dimensional model 16 is preferably a three-dimensional model of the liver created based on three-dimensional image data of the liver that is an organ to be observed. Preferably, it is created to be the same size (real size) as the real organ to be observed, but considering the ease of handling, etc., it is enlarged or enlarged relative to the size of the real organ. It may be created by reducing the size. In this case, the enlargement ratio or reduction ratio of the three-dimensional model 16 with respect to the real organ is input to the computer 12. Measurement results such as distance or length when the three-dimensional model 16 is used as an input interface are converted (converted) into the size of the real organ based on the enlargement ratio or reduction ratio, and the video display Displayed on the device 30. Preferably, a lesion site such as a blood vessel or a tumor inside an organ is made of a colored material, and the surrounding tissue is made of a colorless transparent material. In the three-dimensional model 16, a plurality of (at least three) reference positions serving as references for association by the association control unit 64 described later are determined in advance. For example, recesses (holes) having predetermined depths are formed at the plurality of reference positions, respectively. When the three-dimensional model 16 is formed, a marker described later may be integrally formed with the three-dimensional model 16 at each of the plurality of reference positions.

  The system 10 includes an instruction unit used to point to the three-dimensional model 16. As the instruction unit, various modes such as a distal end portion of a forceps, a fingertip of a practitioner (for example, a fingertip of an index finger), a distal end portion of a rod-shaped pointer (pointing device), and the like can be considered. That is, the instruction unit may be a part of a predetermined instrument or a part of a human body. For example, as will be described in detail below, a part to which a marker indicating the instruction unit is attached or a part having a predetermined relationship with the position corresponds to the instruction unit.

  A plurality (at least three) of markers are preferably attached to the three-dimensional model 16. For example, a marker indicating each reference position is attached to each of a plurality of predetermined reference positions. Preferably, at least one marker is attached to the indicating unit. For example, a marker indicating the indication unit is attached to a tip of a forceps, a fingertip of a practitioner, or a tip of a rod-shaped pointer that is set in advance as an indication unit. The system 10 includes a marker detection device that detects the position of the marker. Hereinafter, specific examples of the instruction unit, the marker, and the marker detection device will be described with reference to FIGS.

  FIG. 3 is a diagram schematically showing an aspect in which the system 10 is provided with two-dimensional codes 34 and 36 as the markers, and the operator's fingertip 70 corresponds to the instruction unit. FIG. 10 is a photograph taken of a configuration example of the three-dimensional model 16. In the embodiments shown in these drawings, the three-dimensional model 16 is formed on a portion 16a corresponding to a blood vessel made of a colored material, and a lesion site made of a material of the same color (preferably a color different from the portion 16a). A corresponding portion 16b and a portion 16c corresponding to the surrounding tissue made of a transparent (preferably colorless and transparent) material are provided (the same applies to FIGS. 4 and 5 used in the following description). A plurality (three in FIG. 3) of two-dimensional codes 34 indicating the reference positions are attached to the surface of the three-dimensional model 16 (part 16c). For example, a preprinted two-dimensional code 34 is attached to each of a plurality of predetermined reference positions in the three-dimensional model 16. The plurality of two-dimensional codes 34 are not particularly distinguished in FIG. 3, but represent different information corresponding to the respective reference positions in the three-dimensional model 16 (markers of different codes). Yes. A fingertip (for example, a fingertip of an index finger) 70 of a practitioner who observes an organ using the system 10 is attached with a two-dimensional code 36 indicating the instruction unit. Preferably, a preprinted two-dimensional code 36 is affixed to the practitioner's fingertip 70. The two-dimensional code 36 is different from at least the two-dimensional code 34 attached to the three-dimensional model 16. Here, in a mode in which it is sufficient to determine the one-dimensional position (position as a point) of the instruction unit, as shown in FIG. 3, if one two-dimensional code 36 is attached to the practitioner's fingertip 70, However, in the aspect of determining the direction (that is, the direction in which the pointing unit is pointing), preferably, two or more two-dimensional codes 36 are attached to the pointing unit. For example, the two-dimensional code 36 is attached to the practitioner's fingertip 70 and the base of the finger, respectively. Alternatively, by determining the direction in advance with respect to the instruction unit and attaching the two-dimensional code 36, the direction indicated by the instruction unit is determined from one two-dimensional code 36, and the two-dimensional code in the photographed image. The distance from the imaging device 38 to be described later may be determined based on the size of the code 36 or the like. The two-dimensional code is an information expression code composed of a two-dimensional (planar) figure such as a QR code (registered trademark) defined in Japanese Industrial Standard “JIS X 0510”.

  In the aspect shown in FIG. 3, the system 10 includes an imaging device 38 that captures the two-dimensional codes 34 and 36. As the imaging device 38, a known digital camera that can input electronic data (image data) corresponding to the photographed image to the position information acquisition device 14 is used. Alternatively, an analog camera or the like provided with an A / D converter may be used. The imaging device 38 is connected to the position information acquisition device 14, and an image (video) taken by the imaging device 38 is input to the position information acquisition device 14. In FIG. 3, the imaging device 38 and the position information acquisition device 14 are illustrated as separate devices, but the imaging device 38 is built in the position information acquisition device 14. It may be. The imaging device 38 may be built in the computer 12. The position information acquisition device 14 preferably includes a known two-dimensional code reader for identifying the two-dimensional codes 34 and 36. The position determination unit 62 described in detail below may have a function as a two-dimensional code reader. In the aspect shown in FIG. 3, the imaging device 38 and the position information acquisition device 14 correspond to the marker detection device.

  The position determination unit 62 determines a position on the model indicated by the instruction unit in the three-dimensional model 16 based on the position information acquired by the position information acquisition device 14. The position on the model corresponds to the position on the surface of the three-dimensional model 16 (the surface of the portion 16c), and is preferably the coordinates of one point on the surface. Alternatively, not only the position on the three-dimensional model 16 but also an arbitrary position on the image space where the three-dimensional model 16 is placed. In addition to the position on the surface of the three-dimensional model 16, the direction of the pointing unit relative to the three-dimensional model 16 (relative direction in which the pointing unit points to the three-dimensional model 16) may also be determined. The position determination unit 62 preferably determines the position on the model based on the detection result by the marker detection device. In this embodiment, the plurality of markers are arranged so that three or more markers can always be read by the marker detection device. In the aspect shown in FIG. 3, when the plurality of two-dimensional codes 34 are photographed by the imaging device 38 and read by a two-dimensional code reader provided in the position information acquisition device 14, the plurality of the two-dimensional codes 34 with respect to the imaging device 38 are captured. The position and direction of the two-dimensional code 34 are specified. Accordingly, the positions and directions of the plurality of reference positions in the three-dimensional model 16 with respect to the imaging device 38 are determined. When the two-dimensional code 36 is photographed by the imaging device 38 and read by a two-dimensional code reader provided in the position information acquisition device 14, the position and direction of the two-dimensional code 36 relative to the imaging device 38 are specified. The Thereby, the position and direction of the practitioner's fingertip 70 with respect to the imaging device 38 are determined. From the position and direction with respect to the imaging device 38, the relative position of the practitioner's fingertip 70 with respect to the three-dimensional model 16 is determined. That is, in the three-dimensional model 16, the position on the model pointed by the practitioner's fingertip 70 is determined.

  In the mode shown in FIG. 3, when the plurality of two-dimensional codes 34 are photographed by the imaging device 38 and read by a two-dimensional code reader provided in the position information acquisition device 14, a predetermined direction (for example, a vertical direction) The orientation of the three-dimensional model 16 with reference to the above is specified. For example, the posture of the three-dimensional model 16 with respect to the vertical direction is specified by the inclination of the plurality of reference positions determined in advance in the three-dimensional model 16 with respect to the vertical direction. That is, in the embodiment shown in FIG. 3, the two-dimensional code 34, the imaging device 38, and the position information acquisition device 14 are posture sensors that detect the posture of the three-dimensional model 16 with reference to a predetermined direction. Function.

  FIG. 4 is a diagram schematically showing an aspect in which the magnetic position sensors 40 and 42 are provided as the markers in the system 10 and the tip 72 of the pointer corresponds to the pointing unit. In the aspect shown in FIG. 4, a plurality of magnetic position sensors 40 (only one is illustrated in FIG. 4) indicating the reference position are attached to the surface of the three-dimensional model 16 (part 16c). . For example, the magnetic position sensor 40 is attached to each of the plurality of reference positions determined in advance in the three-dimensional model 16. A magnetic position sensor 42 is attached to the tip 72 of the pointer, which is a rod-shaped member. The magnetic position sensors 40 and 42 are wired to the position information acquisition device 14. Alternatively, it may be connected wirelessly. The magnetic position sensors 40 and 42 are known magnetic position sensors. For example, a transmitter (not shown) is provided separately from the magnetic position sensors 40 and 42, and a magnetic signal transmitted from the transmitter is transmitted. By receiving the magnetic position sensors 40 and 42, five-dimensional information, that is, three-dimensional position information and two-dimensional direction information, is acquired from the magnetic position sensors 40 and 42 and supplied to the position information acquisition device 14. It has come to be. In the aspect shown in FIG. 4, the position information acquisition device 14 corresponds to the marker detection device.

  In the mode shown in FIG. 4, when position information and direction information corresponding to each of the plurality of magnetic position sensors 40 are supplied to the position information acquisition device 14, the plurality of magnetic position sensors 40 with respect to the transmitter are arranged. The position and direction are specified. Thereby, the position and direction of the plurality of reference positions in the three-dimensional model 16 with respect to the transmitter are determined. When position information and direction information corresponding to the magnetic position sensor 42 are supplied to the position information acquisition device 14, the position and direction of the magnetic position sensor 42 with respect to the transmitter are specified. Thereby, the position and direction of the tip 72 of the pointer with respect to the transmitter are determined. From the position and direction with respect to the transmitter, the relative position of the tip 72 of the pointer with respect to the three-dimensional model 16 is determined. That is, in the three-dimensional model 16, the position on the model indicated by the tip 72 of the pointer is determined.

  In the mode shown in FIG. 4, when position information and direction information corresponding to each of the plurality of magnetic position sensors 40 are supplied to the position information acquisition device 14, a predetermined direction (for example, a vertical direction) is used as a reference. The posture of the three-dimensional model 16 is specified. That is, in the embodiment shown in FIG. 4, the magnetic position sensor 40 and the position information acquisition device 14 function as a posture sensor that detects the posture of the three-dimensional model 16 with a predetermined direction as a reference.

  The system 10 may include an optical marker as the marker. For example, in the embodiment described above with reference to FIG. 3 or FIG. 4, the two-dimensional codes 34 and 36 or the magnetic position sensors 40 and 42 may be replaced with optical markers. As this optical marker, a light emitting element such as an LED that emits near infrared rays, a reflective marker that reflects light, or the like is preferably used. In such an aspect, a marker detection device that detects the position and direction of the optical marker, such as a near-infrared camera that detects near-infrared rays, is provided. The position and direction of each optical marker detected by the marker detection device is supplied to the position information acquisition device 14, and the relative position of the pointing unit with respect to the three-dimensional model 16 is determined by the same method as described above. Is done. That is, in the three-dimensional model 16, the position on the model indicated by the instruction unit is determined.

  FIG. 5 is a diagram schematically showing an aspect in which a gyro sensor 44 as an attitude sensor is attached to the three-dimensional model 16 in the system 10. In the embodiment shown in FIG. 5, a gyro sensor 44 as an attitude sensor is attached to the three-dimensional model 16. The gyro sensor 44 is a known gyro sensor that detects an angle and a direction with respect to a predetermined direction (for example, a vertical direction). In the aspect illustrated in FIG. 5, the position information acquisition device 14 functions as a reception unit that wirelessly receives the detection result of the gyro sensor 44. When the detection result by the gyro sensor 44 is supplied to the position information acquisition device 14, the posture of the three-dimensional model 16 with respect to the predetermined direction is specified. In the embodiment shown in FIG. 5, another marker is attached to the three-dimensional model 16 in order to determine the relative position of the pointing unit with respect to the three-dimensional model 16. For example, the two-dimensional code 34 described above with reference to FIG. 3 or the magnetic position sensor 40 described above with reference to FIG. 4 is attached to the three-dimensional model 16. Here, in the aspect in which the three-dimensional model 16 includes the gyro sensor 44, the number of markers is smaller than that in the aspect described above with reference to FIG. 3 or FIG. The relative position can be determined.

  As mentioned above, although the specific example of the said instruction | indication part in the said system 10, the said marker, and the said marker detection apparatus was demonstrated using FIGS. 3-5, another aspect is also considered. For example, a contact-type position sensor is provided on the surface of the three-dimensional model 16 (that is, the surface of the portion 16c), and the contact position of the pointing unit with respect to the three-dimensional model 16 is detected by the contact-type position sensor. May be. In such an aspect, the relative position of the pointing unit with respect to the three-dimensional model 16 is determined based on the detection result by the contact-type position sensor. That is, in the three-dimensional model 16, the position on the model indicated by the instruction unit is determined. Alternatively, the pointing unit for the three-dimensional model 16 is provided by an ultrasonic detector that includes an ultrasonic transmitter, receives ultrasonic waves generated by the ultrasonic transmitter with a plurality of sensors, and detects a position according to the time difference. The relative position may be determined. Alternatively, an RFID (Radio Frequency Identification) system including an active type radio tag that emits radio waves may be used as the marker and the marker detection device.

  FIG. 6 is a diagram illustrating a state in which the three-dimensional model 16 to which the gyro sensor 44 is attached is enclosed in a bag member 74. The system 10 preferably includes a sterilized bag member 74. The bag member 74 is a colorless and transparent bag made of vinyl resin or the like, and is sterilized to such an extent that it can be brought into the operating room. The three-dimensional model 16 and the gyro sensor 44 are brought into the operating room in a state of being enclosed in the bag member 74 and used for observation support in the system 10. After the three-dimensional model 16 and the gyro sensor 44 are sealed in the bag member 74, the bag member 74 may be used in a so-called vacuum pack-like state in which the air inside the bag member 74 is removed. The three-dimensional model 16 to which the two-dimensional code 34 described above using FIG. 3 or the magnetic position sensor 40 described above using FIG. 4 is attached is the two-dimensional code 34 or the magnetic position sensor 40. In addition, it may be used in a state of being enclosed in the bag member 74.

  The image display unit 50 displays various images (videos) related to control by the system 10 on the video display device 30 via a graphics processor or the like (not shown). The image display unit 50 generates an image of the organ based on the three-dimensional image data of the organ stored in the image database 68 and displays the image on the video display device 30. In order to perform such processing, the image display unit 50 includes a three-dimensional virtual image display unit 52, a tomographic image display unit 54, an indicated position display unit 56, an incision line display unit 58, and a distance display as shown in FIG. Part 60 is provided.

  The three-dimensional virtual image display unit 52 generates a three-dimensional virtual image of the organ based on the three-dimensional image data of the organ stored in the image database 68 and displays it on the video display device 30. The three-dimensional virtual image of the organ is an image showing the shape of the whole organ, and is preferably an outline image of the organ. That is, the three-dimensional virtual image display unit 52 preferably displays a graphic image showing the external shape of the organ (for example, an image representing an external appearance viewed from an arbitrary direction) based on the three-dimensional image data. That is, the three-dimensional image data obtained in advance corresponding to the organ to be observed by the system 10 and stored in the image database 68 is read, and the 3D image data is read by a known technique such as volume rendering or surface rendering. An outline image of the organ is generated based on the dimensional image data and displayed on the video display device 30. 3 to 5 and 7 exemplify a state in which the three-dimensional virtual image 80 of the organ generated based on the three-dimensional image data of the organ is displayed on the video display device 30.

  The tomographic image display unit 54 generates a tomographic image of the organ at a specified position or cutting line based on the three-dimensional image data of the organ stored in the image database 68, and displays it on the video display device 30. Display. This tomographic image of the organ is an image showing a cross section of the organ. A tomographic image obtained in advance by an X-ray CT apparatus or the like may be displayed as it is, or an image corresponding to a virtual cross section generated based on three-dimensional image data. That is, the tomographic image display unit 54 reads out the three-dimensional image data obtained in advance corresponding to the organ to be observed by the system 10 and stored in the image database 68, and the three-dimensional image is read by a known technique. Based on the data, an image showing a cross section of the organ is generated and displayed on the video display device 30. FIG. 7 illustrates a state where a tomographic image 86 of the organ generated based on the three-dimensional image data of the organ is displayed on the video display device 30.

  The association control unit 64 associates the position on the model indicated by the instruction unit in the three-dimensional model 16 with the position in the image of the organ displayed on the video display device 30 or related information related to the position. That is, a coordinate system registration process for associating the coordinate system corresponding to the three-dimensional model 16 and the pointing unit with the coordinate system corresponding to the organ image displayed on the video display device 30 is performed. Based on a plurality of feature points in the organ, not in the coordinate system, so that the positional relationship between the feature points matches between the three-dimensional model 16 and the image of the organ displayed on the video display device 30. Correlation may be performed. The association control unit 64 preferably associates the position on the model with the position in the three-dimensional virtual image 80 of the organ displayed by the three-dimensional virtual image display unit 52. Preferably, the position on the model is associated with a position in the tomographic image 86 of the organ displayed by the tomographic image display unit 54. Hereinafter, an example of a specific algorithm related to the association by the association control unit 64 will be described.

The association (coordinate system registration) between the coordinate system corresponding to the pointing unit and the coordinate system corresponding to the organ image in the system 10 will be described. The coordinate system corresponding to the instruction unit is L, the coordinate system corresponding to the marker attached to the instruction unit is M, and the marker detection device that detects the marker (for example, the position information acquisition device 14 or the imaging device 38). Let P be the coordinate system corresponding to, and I be the coordinate system corresponding to the organ image. For example, as shown in FIG. 3, when the marker of the pointing unit and the marker of the three-dimensional model 16 are photographed by a common imaging device, the coordinate systems L and M of both are common. When the coordinate system L and the coordinate system M are substantially the same, the conversion from the coordinate system L to the coordinate system M described below may not be performed. T _ML is a transformation matrix from the coordinate system L to the coordinate system M, T _PM is a transformation matrix from the coordinate system M to the coordinate system P, and T _IP is a transformation matrix from the coordinate system P to the coordinate system I. When the coordinates p _L in the coordinate system L corresponding to the instruction unit, as shown in the following equation (1), corresponding to an image of the organ using the transformation matrix T _ML, T _PM, T _IP It is converted into the coordinate p _I in the coordinate system I. That is, the coordinates p _L in the coordinate system L are associated by the following equation (1) in the coordinate p _I in the coordinate system I. Rigid body coordinate transformation uses rotation and translation as elements. Therefore, the transformation of the coordinate p by the transformation matrix T is expressed as the following equation (2) using the rotation matrix R and the transformation vector t.
p _I = T _IP T _PM T _ML p _L (1)
Tp = Rp + t (2)

In the system 10, the position and direction corresponding to the pointing unit (for example, the position and direction of the tip of the pointer), the position and direction of the marker attached to the pointing unit (for example, the marker mounting position and (Direction) is determined in advance. This relationship corresponds to the transformation matrix _TML from the coordinate system L to the coordinate system M. Information corresponding to the position and direction of the marker attached to the instruction unit is acquired by the marker detection device. With this information, a transformation matrix T _PM from the coordinate system M to the coordinate system P is determined. That is, by detecting information corresponding to the position and direction of the marker attached to the instruction unit by the marker detection device, the coordinate p _L in the coordinate system L corresponding to the instruction unit is associated with the marker detection device. It can be converted into the coordinate p _P in the coordinate system P of. Coordinate p _P in the coordinate system P corresponding to the marker detecting device, as shown in the following equation (3), the coordinates p _I in a coordinate system I corresponding to the image of the organ using the transformation matrix T _IP Converted.
p _I = T _IP p _P (3)

The conversion from the coordinate system P corresponding to the marker detection device to the coordinate system I corresponding to the image of the organ is realized, for example, by matching reference positions in the respective coordinate systems. Preferably, in the coordinate system I corresponding to the organ image, a reference position relating to the association with the coordinate system L corresponding to the pointing unit is predetermined. This reference position is preferably one of the reference positions defined for the three-dimensional model 16. In the three-dimensional model 16, when the pointing unit points to the reference position, a reference position in the coordinate system P corresponding to the marker detection device, and a reference position in the coordinate system I corresponding to the organ image by performing mapping to match the transformation matrix T _IP to the coordinate system I is determined from the coordinate system P. For example, assuming that the coordinate of the reference position in the coordinate system P corresponding to the marker detection device is pi, the coordinate of the reference position in the coordinate system I corresponding to the organ image is qi, and the number of reference positions is N, the following (4 ) is the transformation matrix T ^* which satisfies the equation, corresponding to the transformation matrix T _IP.

  The association between the coordinate system L corresponding to the instruction unit and the coordinate system I corresponding to the organ image has been described above, but the coordinate system corresponding to the three-dimensional model 16 and the coordinates corresponding to the organ image are described. The association with the system I is also realized by the same algorithm as described above. That is, by the same algorithm as described above, a plurality of reference positions predetermined in the three-dimensional model 16 and a plurality of reference positions predetermined in the coordinate system I corresponding to the image of the organ are converted into the coordinate system. This is realized by matching between L and I. A coordinate system L corresponding to the instruction unit is associated with a coordinate system I corresponding to the organ image, a coordinate system corresponding to the three-dimensional model 16, and a coordinate system I corresponding to the organ image. Are associated with each other, the position on the model indicated by the instruction unit in the three-dimensional model 16 can be associated with the image of the organ displayed by the image display unit 50.

  The distance calculation unit 66 is based on the result of association by the association control unit 64, and on the model indicated by the instruction unit in the three-dimensional model 16 in the organ image displayed by the image display unit 50. The distance corresponding to the position is calculated. Specifically, the distance is calculated based on the three-dimensional image data of the organ stored in the image database 68. The distance calculation unit 66 preferably calculates the distance (blood vessel length) of the main blood vessel indicated by the instruction unit in the three-dimensional model 16 in the organ image. The main blood vessels are arteries or veins existing in the direction (inside the organ) indicated by the pointing unit, and are blood vessels generally given anatomical names, for example. The distance calculation unit 66 preferably calculates a distance between a position on the model indicated by the instruction unit in the three-dimensional model 16 and a lesion site in the organ in the image of the organ. Preferably, the distance calculation unit 66 calculates a distance between a plurality of model positions (for example, two points indicated in succession) indicated by the instruction unit in the three-dimensional model 16 in the organ image. calculate.

  Based on the result of association by the association control unit 64, the image display unit 50 reflects the position on the model indicated by the instruction unit in the three-dimensional model 16 in the image of the organ. That is, the image of the organ displayed on the video display device 30 is changed according to the change in the position on the model indicated by the instruction unit in the three-dimensional model 16. Alternatively, an additional image (including text such as numerical values) is synthesized and displayed on the organ image displayed on the video display device 30. In this case, the additional image corresponds to related information related to the position.

  The distance display unit 60 displays the distance calculated by the distance calculation unit 66 on the video display device 30. Preferably, a numerical value corresponding to the distance calculated by the distance calculation unit 66 is synthesized and displayed on the organ image displayed on the video display device 30. In the aspect in which the distance calculation unit 66 calculates the length of the blood vessel pointed to by the instruction unit in the three-dimensional model 16, the numerical value indicating the blood vessel length as well as the target blood vessel in the organ image. Is displayed. In the aspect in which the distance calculation unit 66 calculates the distance between the position on the model indicated by the instruction unit in the three-dimensional model 16 and the lesion site in the organ, the target site in the organ image is indicated. The numerical value representing the distance is synthesized and displayed. In the aspect in which the distance calculation unit 66 calculates the distance between the plurality of model positions indicated by the instruction unit in the three-dimensional model 16, the distance calculation unit 66 indicates the plurality of positions indicated by the instruction unit in the organ image. The numerical value representing the distance is synthesized and displayed. In this case, the numerical value corresponds to related information related to the position.

  The three-dimensional virtual image display unit 52 is configured to display the three-dimensional image displayed on the video display device 30 based on the posture information of the three-dimensional model 16 acquired by the position information acquisition device 14 or the gyro sensor 44 or the like. The posture of the virtual image 80 is changed. That is, the posture of the three-dimensional model 16 based on a predetermined direction (for example, the vertical direction) determined by the position determination unit 62 based on the detection result by the position information acquisition device 14 or the gyro sensor 44 or the like. Based on the change, the posture (direction and angle) of the three-dimensional virtual image 80 displayed on the video display device 30 is changed.

  The incision line display unit 58 synthesizes and displays the incision line 82 corresponding to the transition of the position on the model indicated by the instruction unit in the three-dimensional model 16 on the three-dimensional virtual image 80 displayed on the video display device 30. Let For example, in the embodiment shown in FIG. 3, when the practitioner's fingertip 70 serving as the pointing unit changes from the position indicated by the broken line to the position indicated by the solid line (tracing the surface of the three-dimensional model 16), the trajectory is 3 This is indicated by a thick broken line on the dimensional model 16. In this case, as shown in FIG. 3, an incision line 82 corresponding to the trajectory is generated and displayed on the three-dimensional virtual image 80. In this case, the incision line 82 corresponds to related information related to the position. Alternatively, a virtual tomographic image showing a cross section (incision plane) of the organ obtained by cutting the three-dimensional virtual image 80 along a plane corresponding to the incision line 82 is generated and displayed on the video display device 30. May be.

  FIG. 7 is a diagram schematically showing how the system 10 is actually used in the field of surgery for the organ. In FIG. 7, a surgeon 92 involved in the operation of the liver 90a in a subject (subject) 90 holds the three-dimensional model 16 created based on the three-dimensional image data of the liver 90a in his hand, and the tip of a forceps A mode in which the three-dimensional model 16 is indicated by the part 76 is illustrated. That is, in the embodiment shown in FIG. 7, the distal end portion 76 of the forceps corresponds to the instruction portion. The assistant 94 and the nurse 96 related to the operation watch the display of the video display device 30 while listening to the explanation of the surgeon 92. The video display device 30 displays the three-dimensional virtual image 80 and the tomographic image 86 generated based on the three-dimensional image data of the liver 90a.

  The indicated position display unit 56 reflects the position on the model indicated by the indicating unit in the three-dimensional model 16 in the three-dimensional virtual image 80. For example, in the aspect shown in FIG. 4, a mark 84 representing the position in the three-dimensional virtual image 80 corresponding to the position on the model indicated by the tip 72 of the pointer as the pointing unit is displayed on the three-dimensional virtual image 80. Composite display on the front side. In the embodiment shown in FIG. 7, the mark 84 representing the position in the three-dimensional virtual image 80 corresponding to the position on the model indicated by the distal end portion 76 of the forceps as the pointing portion is provided on the front side of the three-dimensional virtual image 80. Is displayed as a composite. In this case, the mark 84 corresponds to related information related to the position. 4 and 7, the mark 84 is indicated by an asterisk. The indicated position display unit 56 changes the relative position of the mark 84 with respect to the three-dimensional virtual image 80 based on the change in the position on the model determined by the position determination unit 62.

  The tomographic image display unit 54 displays a tomographic image 86 corresponding to a cross section including a position on the model indicated by the instruction unit in the three-dimensional model 16. That is, the tomographic image 86 corresponding to the cross section including the position on the model is selected and displayed from the tomographic images 86 that can be displayed based on the three-dimensional image data of the organ. The indicated position display unit 56 reflects the position on the model indicated by the indicating unit in the three-dimensional model 16 in the tomographic image 86. For example, in the embodiment shown in FIG. 7, a mark 88 representing the position in the tomographic image 86 corresponding to the position on the model indicated by the distal end portion 76 of the forceps as the indicating unit is synthesized on the front side of the tomographic image 86. It is displayed. In FIG. 7, the mark 88 is indicated by an asterisk. The indicated position display unit 56 changes the relative position of the mark 88 with respect to the tomographic image 86 based on the change in the position on the model determined by the position determination unit 62.

  FIG. 9 is a diagram illustrating a specific example of the three-dimensional virtual image 80 and the tomographic image 86 displayed on the video display device 30 by the image display unit 50. In the example shown in FIG. 9, a lesion site (tumor or the like) of the target organ is indicated by a site 80 b in the three-dimensional virtual image 80 and a site 86 b in the tomographic image 86. An incision line 82 corresponding to the transition of the position on the model indicated by the instruction unit in the three-dimensional model 16 is synthesized and displayed on the front side of the three-dimensional virtual image 80. Further, a pointer image 98 indicating the instruction unit is displayed in a composite manner on the front side of the three-dimensional virtual image 80. In the three-dimensional model 16, the position of the pointer image 98 with respect to the three-dimensional virtual image 80 is changed according to the transition of the position on the model indicated by the instruction unit. In the three-dimensional model 16, an x coordinate line 100 x and a y coordinate line 100 y representing the position on the model pointed to by the instruction unit are combined and displayed on the front side of the tomographic image 86. The x-coordinate line 100x and the y-coordinate line 100y correspond to the x-coordinate and y-coordinate of the position on the model, respectively, in the plane corresponding to the tomographic image 86, and the intersections correspond to the position on the model. Further, in the example shown in FIG. 9, the text 102 indicating the xy coordinates (x1, y1) of the position on the model in the plane corresponding to the tomographic image 86 is synthesized and displayed on the front side of the tomographic image 86. Yes.

  As described above, the system 10 of the present embodiment uses the three-dimensional model 16 as an input interface, and interacts with the three-dimensional virtual image 80 and the tomographic image 86 displayed on the video display device 30. Is possible. The three-dimensional model 16, the three-dimensional virtual image 80, and the tomographic image 86 are organ models in which the three-dimensional image data is embodied in different modes based on common three-dimensional image data. In other words, the three-dimensional model 16 is a realistic organ model having the same shape (substance) as the organ, which is embodied so that the three-dimensional image data can be tactilely confirmed. The three-dimensional virtual image 80 and the tomographic image 86 are virtual organ models having the same shape information and detailed structure information as the organ, which are embodied so that the three-dimensional image data can be visually confirmed. By synchronizing the display of the three-dimensional virtual image 80 and the tomographic image 86 as virtualized organ models with the operation using the three-dimensional model 16 as a realistic organ model, the organs in real space are synchronized. While confirming the shape of the object by touch, the distance of each part in the organ, the virtual cross section, and the like can be visually recognized in the virtual space. That is, the advantages of the realized organ model and the virtual organ model can be achieved. At the same time, by utilizing the 3D model 16 itself as an actualized organ model as an operation (input) device, it is possible to construct a navigation system that realizes appropriate surgical support by intuitive operation.

  FIG. 8 is a flowchart for explaining a main part of an example of the observation support control of the present embodiment by the CPU 20 of the computer 12, which is repeatedly executed.

  First, in step (hereinafter, step is omitted) S1, three-dimensional image data of an organ to be observed is read from the image database 68. Next, in S <b> 2, a three-dimensional virtual image 80 and a tomographic image 86 of the organ are generated based on the three-dimensional image data read in S <b> 1 and displayed on the video display device 30. Next, in S3, information indicating the positions and directions of the three-dimensional model 16 and the instruction unit is acquired by the position information acquisition device 14, and the position on the model indicated by the instruction unit in the three-dimensional model 16 is determined. The Next, in S4, an associating process that associates the position on the model determined in S3 with the position of the organ displayed in S2 in the three-dimensional virtual image 80 and the tomographic image 86, that is, coordinate system registration is performed. .

  Next, in S5, it is determined whether or not the three-dimensional model 16 has been moved. Specifically, it is determined whether or not the posture of the three-dimensional model 16 with respect to a predetermined direction has been changed. If the determination in S5 is negative, the processing from S10 is executed. If the determination in S5 is positive, the three-dimensional model acquired by the position information acquisition device 14 in S6. Based on the 16 posture information, the displacement of the three-dimensional model 16 with respect to the predetermined direction is calculated. Next, in S7, the three-dimensional virtual image 80 is rotated based on the displacement calculated in S6. Next, in S8, 3D rendering is performed based on the rotation result in S7. That is, the three-dimensional virtual image 80 (the three-dimensional virtual image 80 corresponding to the posture rotated based on the displacement calculated in S6) as a result of the rotation in S7 is generated. Next, in S9, after the three-dimensional virtual image 80 generated in S8 is displayed on the video display device 30, this routine is terminated.

  In S10, it is determined whether or not the input of the indicated position, that is, the input of the position on the model indicated by the indicating unit in the three-dimensional model 16 is performed. If the determination in S10 is negative, the processing of S14 and subsequent steps is executed. If the determination in S10 is positive, in S11, the distance corresponding to the input position on the model is It is calculated in the three-dimensional virtual image 80 (tomographic image 86) of the organ. Next, in S12, the distance calculated in S11 is displayed on the video display device 30. Next, in S13, the inputted position on the model is combined and displayed on the three-dimensional virtual image 80 and the tomographic image 86 displayed on the video display device 30, and then this routine is terminated. . In S <b> 14, it is determined whether or not an incision line has been input by the transition of the position on the model indicated by the instruction unit in the three-dimensional model 16. If the determination in S14 is negative, the routine is terminated accordingly. If the determination in S14 is affirmative, an incision simulation based on the incision line based on the transition of the position on the model is performed in S15. Executed. For example, a virtual tomographic image showing a cross section (incision plane) of the organ, which is obtained by cutting the three-dimensional virtual image 80 along a plane corresponding to the incision line, is generated. Next, in S16, the incision line 82 resulting from the transition of the position on the model is combined with the three-dimensional virtual image 80, and the result of the incision simulation in S15 is displayed on the video display device 30, and then the main line is used. The routine is terminated.

  In the above control, S2, S8, S9, S12, S13, and S16 are the operations of the image display unit 50, S2, S7 to S9 are the operations of the three-dimensional virtual image display unit 52, and S2 is the tomographic image display. S13 is the operation of the indicated position display unit 56, S15 and S16 are the operation of the incision line display unit 58, S12 is the operation of the distance display unit 60, and S3 and S6 are the position determination. S4 corresponds to the operation of the unit 62, S4 corresponds to the operation of the association control unit 64, and S11 corresponds to the operation of the distance calculation unit 66.

  According to the present embodiment, based on the three-dimensional image data of the organ obtained in advance, the three-dimensional model 16 of the organ created by the three-dimensional molding apparatus and the fingertip 70 of the operator in the three-dimensional model 16 Based on the position determination unit 62 (S3 and S6) for determining the position on the model indicated by the pointing unit such as the distal end portion 72 of the pointer and the distal end portion 76 of the forceps, and the three-dimensional image data of the organ, the image of the organ The image display unit 50 (S2, S8, S9, S12, S13, S16) for generating and displaying the image and the position on the model at the position or the position in the organ image displayed by the image display unit 50 Since the correspondence control unit 64 (S4) that associates with related information is provided, the instruction unit uses the three-dimensional model 16 of the organ as an input interface. An operation of pointing to 3D model 16 of the vessel, can be reflected in the image of the organ to be displayed by the image display unit 50. That is, it is possible to provide the system 10 that realizes highly practical observation support through intuitive operation.

  Since the distance calculation unit 66 (S11) for calculating the distance corresponding to the position on the model in the image of the organ is provided, the three-dimensional model 16 of the organ is used as an input interface, and the indication unit The distance corresponding to the operation can be calculated easily.

  The image display unit 50 generates and displays a three-dimensional virtual image 80 of the organ based on the three-dimensional image data of the organ, and an incision line 82 corresponding to the transition of the position on the model is displayed. Since the three-dimensional virtual image 80 of the organ is compositely displayed, the three-dimensional model 16 of the organ is used as an input interface, and the three-dimensional virtual image 80 of the organ is operated according to the operation of the instruction unit. The incision line 82 can be synthesized and displayed easily.

  The image display unit 50 generates and displays a three-dimensional virtual image 80 of the organ based on the three-dimensional image data of the organ, and displays the position on the model as a three-dimensional virtual image 80 of the organ. Since the organ 3D model 16 is used as an input interface, the position indicated by the pointing unit can be simply synthesized and displayed on the 3D virtual image 80 of the organ.

  The image display unit 50 generates and displays a tomographic image 86 of the organ based on the three-dimensional image data of the organ, and displays the model position on the tomographic image 86 of the organ. Therefore, the position pointed to by the pointing unit can be easily synthesized and displayed on the tomographic image 86 of the organ using the three-dimensional model 16 of the organ as an input interface.

  A gyro sensor 44 as a posture sensor for detecting the posture of the three-dimensional model 16 of the organ with a predetermined direction as a reference, and the image display unit 50 based on the three-dimensional image data of the organ The three-dimensional virtual image 80 is generated and displayed, and the posture of the three-dimensional virtual image 80 of the organ is changed based on the detection result by the posture sensor. 16 as an input interface, it is possible to realize display control in which the posture of the three-dimensional virtual image 80 of the organ follows the posture of the three-dimensional model 16 of the organ.

  Since the sterilized bag member 74 is provided and the three-dimensional model 16 of the organ and the posture sensor are sealed in the bag member 74, the three-dimensional model 16 of the organ is input to the interface even during the operation. Can be used as

  A plurality of markers attached to the three-dimensional model 16 of the organ; a marker attached to the pointing unit; and a marker detection device that detects a position of the marker; and the position determination unit 62 includes the marker Since the position on the model is determined based on the detection result by the detection device, the position indicated by the instruction unit in the three-dimensional model 16 of the organ can be determined in a practical manner.

  Since the marker detection devices are magnetic position sensors 40 and 42 that magnetically detect the marker, the position indicated by the pointing unit in the three-dimensional model 16 of the organ can be determined in a practical manner.

  Since the marker is an optical marker, and the marker detection device is a device that optically detects the marker, the position indicated by the indication unit in the three-dimensional model 16 of the organ is practically used. Can be judged.

  The marker is a two-dimensional code 34, 36, and the marker detection device is the imaging device 38 and the position information acquisition device 14 as a two-dimensional code reader. The position pointed to by the instruction unit in the model 16 can be determined.

  In the three-dimensional model 16 of the organ used in the system 10, one or a plurality of reference positions serving as a reference for association by the association control unit 64 are determined in advance, and the marker is attached to each reference position Therefore, in a practical aspect, the position on the model indicated by the pointing unit in the three-dimensional model 16 of the organ can be associated with the position in the image of the organ displayed by the image display unit 50.

  The preferred embodiments of the present invention have been described in detail with reference to the drawings. However, the present invention is not limited to these embodiments, and various modifications can be made without departing from the spirit of the present invention. To be implemented. For example, the related information related to the position is not limited to the related information in the embodiment, and various related information other than the above can be displayed in various modes.

  10: medical observation support system, 14: position information acquisition device (marker detection device), 16: three-dimensional model, 34, 36: two-dimensional code (marker), 38: imaging device, 40, 42: magnetic position sensor ( Marker), 44: Gyro sensor (posture sensor), 50: Image display unit, 62: Position determination unit, 64: Control unit with correspondence, 66: Distance calculation unit, 70: Fingertip (instruction unit) of practitioner, 72: Pointer of pointer (instruction part), 74: bag member, 76: tip of forceps (instruction part), 80: three-dimensional virtual image, 82: incision line, 86: tomographic image, 90: subject (subject), 90a: Liver (organ)

Claims (12)

A medical observation support system for supporting observation of an organ in a subject,
A three-dimensional model of the organ created by a three-dimensional molding apparatus based on the three-dimensional image data of the organ obtained in advance;
In the three-dimensional model of the organ, a position determination unit that determines a position on the model indicated by a predetermined instruction unit;
A display unit configured to generate and display an image of the organ based on the three-dimensional image data of the organ;
A medical observation support system comprising: an association control unit that associates the position on the model with a position in the image of the organ displayed by the display unit or related information related to the position. The medical observation support system according to claim 1, further comprising a distance calculation unit that calculates a distance corresponding to the position on the model in the image of the organ. The display unit
Generating and displaying a three-dimensional virtual image of the organ based on the three-dimensional image data of the organ;
The medical observation support system according to claim 1, wherein an incision line corresponding to the transition of the position on the model is displayed in a synthesized manner on a three-dimensional virtual image of the organ. The display unit
Generating and displaying a three-dimensional virtual image of the organ based on the three-dimensional image data of the organ;
The medical observation support system according to any one of claims 1 to 3, wherein the position on the model is synthesized and displayed on a three-dimensional virtual image of the organ. The display unit
Generating and displaying a tomographic image of the organ based on the three-dimensional image data of the organ;
The medical observation support system according to any one of claims 1 to 4, wherein the position on the model is combined and displayed on a tomographic image of the organ. A posture sensor for detecting the posture of the three-dimensional model of the organ with reference to a predetermined direction;
The display unit
Generating and displaying a three-dimensional virtual image of the organ based on the three-dimensional image data of the organ;
The medical observation support system according to any one of claims 1 to 5, wherein the posture of the three-dimensional virtual image of the organ is changed based on a detection result by the posture sensor. It has a sterilized bag member,
The medical observation support system according to claim 6, wherein the three-dimensional model of the organ and the posture sensor are sealed in the bag member. A plurality of markers attached to the three-dimensional model of the organ;
A marker attached to the indicator;
A marker detection device for detecting the position of the marker,
The medical observation support system according to any one of claims 1 to 7, wherein the position determination unit determines the position on the model based on a detection result by the marker detection device. The marker is a magnetic marker,
The medical observation support system according to claim 8, wherein the marker detection device is a device that magnetically detects the marker. The marker is an optical marker,
The medical observation support system according to claim 8, wherein the marker detection device is a device that optically detects the marker. The marker is a two-dimensional code;
The medical observation support system according to claim 8, wherein the marker detection device is an imaging device and a two-dimensional code reader. A three-dimensional model of an organ used in the medical observation support system according to any one of claims 8 to 11,
A three-dimensional model of an organ in which one or a plurality of reference positions serving as references for association by the association control unit are determined in advance, and the marker is attached to each reference position.