JP2008036447A - Probe apparatus within body cavity - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a probe apparatus within a body cavity which can more correctly guide a real-time image as a guide image. <P>SOLUTION: An ultrasonic vibrator array 29 which acquires the echo signal of an ultrasound is formed on the leading edge of an ultrasonic endoscope 2 as a probe within the body cavity inserted into the body cavity. An ultrasonic tomographic image as the real-time image is formed from the echo signal. A guide image preparing circuit gives correction processing to the guide image by the detected value of the position of an element for detection within the body cavity when the guide image is created based on the detected value of a coil 31 for image position orientation detection from the three-dimensional data of the human body. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an in-vivo probe device for diagnosing the inside of a body cavity using an in-vivo probe inserted into the body cavity.

2. Description of the Related Art Conventionally, an intracorporeal probe used for diagnosis and treatment by inserting into a body cavity such as a digestive tract, a bile pancreatic duct, or a blood vessel, such as an endoscope, an ultrasonic endoscope, and a small-diameter ultrasonic probe, is well known. These intra-body-cavity probes usually have an imaging device such as a CCD camera or an ultrasonic transducer at the tip.
These body cavity probes are typically used as body cavity probe devices integrated with a processor that creates an optical image or an ultrasonic tomographic image from signals obtained from an image sensor or an ultrasonic transducer.
Further, in recent years, an intra-body cavity probe apparatus having a navigation function for assisting these intra-body probes to easily reach a target site is known.

Among these, the intracavity probe apparatus disclosed in Japanese Patent Application Laid-Open No. 2004-113629 as a first conventional example generates an ultrasound image by an ultrasound signal obtained by transmitting and receiving ultrasound to a subject. An ultrasonic diagnostic apparatus, an ultrasonic scanning position detecting means for detecting the position of a part for transmitting and receiving ultrasonic waves, an ultrasonic image generating means for generating an ultrasonic image based on an ultrasonic signal, and ultrasonic scanning The anatomical image information of the part of the subject corresponding to the position information obtained by the position detection means is acquired from the image information holding means having schematic diagram data of the human body as a guide image, and is displayed on the same screen as the ultrasonic image. Control means for displaying.
In this body cavity probe device, an ultrasonic image is displayed as a real-time image.

Further, in this body cavity probe device, a transmission coil that generates a magnetic field and a reception coil that receives a magnetic field are used for actual position information detection, and one of these coils is an ultrasonic wave as an intracavity probe. A coil provided at the insertion end of the endoscope is attached to the subject. Therefore, this body cavity probe device can detect the posture of the subject and detect the position of the portion of the subject that transmits and receives the ultrasonic wave.
On the other hand, an intracorporeal probe device disclosed in Japanese Patent Application Laid-Open No. 2002-306403 as a second conventional example detects an insertion shape of an endoscope and obtains a video signal that depicts the insertion shape. An image generation means for generating a three-dimensional image of a subject from consecutive slice tomographic images of a three-dimensional region in advance by a CT scan of the subject; Display means for combining and displaying.

In this body cavity probe apparatus, an endoscopic image is displayed as a real-time image.
In this body cavity probe device, the actual insertion shape is detected by combining a radioactive substance that emits γ rays filled in the flexible tube of the endoscope, a scintillator that emits light by γ ray absorption, and a light receiving element. A bottom detection unit and a vertical detection unit are used.
JP 2004-113629 A JP 2002-306403 A

  The X-ray three-dimensional helical CT apparatus and the three-dimensional MRI apparatus are normally set to image in the supine position, and are different in posture from the endoscopic examination in the left supine position.

  Conventionally, since the guide image is not subjected to correction processing, various organs in the subject are displaced according to gravity at the time of endoscopic examination in the left lateral position as compared with the X-ray three-dimensional helical CT apparatus and the three-dimensional MRI apparatus. However, the mapping of points within the subject also did not match anatomically accurately.

(Object of invention)
The present invention has been made in view of the above-described points, and an object thereof is to provide an intra-body-cavity probe apparatus that can more accurately guide a real-time image as a guide image.

The present invention relates to an intracavity probe provided with an image signal acquisition means that is inserted into a body cavity of a subject and acquires a signal for creating an image of the subject at the distal end on the insertion side into the body cavity;
Image creating means for creating a real-time image in the subject from the signal obtained by the image signal obtaining means;
Guide image creation means for creating a guide image for guiding the position or orientation of the real-time image in the subject relative to the subject from three-dimensional data of the human body;
In a body cavity probe device comprising:
An image position orientation detecting element whose position is fixed with respect to the image signal acquisition means;
A subject detection element including a body cavity detection element capable of contacting the subject body cavity;
Detecting means for detecting the position and orientation of the image position / orientation detecting element and the position of the in-body cavity detecting element, and outputting the detected value;
When the guide image creation means creates the guide image based on the position and orientation detection value of the image position / orientation detection element output from the detection means, the detection value of the position of the body cavity detection element is used. Correction means for performing correction processing on the guide image;
It is characterized by comprising.

  According to the present invention, the guide image can more accurately guide the real-time image.

  Embodiments of the present invention will be described below with reference to the drawings.

A first embodiment of the present invention will be described with reference to FIGS. FIG. 1 shows a configuration of an in-vivo probe device according to a first embodiment of the present invention, FIG. 2 shows an example of using a body surface detection coil, FIG. 3 shows an in-body contact probe, and FIG. 4 shows an image processing apparatus. FIG. 5 shows reference image data stored in the reference image storage unit.
FIG. 6 shows a voxel space, FIG. 7 shows an orthogonal base in which the origin is set on the transmitting antenna to represent position / orientation data, and FIG. 8 shows the center of the ultrasonic tomogram on the subject side to the voxel space. FIG. 9 shows an explanatory diagram for mapping, and FIG. 9 shows an explanatory diagram for mapping feature points in the body cavity on the subject side to the voxel space, and FIG. 10 shows how image index data is created by the image index creating circuit.

FIG. 11 shows how the insertion shape data created by the insertion shape creation circuit is created, FIG. 12 shows three-dimensional human body image data, and FIG. 13 shows the synthesis memory that combines the image index data and the insertion shape data. FIG. 14 shows three-dimensional guide image data when observed from the ventral side of the subject, and FIG. 15 shows three-dimensional guide when observed from the foot side of the subject. Image data is shown.
FIG. 16 shows a three-dimensional guide image and ultrasonic tomographic image displayed on the display device, FIG. 17 shows a flowchart of the entire processing contents of this embodiment, and FIG. 18 shows the body on the reference image in FIG. Specific processing contents of the table feature point and body cavity feature point designation processing are shown in a flowchart, FIG. 19 is a flowchart showing specific processing contents of the correction value calculation processing in FIG. 17, and FIG. 20 is a flowchart of the processing in FIG. FIG. 21 is a flowchart showing specific processing contents of the ultrasonic tomographic image / three-dimensional guide image creation / display processing in FIG.

First, the configuration of the body cavity probe device 1 according to the first embodiment of the present invention will be described.
As shown in FIG. 1, an intra-body-cavity probe apparatus 1 according to the first embodiment includes an electronic radial scanning type ultrasonic endoscope 2 as an intra-body probe, an optical observation apparatus 3, an ultrasonic observation apparatus 4, and a position orientation. Calculation device 5, transmitting antenna 6, body surface detection coil 7, body cavity contact probe 8, A / D unit 9, image processing device 11, mouse 12, keyboard 13, and display device 14 These are connected by signal lines.
An X-ray three-dimensional helical CT device (X-ray 3 dimensional computer tomography system) 15, a three-dimensional MRI device (3 dimensional magnetic resonance imaging system) 16, and optical communication that connects them are provided outside the body cavity probe device 1. There is a high-speed network 17 such as ADSL. The X-ray three-dimensional helical CT apparatus 15 and the three-dimensional MRI apparatus 16 are connected to the image processing apparatus 11 of the main body intracavity probe apparatus 1 via the network 17.

The ultrasonic endoscope 2 has a rigid portion 21 made of a hard material such as stainless steel at the tip so that it can be inserted into body cavities such as the esophagus, stomach, and duodenum, and is flexible on the rear end side from the rigid portion 21. A long flexible portion 22 made of a certain material, and an operation portion 23 made of a hard material on the rear end side (front side) from the flexible portion 22. The hard part 21 and the flexible part 22 form an insertion part that is inserted into the body cavity.
The rigid portion 21 is provided with image signal acquisition means that optically captures and acquires an image signal as described below, fixed to the rigid portion 21.
The rigid portion 21 is provided with an optical observation window 24 formed of a cover glass, and an objective lens 25 that connects an optical image inside the optical observation window 24 and an image pickup device disposed at the image formation position thereof. For example, a CCD (charge coupled device) camera 26 is provided. An illumination light irradiation window (illumination window) (not shown) that irradiates illumination light into the body cavity is provided adjacent to the optical observation window 24.

The CCD camera 26 is connected to the optical observation device 3 through a signal line 27. An illumination light irradiation window (not shown) is configured to irradiate illumination light and illuminate the inside of the body cavity. The image of the surface of the body cavity is formed on the CCD camera 26 from the optical observation window 24 via the objective lens 25, and the CCD signal from the CCD camera 26 is an image that generates a real-time image of the optical image via the signal line 27. It is output to the optical observation apparatus 3 as a creation means.
In addition, the rigid portion 21 is also provided with image signal acquisition means for acquiring an echo signal as an image signal by acoustic imaging.
For example, a cylindrical tip portion of the rigid portion 21 is finely cut into a strip shape, and an ultrasonic transducer group arranged in an annular shape around the insertion shaft is provided. A sound wave transducer array 29 is formed.
Each ultrasonic transducer 29a constituting the ultrasonic transducer array 29 is connected to the ultrasonic observation device 4 as an image creating means for generating a real-time image by ultrasonic waves via the operation unit 23 via the signal line 30, respectively. Yes. The center of the ring of the ultrasonic transducer array 29 is the turning center of the ultrasonic beam by radial scanning described later.

Here, orthonormal bases (unit vectors in each direction) V, V3, and V12 fixed to the rigid portion 21 are defined as shown in FIG.
That is, V is parallel to the longitudinal direction (insertion axis direction) of the rigid portion 21 and, as will be described later, this V is a normal direction vector of the ultrasonic tomographic image. V ₃ orthogonal to the vector V is a 3 o'clock direction vector, and V ₁₂ is a 12 o'clock direction vector.
In the rigid portion 21, an image position / orientation detection coil 31 as an image position / orientation detection element for the ultrasonic transducer array 29 is fixed in the very vicinity of the center of the ring of the ultrasonic transducer array 29. . The image position / orientation detection coil 31 is formed integrally with a coil wound in two directions so that the two directions (axes) of the vectors V and V ₃ are directed, and detects both directions of the vectors V and V _3. It is set to be possible.

In the flexible portion 22, a plurality of insertion shape detection coils, for example, at regular intervals along the insertion axis in order to detect the insertion shape of the flexible portion 22 constituting the insertion portion in the ultrasonic endoscope 2. 32 is provided.
As shown in FIG. 1, the insertion shape detection coil 32 is a coil wound in one axial direction, and is fixed inside the flexible portion 22 so that the winding axis direction coincides with the insertion axis direction of the flexible portion 22. ing. The hard portion 21 can be detected from the position of the image position / orientation detection coil 31.
Therefore, the insertion shape detection element is more precisely constituted by the image position / orientation detection coil 31 provided in the rigid portion 21 and the insertion shape detection coil 32 provided in the flexible portion 22. Note that the plurality of insertion shape detection coils 32 as insertion shape detection elements for detecting the insertion shape are provided, for example, only at the distal end portion of the flexible portion 22, and the distal end of the insertion portion of the ultrasonic endoscope 2. You may make it detect the insertion shape of a side part.

In the present embodiment, by using a plurality of insertion shape detection coils 32 as the insertion shape detection element, the insertion shape is detected using a magnetic field. Thereby, it is possible to prevent the surgeon and patient (subject) from being exposed to radiation for detecting the insertion shape.
A flexible bending portion is often provided near the distal end of the flexible portion 22, and a plurality of insertion shape detection coils 32 may be provided only in the vicinity of the bending portion.
A position / orientation calculation apparatus 5 constituting detection means for detecting the position and orientation of the image position / orientation detection coil 31 includes a transmission antenna 6 and a plurality of A / D units 9a and 9b constituting the A / D unit 9. 9c is connected to the image processing apparatus 11 having a built-in insertion shape creating means, a three-dimensional image creating means, a synthesizing means, an image index creating means and the like through signal lines.
Among these, the position / orientation calculation device 5 and the image processing device 11 are connected by, for example, an RS-232C standard cable 33.
The transmission antenna 6 is composed of a plurality of transmission coils (not shown) having different winding axis orientations, and these transmission coils are integrally accommodated in, for example, a rectangular parallelepiped housing. Each of the plurality of transmission coils is connected to the position / orientation calculation device 5.

The A / D unit 9i (i = a to c) includes an amplifier (not shown) that amplifies an input analog signal and an analog-digital conversion circuit (not shown) that samples the amplified signal and converts it into digital data.
The A / D unit 9a is connected to the image position / orientation detection coil 31 and each of the plurality of insertion shape detection coils 32 via signal lines 34 individually.
The A / D unit 9 b is connected to the long body cavity contact probe 8 through a signal line 35. The A / D unit 9 c is individually connected to each of the plurality of body surface detection coils 7 by a signal line 36.
In addition, each arrow line of FIG. 1 and FIG. 4 mentioned later shows the flow of the following signals and data.

(A) The first dotted line is the flow of signals and data related to the optical image,
(B) Second: the broken line shows the flow of signals and data related to the ultrasonic tomogram,
(C) Third: The solid line is the flow of signals and data related to the position and the data created by processing it,
(D) Fourth: A one-dot chain line is a flow of reference image data or data created by processing the reference image data,
(E) No. 5: A thick line represents a flow of signals and data related to a final display screen obtained by combining ultrasonic tomographic image data (described later) and three-dimensional guide image data (described later).
(F) Sixth: The flow curve is the flow of signals and data related to other controls.
FIG. 2 shows a body surface detection coil 7 which forms an object detection element.

The body surface detection coil 7 is composed of four coils each wound in one axial direction, and each coil is a tape, a belt, a band, etc., and the body surface of the subject 37, specifically the abdominal body surface. A feature point (hereinafter simply referred to as a body surface feature point) is detachably fixed, and is used for position detection using the magnetic field of the body surface feature point.
In normal upper endoscopy, the subject 37 takes a so-called left-side-down position lying on the bed 38 with the left side down, and the endoscope is inserted through the mouth. I draw with a posture.
In this example, the body surface feature points are the skeletal features `` xiphoid process '' on the skeleton, `` left anterior superior iliac spine '' on the left side of the pelvis (pelvis), pelvis Explained for the four points of "right anterior superior iliac spine" on the right side of the spine and "spinous process of vertebral body" on the spine between the left and right upper anterior iliac spines To do.

These four points can be identified by the operator through palpation. Further, these four points are not in the same plane, but form an oblique coordinate system (un-orthogonal reference frame) having three vectors from the sword-like projection as an origin to other feature points as basic vectors. This oblique coordinate system is indicated by a thick line in FIG.
FIG. 3 shows the body cavity contact probe 8. The body cavity contact probe 8 has an outer cylinder 41 made of a flexible material. A body cavity detection coil 42 is fixedly provided at the front end of the outer cylinder 41, and a connector 43 is provided at the rear end of the outer cylinder 41.
As shown in FIG. 3, the body cavity detection coil 42 is a coil wound in one axial direction and is fixed to the distal end of the body cavity contact probe 8. The body cavity detection coil 42 is fixed so that the winding axis direction thereof coincides with the insertion axis direction of the body cavity contact probe 8. The body cavity detection coil 42 is used to detect the position of a region of interest in the body cavity with which the tip of the body cavity contact probe 8 is in contact.

As shown in FIG. 1, the ultrasonic endoscope 2 is a treatment instrument insertion port for inserting forceps or the like as a first opening from the operation unit 23 through the flexible unit 22 to the rigid unit 21 (first opening). In the following, for the sake of simplicity, a tubular treatment instrument channel 46 is provided, which is provided with a forceps opening 44, and the rigid portion 21 is provided with a protruding opening 45 as a second opening.
The treatment instrument channel 46 is configured such that the body cavity contact probe 8 can be inserted through the forceps opening 44 and protrude from the protrusion 45. The opening direction of the protrusion 45 is directed so that the body cavity contact probe 8 falls within the optical field of view of the optical observation window 24 when the body cavity contact probe 8 protrudes from the protrusion 45.
FIG. 4 shows an image processing apparatus 11 incorporating an insertion shape creation means, a three-dimensional image creation means, a synthesis means, an image index creation means, and the like.

The image processing apparatus 11 includes a matching circuit 51, an image index creation circuit 52, an insertion shape creation circuit 53, a communication circuit 54, a reference image storage unit 55, an interpolation circuit 56, and a three-dimensional human body image creation circuit 57. The composition circuit 58, the rotation conversion circuit 59, and a three-dimensional image creation circuit 60 for creating three-dimensional guide images in two different gaze directions (hereinafter referred to as a three-dimensional guide image creation circuit A and a three-dimensional guide image creation circuit B) A mixing circuit 61, a display circuit 62, and a control circuit 63.
The matching circuit 51 receives position / orientation data output from the position / orientation calculation device 5 constituting detection means for detecting the position and orientation of the insertion shape detecting element or the like.
Then, the matching circuit 51 maps the position / orientation data calculated on the orthogonal coordinate axis O-xyz as described later according to a predetermined conversion formula, and creates a new position on the orthogonal coordinate axis O′-x′y′z ′. -Calculate orientation data.

Then, the matching circuit 51 outputs the new position / orientation data as position / orientation mapping data to an image index creating circuit 52 for creating image index data and an insertion shape creating circuit 53 for creating insertion shape data. To do.
The communication circuit 54 includes a large-capacity and high-speed communication modem, and is connected to the X-ray three-dimensional helical CT device 15 that generates three-dimensional data of the human body, the three-dimensional MRI device 16 and the network 17. Yes.
The reference image storage unit 55 includes a hard disk drive or the like that can store a large amount of data. The reference image storage unit 55 stores a plurality of reference image data as anatomical image information.

As shown in FIG. 5, the reference image data is tomographic image data of the subject 37 obtained from the X-ray three-dimensional helical CT apparatus 15 and the three-dimensional MRI apparatus 16 via the network 17. In the present embodiment, hereinafter, the reference image data is data of a square tomogram perpendicular to the body axis (the axis extending from the head to the foot) of the subject 37, with a pitch of 0.5 mm to several mm and several tens of cm on each side. .
When obtaining a tomographic image of the subject 37, the X-ray exposure received by the subject 37 can be reduced or eliminated by using the three-dimensional MRI apparatus 16 more frequently than the X-ray three-dimensional helical CT apparatus 15.
The reference image data in the reference image storage unit 55 in FIG. 5 are numbered from 1 to N for convenience of explanation.
Here, as shown in FIG. 5, the orthogonal coordinate axes O′-x′y′z ′ fixed for a plurality of reference image data and their orthonormal bases (unit vectors in each axis direction) i ′, j ′, k 'Is defined on the reference image data by defining the origin O as the lowermost left of the first reference image data.

As shown in FIG. 4, the interpolation circuit 56 and the synthesis circuit 58 each have a built-in volume memory VM. For convenience of explanation, hereinafter, the volume memory VM provided in the interpolation circuit 56 is referred to as an interpolation memory 56a, and the volume memory provided in the synthesis circuit 58 is referred to as a synthesis memory 58a.
The volume memory VM is configured to store a large amount of data. A voxel space is allocated to a part of the storage area of the volume memory VM. As shown in FIG. 6, the voxel space is composed of memory cells (hereinafter referred to as voxels) having addresses corresponding to the orthogonal coordinate axes O′-x′y′z ′.
A three-dimensional human body image creation circuit 57 that creates a three-dimensional human body image shown in FIG. 4 and a rotation conversion circuit 59 that performs rotation conversion perform image processing such as extraction of voxels and pixels by luminance, rotation conversion, similarity conversion, and parallel movement. It incorporates a high-speed processor (not shown) that performs at high speed.

The display circuit 62 has a switch 62a for switching the input. The switch 62a has an input terminal α, an input terminal β, an input terminal γ, and one output terminal. The input terminal α is connected to the reference image storage unit 55. The input terminal β is connected to an output terminal (not shown) of the optical observation device 3. The input terminal γ is connected to the mixing circuit 61. The output terminal is connected to a display device 14 that displays an optical image, an ultrasonic tomographic image, a three-dimensional guide image, and the like.
The control circuit 63 is connected to each unit and each circuit by a signal line (not shown) so that a command can be output to each unit and each circuit in the image processing apparatus 11. The control circuit 63 is directly connected to the ultrasound observation apparatus 4, the mouse 12, and the keyboard 13 through control lines.
As shown in FIG. 1, the keyboard 13 is provided with a body cavity feature point designation key 65, a scanning control key 66, a display switching key 13α, a display switching key 13β, and a display switching key 13γ.

When the display switching key 13α, 13β, or 13γ is pressed, the control circuit 63 outputs a command to the display circuit 62 to switch the switch 62a to the input terminal α, β, or γ. The switch 62a switches to the input terminal α when the display switching key 13α is pressed, switches to the input terminal β when the display switching key 13β is pressed, and switches to the input terminal γ when the display switching key 13γ is pressed.
The above-described signals and data from (a) No. 1 to (f) No. 6 will be described in order. (A) The operation of this embodiment will be described along the flow of signals and data related to the first optical image indicated by the dotted line.
An illumination light irradiation window (not shown) of the rigid portion 21 irradiates the illumination light toward the optical visual field range side. The CCD camera 26 images an object in the optical field of view, photoelectrically converts it, and outputs a CCD signal to the optical observation device 3.

The optical observation device 3 creates real-time image data in the optical visual field range based on the input CCD signal, and uses this data as optical image data for the input terminal β of the switch 62a of the display circuit 62 in the image processing device 11. Output to.
(B) The operation of this embodiment will be described along the flow of signals and data related to the second ultrasonic tomographic image.
When the surgeon presses the scanning control key 66, the control circuit 63 outputs a scanning control signal for instructing the ON / OFF control of the radial scanning described later to the ultrasonic observation apparatus 4.
The ultrasonic observation device 4 selects a part and a plurality of ultrasonic transducers 29a among the ultrasonic transducers 29a constituting the ultrasonic transducer array 29, and transmits a pulse voltage-like excitation signal. The part and the plurality of ultrasonic transducers 29a receive the excitation signal and convert it into ultrasonic waves that are a dense wave of the medium.

At this time, the ultrasonic observation apparatus 4 delays each excitation signal so that the time at which each excitation signal arrives at each ultrasonic transducer 29a is different. The value (delay amount) of the delay is adjusted so that one ultrasonic beam is formed when the ultrasonic waves excited by the ultrasonic transducers 29 a are superimposed in the subject 37.
The ultrasonic beam is irradiated to the outside of the ultrasonic endoscope 2, and the reflected wave from the inside of the subject 37 returns to each ultrasonic transducer 29a along a path opposite to the ultrasonic beam.
Each ultrasonic transducer 29a converts the reflected wave into an electrical echo signal and transmits it to the ultrasonic observation apparatus 4 through a path opposite to the excitation signal.
The ultrasonic observation apparatus 4 includes a center of the ring of the ultrasonic transducer array 29 so that the ultrasonic beam swirls within a plane perpendicular to the rigid portion 21 and the flexible portion 22 (hereinafter referred to as a radial scanning plane). The plurality of ultrasonic transducers 29a involved in the formation of the ultrasonic beam are selected again, and the excitation signal is transmitted again. In this way, the transmission angle of the ultrasonic beam changes. By repeating this repeatedly, so-called radial scanning is realized.

At this time, the ultrasonic observation device 4 uses the echo signal converted from the reflected wave by the ultrasonic transducer 29 a to 1 perpendicular to the insertion axis of the rigid portion 21 for one radial scan of the ultrasonic transducer array 29. One piece of digitized ultrasonic tomographic image data is created as a real-time image and output to the mixing circuit 61 of the image processing apparatus 11. At this time, the ultrasonic observation apparatus 4 creates ultrasonic tomographic image data by processing it into a square.
As described above, in this embodiment, since the ultrasonic observation apparatus 4 reselects the plurality of ultrasonic transducers 29a involved in the formation of the ultrasonic beam and transmits the excitation signal again, For example, the 12 o'clock direction is determined by which ultrasonic transducer 29a selects and transmits the excitation signal as the 12 o'clock direction.
Thus, the normal direction vector V, 3 o'clock direction vector V ₃ , and 12 o'clock direction vector V _{12 of the} ultrasonic tomographic image are defined. Furthermore, the ultrasonic observation apparatus 4 creates ultrasonic tomographic image data in a direction observed from a direction −V opposite to the normal direction vector V.

Radial scanning by the ultrasonic transducer array 29, generation of ultrasonic tomographic image data by the ultrasonic observation apparatus 4, and output to the mixing circuit 61 are performed in real time. In this embodiment, an ultrasonic tomographic image is generated as a real-time image.
(C) Next, the operation of the present embodiment will be described along the flow of signals and data related to the third position and data created by processing it.
The position / orientation calculation apparatus 5 excites a transmission coil (not shown) of the transmission antenna 6. The transmitting antenna 6 applies an alternating magnetic field in the space. Two coils with winding axes wound in the directions of vectors V and V ₃ constituting the image position / orientation detection coil 31 for detecting the position and orientation (direction) of the image signal acquisition means using ultrasonic waves, and The plurality of insertion shape detection coils 32 for detecting the insertion shape of the flexible portion 22, the body cavity detection coil 42 and the body surface detection coil 7 as the subject detection elements respectively detect alternating magnetic fields. The alternating magnetic field is converted into each position electric signal and output to the A / D units 9a, 9b, 9c.

The A / D units 9 a, 9 b, 9 c amplify the position electrical signal with an amplifier, sample it with an analog-digital conversion circuit, convert it into digital data, and output the digital data to the position orientation calculation device 5.
Next, the position / orientation calculation apparatus 5 calculates the position of the image position / orientation detection coil 31 and the direction of the winding axis perpendicular thereto, that is, the vectors V and V ₃ based on the digital data from the A / D unit 9a. calculate. Then, the position orientation calculation device 5, by calculating the outer product V × V ₃ of the winding axis direction of the vector V and V ₃ of orthogonal, to calculate the 12 o'clock direction of the vector V ₁₂ is the remaining orthogonal direction . In this way, the position / orientation calculation apparatus 5 calculates three orthogonal directions, that is, vectors V, V ₃ and V ₁₂ .
Next, the position / orientation calculation device 5 uses the digital data from the A / D units 9a to 9c to determine the position of each of the plurality of insertion shape detection coils 32 and each of the body surface detection coils 7. The position and the position of the body cavity detection coil 42 are calculated.

Next, the position / orientation calculation apparatus 5 includes the position and orientation of the image position / orientation detection coil 31, the positions of the plurality of insertion shape detection coils 32, and the four body surface detection coils 7. The position and the position of the body cavity detection coil 42 are output to the matching circuit 51 of the image processing apparatus 11 as position / orientation data.
Next, details of the position / orientation data will be described below.
In this embodiment, the origin O is defined on the transmitting antenna 6 as shown in FIG. 7, and the orthogonal coordinate axis O-xyz and its normal orthogonal base (in each axial direction) are set on the actual space where the operator examines the subject 37. Unit vectors) i, j, k are defined.
The position of the image position / orientation detection coil 31 is O ". Since the image position / orientation detection coil 31 is fixed very close to the center of the ring of the ultrasonic transducer array 29, the position O" It coincides with the center of the ultrasonic tomogram.

Here, the position / orientation data is defined as follows.
Each direction component of the position vector OO "of the position O" of the image position / orientation detection coil 31 on the orthogonal coordinate axis O-xyz:
(x0, y0, z0)
Each angle component of Euler angles (described later) indicating the orientation of the image position orientation detection coil 31 with respect to the orthogonal coordinate axis O-xyz:
(ψ, θ, φ)
Each direction component of each position vector of the plurality of insertion shape detection coils 32 on the orthogonal coordinate axis O-xyz:
(xi, yi, zi) (i is a natural number from 1 to the total number of insertion shape detection coils 32)
Each direction component of each position vector of the four body surface detection coils 7 on the orthogonal coordinate axis O-xyz:
(xa, ya, za), (xb, yb, zb), (xc, yc, zc), (xd, yd, zd)
Each direction component of the position vector of the body cavity detection coil 42 on the orthogonal coordinate axis O-xyz:
(xp, yp, zp)
Here, the Euler angle is obtained by adding rotation around the z axis, rotation around the y axis, and rotation around the z axis in this order to the orthogonal coordinate axis O-xyz in FIG. The angle is such that the directions of the axes coincide.

I = V ₃ after rotation, j = V ₁₂ after rotation, k = V after rotation
ψ is the rotation angle around the first z axis, θ is the rotation angle around the y axis, and φ is the rotation angle around the z axis again.
H in FIG. 7 is an intersection of a perpendicular line from the position O ″ to the xy plane and the xy plane. Each angular component (ψ, θ, φ) of this Euler angle is the orientation of the image position orientation detecting coil 31. That is, it corresponds to the orientation of ultrasonic tomographic image data.
The matching circuit 51 converts the position / orientation expressed on the orthogonal coordinate axis O-xyz from the following first, second, third, and fourth data groups to the orthogonal coordinate axis O′-x′y′z ′. The conversion formula that maps to the position / orientation in the voxel space expressed above is calculated.
This calculation method will be described later. In addition, the position / orientation data described in the following first and second changes depending on the body movement of the subject 37. A conversion formula is also newly created as the body movement of the subject 37 changes. New creation of this conversion formula will also be described later.

The first data group includes a body surface detection coil 7 attached to each of the xiphoid process, upper left anterior iliac spine, upper right anterior iliac spine, and lumbar spine process of the subject 37 in the position / orientation data. Direction component (xa, ya, za), (xb, yb, zb), (xc, yc, zc), (xd, yd, zd) of the position vector on the orthogonal coordinate axis O-xyz.
FIG. 8 shows the body surface detection coil 7 attached thereto.
The second data group is each direction component (xp, yp, zp) of the position vector of the body cavity detection coil 42 in the orthogonal coordinate axis O-xyz in the position / orientation data.
In FIG. 9, the body cavity contact probe 8 in which the body cavity detection coil 42 is fixed and housed is shown by a thick dotted line.
The third data group includes the xiphoid process, the upper left anterior iliac spine, the upper right anterior iliac spine, and the lumbar spine spinous processes on any of the reference image data Nos. 1 to N, respectively. (Xa ', ya', za '), (xb', yb ', zb'), (xc ', yc',) on the orthogonal coordinate axis O'-x'y'z 'of the pixel closest to the body surface zc '), (xd', yd ', zd').

These pixels are designated in advance on any of the reference image data Nos. 1 to N by the operator. This designation method will be described later.
FIG. 9 shows these pixels with black circles ● and white circles ○. (xa ', ya', za '), (xb', yb ', zb'), (xc ', yc', zc '), (xd', yd ', zd') are as shown in FIG. The body surface feature point coordinates are read from the reference image storage unit 55 to the matching circuit 51.
The fourth data group includes coordinates (xp ", yp", xp ", yp", pixels on the orthogonal coordinate axes O'-x'y'z 'of pixels corresponding to the duodenal papilla on any of the reference image data Nos. 1 to N. zp ").
These pixels are designated in advance on any of the reference image data Nos. 1 to N by the operator.

This designation method will be described later. In FIG. 9, this pixel is indicated by P ″. The coordinates (xp ″, yp ″, zp ″) of the fourth pixel are matched from the reference image storage unit 55 as body cavity feature point coordinates as shown in FIG. Read to circuit 51.
Next, the matching circuit 51 maps the position / orientation data calculated on the orthogonal coordinate axis O-xyz in accordance with the above conversion formula, and calculates new position / orientation data on the orthogonal coordinate axis O′-x′y′z ′. To do.
Next, the matching circuit 51 outputs the new position / orientation data to the image index creating circuit 52 and the insertion shape creating circuit 53 as position / orientation mapping data.
The image index creation circuit 52 includes each direction component (x0, y0, z0) of the position vector OO "of the position O" of the image position / orientation detection coil 31 on the orthogonal coordinate axis O-xyz, and the image position relative to the orthogonal coordinate axis O-xyz. Image index data is created from position / orientation mapping data having a total of six degrees of freedom with each of the Euler angle components (ψ, θ, φ) indicating the orientation of the orientation detection coil 31, and is output to the synthesis circuit 58.

This is shown in FIG. That is, image index data is created from the upper position / orientation mapping data in FIG. 10 as shown in the lower part of FIG.
This image index data includes, for example, a parallelogram ultrasonic tomogram marker Mu, a blue tip direction marker Md (indicated as blue in FIG. 10), and a yellow green arrow-shaped 6 o'clock direction marker Mt (in FIG. 10). Image data on the Cartesian coordinate axis O'-x'y'z '.
The insertion shape creation circuit 53 includes each direction component (x0, y0, z0) of the position vector OO "of the position O" of the image position / orientation detection coil 31 and a plurality of insertion shape detection coils on the orthogonal coordinate axis O-xyz. Insertion shape data is created (by interpolation and marker creation processing) from the position / orientation mapping data of each of the 32 position vectors and each direction component (xi, yi, zi), and output to the synthesis circuit 58.
This is shown in FIG. The insertion shape data includes a string-like insertion shape marker Ms obtained by interpolating the positions of the image position / orientation detection coil 31 and the plurality of insertion shape detection coils 32 in order, and a coil position marker Mc indicating each coil position. Is the image data on the orthogonal coordinate axis O′-x′y′z ′.

(D) Next, the operation of this embodiment will be described along the flow of fourth reference image data and data created by processing it.
The surgeon acquires reference image data for the entire abdomen of the subject 37 in advance using the X-ray three-dimensional helical CT apparatus 15 and the three-dimensional MRI apparatus 16 on the subject 37.
The surgeon presses a predetermined key on the keyboard 13 or selects a menu on the screen with the mouse 12 to instruct acquisition of reference image data. At the same time, the surgeon also instructs the supplier. In response to this instruction, the control circuit 63 instructs the communication circuit 54 to take in reference image data and obtain it.
For example, when the acquisition destination is the X-ray three-dimensional helical CT apparatus 15, the communication circuit 54 takes in a plurality of two-dimensional CT images from the network 17 as reference image data and stores them in the reference image storage unit 55. .

When imaging with the X-ray three-dimensional helical CT apparatus 15, an X-ray contrast medium is injected from the blood vessel of the subject 37 before imaging, and blood vessels such as aorta and superior mesenteric vein (superior mesenteric vein) In a broad sense, a blood vessel) or an organ containing many blood vessels is displayed on a two-dimensional CT image with high or medium luminance so that a difference in luminance from the surrounding tissue is likely to occur.
For example, when the acquisition destination is the three-dimensional MRI apparatus 16, the communication circuit 54 captures a plurality of two-dimensional MRI images from the network 17 as reference image data and stores them in the reference image storage unit 55.
When imaging with the three-dimensional MRI apparatus 16, an MRI contrast agent with high sensitivity of nuclear magnetic resonance is injected from the blood vessel of the subject 37 before imaging, and many blood vessels such as the aorta and superior mesenteric vein and many blood vessels are injected. The organs included are displayed on the two-dimensional MRI image with high or medium luminance so that a difference in luminance from the surrounding tissue is likely to occur.

Hereinafter, since the operation is the same as when the operator selects the X-ray 3D helical CT apparatus 15 as the acquisition destination and when the 3D MRI apparatus 16 is selected, the X-ray 3D helical CT apparatus 15 as the acquisition destination. Only when the communication circuit 54 captures a plurality of two-dimensional CT images as reference image data will be described.
FIG. 5 shows an example of reference image data stored in the reference image storage unit 55. Due to the action of the X-ray contrast medium, blood vessels such as the aorta and superior mesenteric veins have high brightness, organs containing many peripheral blood vessels such as pancreas have medium brightness, and duodenum etc. have low brightness. Has been.
The interpolation circuit 56 reads all the reference image data from No. 1 to No. N from the reference image storage unit 55. Next, the interpolation circuit 56 fills the read reference image data into the voxel space of the interpolation memory 56a.

Specifically, the luminance of each pixel of the reference image data is output to a voxel having an address corresponding to the pixel. Next, the interpolation circuit 56 performs interpolation based on the luminance value of the adjacent reference image data, and fills the vacant voxel with the data. In this way, all the voxels in the voxel space are filled with data based on the reference image data (hereinafter referred to as voxel data). The three-dimensional human body image creation circuit 57 extracts voxels (mainly blood vessels) with high luminance values and voxels with medium luminance values (mainly organs including many peripheral blood vessels such as pancreas) from the interpolation circuit 56 for each luminance value range. Separately and color.
Next, the three-dimensional human body image creation circuit 57 fills the extracted voxels into the voxel space of the synthesis memory 58a of the synthesis circuit 58 as three-dimensional human body image data. At this time, the three-dimensional human body image creation circuit 57 fills the extracted voxel addresses in the interpolation memory 56a so that the addresses in the voxel space in the synthesis memory 58a are the same.

FIG. 12 shows an example of 3D human body image data. In the example shown in FIG. 12, the three-dimensional human body image data is obtained by extracting the aorta, which is a high-luminance blood vessel, the superior mesenteric vein, and the pancreas, which is a medium-luminance organ. The blood vessel is red and the pancreas is green. And is shown as three-dimensional data observed from the ventral side with the head side of the subject 37 on the right side and the foot side on the left side.
The three-dimensional human body image creation circuit 57 also has a function of extraction means for extracting organs, blood vessels, and the like. This extraction means may be provided on the three-dimensional guide image creation circuits A and B side. The three-dimensional guide image creation circuits A and B may be configured so that organs and blood vessels can be selected when creating a three-dimensional guide image.

The synthesis circuit 58 fills the image index data and the insertion shape data in the voxel space in the synthesis memory 58a. This is shown in FIG.
In FIG. 13, for convenience of explanation, three-dimensional human body image data existing in the voxel space is omitted (when three-dimensional human body image data is not omitted, it is shown in FIG. 14 and the like). In this way, the synthesis circuit 58 embeds the three-dimensional human body image data, the image index data, and the insertion shape data in the same synthesis memory in the same voxel space. Hereinafter, they are synthesized as synthesized three-dimensional data).

The rotation conversion circuit 59 reads the combined three-dimensional data and performs a rotation process on the combined three-dimensional data in accordance with the rotation instruction signal from the control circuit 63.
The three-dimensional guide image creation circuit A performs rendering processing such as hidden surface removal and shading on the combined three-dimensional data, and creates image data (hereinafter, three-dimensional guide image data) that can be output to the screen.
The default direction of the three-dimensional guide image data is the direction from the ventral side of the human body. Therefore, the three-dimensional guide image creation circuit A creates three-dimensional guide image data observed in the direction from the ventral side of the subject 37. Note that the default orientation of the 3D guide image data is the orientation from the abdomen side of the human body, but 3D guide image data of the orientation from the back side may be created. Further, three-dimensional guide image data from other directions may be created.
The three-dimensional guide image creation circuit A outputs the three-dimensional guide image data observed from the subject's ventral side to the mixing circuit 61. The three-dimensional guide image data is shown in FIG. The right side of FIG. 14 is the subject head side, and the left side is the subject foot side.

In the three-dimensional guide image data of FIG. 14, the ultrasonic tomographic image marker Mu in the image index data is made translucent, the 6 o'clock direction marker Mt and the tip direction marker Md of the image index data, and the insertion shape of the insertion shape data The marker Ms and the coil position marker Mc are seen through.
For other organs, the ultrasonic tomographic image marker Mu is made opaque so that the portion on the back side of the ultrasonic tomographic image marker Mu cannot be seen. In FIG. 14, each marker that is behind the ultrasonic tomographic image marker Mu and overlaps the ultrasonic tomographic image marker Mu is indicated by a broken line.
The three-dimensional guide image creation circuit B performs rendering processing such as hidden surface removal and shading on the combined three-dimensional data subjected to the rotation processing, and creates three-dimensional guide image data that can be output to the screen.

In the present embodiment, as an example, the rotation instruction signal from the control circuit 63 is rotated by 90 degrees by the operator's mouse 12 and keyboard 13 and the instruction content is to be observed from the foot side. It shall have been.
Accordingly, the 3D guide image creation circuit B creates 3D guide image data observed in the direction from the subject's foot side.
The three-dimensional guide image creation circuit B outputs the three-dimensional guide image data observed from the subject's foot side to the mixing circuit 61. This three-dimensional guide image data is shown in FIG. The right side of FIG. 15 is the subject right side, and the left side is the subject left side.
In the three-dimensional guide image data of FIG. 15, the ultrasonic tomographic image marker Mu in the image index data is made translucent, the 6 o'clock direction marker Mt and the tip direction marker Md of the image index data, and the insertion shape of the insertion shape data The marker Ms and the coil position marker Mc are seen through.

For other organs, the ultrasonic tomographic image marker Mu is made opaque so that the portion on the back side of the ultrasonic tomographic image marker Mu cannot be seen. In FIG. 15, each marker that is behind the ultrasonic tomographic image marker Mu and overlaps the ultrasonic tomographic image marker Mu is indicated by a broken line.
Note that the display of the ultrasonic tomographic image marker Mu in FIG. 15 is not a case where the normal line of the ultrasonic tomographic image marker Mu is aligned so as to coincide with the observation line of sight, that is, the screen normal line of the display device 14, that is, non-facing. Display.
(E) Next, the operation of this embodiment will be described along the flow of signals and data related to the final display screen obtained by synthesizing the fifth ultrasonic tomographic image data and the three-dimensional guide image data.
The mixing circuit 61 in FIG. 4 includes ultrasonic tomographic image data from the ultrasonic observation apparatus 4, three-dimensional guide image data obtained by observing the subject 37 from the three-dimensional guide image creation circuit A from the ventral side, and a three-dimensional guide. 3D guide image data obtained by observing the object 37 from the image creation circuit B from the foot side is arranged to create mixed data for display.

The display circuit 62 converts the mixed data into an analog video signal and outputs it to the display device 14.
The display device 14 displays the ultrasonic tomographic image, the three-dimensional guide image obtained by observing the subject 37 from the foot side, and the three-dimensional guide image observed from the abdominal side, side by side, based on the analog video signal.
As shown in FIG. 16, the display device 14 displays each organ represented on the three-dimensional guide image by color-coding by organ according to the color corresponding to the luminance value on the reference image data.
In the display example of FIG. 16, the pancreas is displayed in green, the aorta, and the superior mesenteric vein is displayed in red. In FIG. 16, each marker that is behind the ultrasonic tomographic image marker Mu and overlaps the ultrasonic tomographic image marker Mu is indicated by a broken line.
In addition, as shown by white arrows in FIG. 16, the two three-dimensional guide images move in conjunction with the movement of the radial scanning plane.

(F) Sixth, the operation of this embodiment will be described along the flow of signals and data related to control.
The matching circuit 51, the image index creation circuit 52, the insertion shape creation circuit 53, the communication circuit 54, the reference image storage unit 55, the interpolation circuit 56, and the 3D human body image creation in the image processing apparatus 11 of FIG. The circuit 57, the synthesis circuit 58, the rotation conversion circuit 59, the three-dimensional guide image creation circuit A, the three-dimensional guide image creation circuit B, the mixing circuit 61, and the display circuit 62 are commands from the control circuit 63. Controlled by
Details of the control will be described later.
Hereinafter, the entire operation of the image processing device 11, the keyboard 13, the mouse 12, and the display device 14 according to the present embodiment will be described in accordance with the usage pattern of the surgeon. FIG. 17 is a flowchart of the entire process, and the processes in steps S1 to S4 are executed in this order.

The first step S1 is a body surface feature point and body cavity feature point designation process on the reference image data. That is, in this step S1, processing for designating body surface feature points and body cavity feature points is performed on the reference image data.
In the next step S <b> 2, the surgeon fixes the body surface detection coil 7 to the subject 37. The surgeon places the subject 37 in a body position with its left side thin, so-called left-sided position. The surgeon palpates the subject 37 and places the body at a position on the body surface closest to the four body surface feature points xiphoid process, left upper anterior iliac spine, right upper iliac spine, and lumbar spine process. The table detection coil 7 is fixed.
The next step S3 is a correction value calculation process.
In this step S3, the image processing apparatus 11 acquires the position / orientation data of the feature points in the body cavity, and the position / orientation data expressed on the orthogonal coordinate axis O-xyz on the orthogonal coordinate axis O'-x'y'z '. The conversion formula for mapping to the position / orientation mapping data in the voxel space expressed in (4) is calculated, and further, the correction value of the conversion formula is calculated from the position / orientation data of the feature points in the body cavity.

In the next step S4, ultrasonic tomographic image / three-dimensional guide image creation / display processing is performed. This step S4 is processing for creating and displaying an ultrasonic tomographic image and a three-dimensional guide image.
Next, the processing in step S1 in FIG. 17, that is, the body surface feature point and body cavity feature point designation processing on the reference image data will be specifically described.
FIG. 18 shows details of processing for designating body surface feature points and body cavity feature points on the reference image data in step S1 of FIG.
In the first step S1-1, the operator presses the display switching key 13α. The control circuit 63 issues a command to the display circuit 62. The switch 62a of the display circuit 62 is switched to the input terminal α according to a command.
In the next step S1-2, the surgeon uses the mouse 12 and the keyboard 13 to specify any reference image data from 1 to N.

In the next step S <b> 1-3, the control circuit 63 reads the designated reference image data from any one of the reference image data 1 to N stored in the reference image storage unit 55 in the display circuit 62. Let
The display circuit 62 converts the reference image data from the reference image storage unit 55 into an analog video signal, and outputs the reference image data to the display device 14. The display device 14 displays reference image data.
In the next step S1-4, the operator uses the mouse 12 and the keyboard 13 to designate body surface feature points on the reference image data. Specifically, it is as follows.
The surgeon shows in the displayed reference image data any of the four body surface feature points of the subject 37, the xiphoid process, the upper left anterior iliac spine, the upper right anterior iliac spine, and the lumbar spine spinous process. Like that. If none is shown, the process returns to step S1-2, and the surgeon respecifies other reference image data and repeats display of different reference image data until the reference image data shown in step S1-3 is displayed. .

The operator uses the mouse 12 and the keyboard 13 to display four points on the surface of the body of the subject 37 on the displayed reference image data, the upper left iliac spine, the upper right iliac spine, the lumbar vertebrae A pixel corresponding to a point on the body surface closest to the spinous process is designated.
The designated points are indicated by black circles ● and white circles ○ in FIGS. 8 and 9. In this embodiment, for the convenience of explanation, the xiphoid process ○ is shown on the reference image data of n1 (1 ≦ n1 ≦ N), and the upper left anterior iliac spine, the upper right anterior iliac spine, and the lumbar spinous process In the following description, it is assumed that ● is reflected on the reference image data of n2 (1 ≦ n2 ≦ N).
In FIGS. 8 and 9, for the convenience of explanation, sword-shaped protrusions are indicated by circles at positions corresponding to the sword-shaped protrusions on the reference image data of the number n2.

In the next step S1-5, the operator designates the body cavity feature point P "using the mouse 12 and the keyboard 13. In this embodiment, as the body cavity feature point P", the duodenal papilla (to the duodenum of the common bile duct) is designated. A description will be given by taking an example of an opening (duodenal papilla). Specifically, it is as follows.
The surgeon uses the mouse 12 and the keyboard 13 to specify any one of the reference image data from 1 to N.
The control circuit 63 reads designated reference image data out of any one of reference image data 1 to N stored in the reference image storage unit 55 via a signal line (not shown) in the display circuit 62. Let
The display circuit 62 outputs the read reference image data to the display device 14. The display device 14 displays this reference image data. If the duodenal papilla, which is a feature point in the body cavity of the subject 37, is not reflected in the displayed reference image data, the surgeon respecifies the other reference image data and is different until the reflected reference image data is displayed. Repeat display of reference image data.

The operator uses the mouse 12 and the keyboard 13 to designate a pixel corresponding to the duodenal papilla that is a point in the body cavity of the subject 37 on the displayed reference image data.
The designated point is indicated by P "in FIG. 9. In this embodiment, for convenience of explanation, it is assumed that the duodenal papilla P" is shown on the reference image data of n2 (1 ≦ n2 ≦ N).
In the next step S1-6, the control circuit 63 determines each pixel corresponding to each body surface feature point designated in step S1-4 and a pixel corresponding to the body cavity feature point P "designated in step S1-5. , The coordinates on the orthogonal coordinate axes O′-x′y′z ′ stretched in the voxel space from the address on the reference image data are calculated and output to the matching circuit 51.

The calculated values of the coordinates on the orthogonal coordinate axes O′-x′y′z ′ of the pixels corresponding to the body surface feature points designated in step S1-4 are (xa ′, ya ′, za ′), (xb ', yb', zb '), (xc', yc ', zc'), (xd ', yd', zd ').
The calculated value of each coordinate on the pixel orthogonal coordinate axis O′-x′y′z ′ corresponding to the feature point in the body cavity designated in step S1-5 is set to (xp ″, yp ″, zp ″).
The matching circuit 51 stores this coordinate. After step S1-6 is completed, the process proceeds to step S2 in FIG. Then, after the process of step S2, the process proceeds to the correction value calculation process of step S3 of FIG.
FIG. 19 shows details of the correction value calculation processing in step S3. As described above, this step S3 obtains the position / orientation data of the feature points in the body cavity, and the position / orientation data expressed on the orthogonal coordinate axis O-xyz on the orthogonal coordinate axis O'-x'y'z '. This is a process of calculating a conversion formula that maps to the expressed position / orientation mapping data in the voxel space, and further calculating a correction value of the conversion formula from the position / orientation data of the feature points in the body cavity.

When the correction value calculation process in step S3 is started, in the first step S3-1, the operator presses the display switching key 13β. In response to this instruction, the control circuit 63 issues a command to the display circuit 62. The switch 62a of the display circuit 62 is switched to the input terminal β according to a command.
Next, in step S 3-2, the display circuit 62 converts the optical image data from the optical observation device 3 into an analog video signal, and outputs the optical image to the display device 14. The display device 14 displays an optical image.
In the next step S3-3, the operator inserts the rigid portion 21 and the flexible portion 22 of the ultrasonic endoscope 2 into the body cavity into the subject 37.
In the next step S3-4, the surgeon searches the feature point in the body cavity by moving the rigid portion 21 while observing the optical image. After the feature point in the body cavity is found, the operator moves the rigid portion 21 to the vicinity of the feature point in the body cavity.

In the next step S3-5, the surgeon inserts the body cavity contact probe 8 from the forceps port 44 and projects from the projecting port 45 while observing the optical image. Then, the operator brings the tip of the body cavity contact probe 8 into contact with the body cavity feature point under the optical image field of view.
This is shown in FIG. In FIG. 20, an optical image is displayed on the display screen. The optical image displays the duodenal papilla P and the body cavity contact probe 8 as examples of body cavity feature points.
In the next step S3-6, the operator presses the body cavity feature point designation key 65.
In the next step S3-7, the control circuit 63 issues a command to the matching circuit 51. The matching circuit 51 takes in the position / orientation data from the position / orientation calculation device 5 according to the command and stores it. The position / orientation data includes the following two types of data as described above.

Each direction component of each position vector of the four body surface detection coils 7 on the orthogonal coordinate axis O-xyz, that is, in this case, the coordinates of the four body surface feature points on the orthogonal coordinate axis O-xyz: (xa, ya, za), (xb, yb, zb), (xc, yc, zc), (xd, yd, zd).
Each direction component of the position vector of the body cavity detection coil 42 on the orthogonal coordinate axis O-xyz, that is, in this case, the coordinates of the body cavity feature point on the orthogonal coordinate axis O-xyz: (xp, yp, zp).
In the next step S3-8, the matching circuit 51 creates a first conversion expression that expresses the first mapping from the coordinates of the body surface feature points. Specifically, it is as follows.
First, the matching circuit 51 stores the following contents.

First, each coordinate on the orthogonal coordinate axis O′-x′y′z ′ in the voxel space of each pixel corresponding to each body surface feature point specified in step S1:
(xa ', ya', za '), (xb', yb ', zb'), (xc ', yc', zc '), (xd', yd ', zd')
Second, the coordinates of the pixel corresponding to the feature point in the body cavity designated in step S1) on the orthogonal coordinate axis O'-x'y'z 'in the voxel space:
(xp ", yp", zp ")
Thirdly, each coordinate on the orthogonal coordinate axis O-xyz of the body surface feature point captured in step S3-7:
(xa, ya, za), (xb, yb, zb), (xc, yc, zc), (xd, yd, zd)
Fourth, the coordinates of the body cavity feature points taken in step S3-7 on the orthogonal coordinate axis O-xyz:
(xp, yp, zp)
The matching circuit 51 includes the third coordinates (xa, ya, za), (xb, yb, zb), (xc, yc, zc), (xd, yd, zd) and the first coordinates. From (xa ', ya', za '), (xb', yb ', zb'), (xc ', yc', zc '), (xd', yd ', zd'), the orthogonal coordinate axis O- A first transformation expression expressing a first mapping of an arbitrary point on xyz to a point on the orthogonal coordinate axis O'-x'y'z 'in the voxel space is created. The first mapping and the first conversion formula are defined as follows.

As shown in FIG. 8, three vectors from the xiphoid process to another point using the xiphoid process, the upper left anterior iliac spine, the upper right anterior iliac spine, and the lumbar spine process, which are body surface feature points, are shown in FIG. Two oblique coordinate systems having a basic vector as the base vector and the object 37 in the voxel space (represented as reference image data in FIG. 8, but in the data space obtained by interpolating them) are virtually ( Set).
The first mapping is “coordinates of an arbitrary point on the orthogonal coordinate axis O-xyz expressed in an oblique coordinate system on the subject 37” and “this arbitrary coordinate on the orthogonal coordinate axis O′-x′y′z ′”. This is a mapping from the subject 37 to the voxel space so that the “coordinates expressed by the oblique coordinate system in the voxel space of the point after mapping the point” are the same.
In addition, the first conversion formula converts “coordinates of the arbitrary point on the orthogonal coordinate axis O-xyz” to “coordinates of the point after the first mapping in the voxel space on the orthogonal coordinate axis O′-x′y′z ′”. It is an expression to do.

For example, as shown in FIG. 8, the position of the image position / orientation detection coil 31, that is, the point after mapping by the first mapping of the center of radial scanning and the center O ″ of the ultrasonic tomographic image is Q ′.
The coordinates of the point Q ′ on the orthogonal coordinate axes O′-x′y′z ′ are (x0 ′, y0 ′, z0 ′). Using the first conversion formula, the coordinate (x0, y0, z0) of the point O ″ on the orthogonal coordinate axis O-xyz is the coordinate (x0 ′, y0) of the point Q ′ on the orthogonal coordinate axis O′-x′y′z ′. ', z0').
In the next step S3-9, as shown in FIG. 9, the matching circuit 51 maps the body cavity feature point P to the point P ′ in the voxel space by the first conversion formula. The coordinates of the body cavity feature point P on the orthogonal coordinate axis O-xyz are (xp, yp, zp). The coordinates of the point P ′ after the first mapping on the orthogonal coordinate axes O′-x′y′z ′ are defined as (xp ′, yp ′, zp ′).
In the next step S3-10, the matching circuit 51 sets the coordinates of the point P ′ on the orthogonal coordinate axes O′-x′y′z ′ in the voxel space to (xp ′, yp ′, zp ′), and step S1. From the coordinates (xp ", yp", zp ") on the orthogonal coordinate axis O'-x'y'z 'in the voxel space of the point P" corresponding to the feature point in the body cavity specified by Vector P'P "is calculated.

P'P "= (xp", yp ", zp")-(xp ', yp', zp ') = (xp "-xp', yp" -yp ', zp "-zp')
In the next step S3-11, the matching circuit 51 stores the vector P'P ". The vector P'P" is used for correcting the first conversion formula and creating the second conversion formula in the process described later. Acts as a correction value. After step S3-11 ends, the process proceeds to next step S4.
Next, the ultrasonic tomographic image / three-dimensional guide image creation / display process in step S4 will be described. FIG. 21 shows details of processing for creating and displaying an actual ultrasonic tomographic image / three-dimensional guide image of the subject 37 in step S4.
When the processing of step S4 is started, the operator presses the display switching key 13γ in the first step S4-1. The control circuit 63 issues a command to the display circuit 62. The switch 62a of the display circuit 62 is switched to the input terminal γ by this command.
In the next step S4-2, the operator presses the scan control key 66.

In the next step S4-3, the control circuit 63 outputs a scanning control signal to the ultrasonic observation apparatus 4. Then, the ultrasonic transducer array 29 starts radial scanning.
In the next step S4-4, the control circuit 63 issues a command to the mixing circuit 61. The mixing circuit 61 sequentially captures the ultrasonic tomographic image data input according to the radial scanning from the ultrasonic observation apparatus 4 in accordance with this command.
In the next step S4-5, the control circuit 63 issues a command to the matching circuit 51. The matching circuit 51 takes in the position / orientation data from the position / orientation calculation device 5 according to the command and stores it. This capture is instantaneous. Therefore, the matching circuit 51 captures position / orientation data including the following data at the moment when the mixing circuit 61 captures the ultrasonic tomographic image data in step S4-4.

Each direction component of the position of the image position / orientation detection coil 31 on the orthogonal coordinate axis O-xyz, that is, the position vector OO "of the center of radial scanning and the center O" of the ultrasonic tomographic image:
(x0, y0, z0)
Each angular component of the Euler angle indicating the orientation of the image position orientation detecting coil 31 with respect to the orthogonal coordinate axis O-xyz, that is, the orientation of the ultrasonic tomographic image:
(ψ, θ, φ)
Each direction component of each position vector of the plurality of insertion shape detection coils 32 on the orthogonal coordinate axis O-xyz:
(xi, yi, zi) (i is a natural number from 1 to the total number of insertion shape detection coils 32)
Each direction component of each position vector of the four body surface detection coils 7 on the orthogonal coordinate axis O-xyz:
(xa, ya, za), (xb, yb, zb), (xc, yc, zc), (xd, yd, zd)
In the next step S4-6, the matching circuit 51 determines each direction of the position vector of each of the four body surface detection coils 7 on the orthogonal coordinate axis O-xyz out of the position / orientation data acquired in step S4-5. Using the components (xa, ya, za), (xb, yb, zb), (xc, yc, zc), (xd, yd, zd), the first conversion formula stored in step S3) is updated.

Next, the matching circuit 51 newly creates a second conversion expression expressing the second mapping by combining the updated first conversion expression with the parallel movement by the vector P′P ″ stored in step S3. The circuit 51 creates a second conversion expression that represents the second mapping by combining the first conversion expression and the parallel movement by the vector P′P ″. The concept of the second mapping is as follows.
Second map = first map + translation by vector P′P ″ The translation by vector P′P ″ has the following correction effect. Vector P'P "acts as a correction value.
The first mapping is “coordinates of an arbitrary point on the orthogonal coordinate axis O-xyz expressed in the oblique coordinate system on the subject 37” and “the coordinates of the arbitrary point on the orthogonal coordinate axis O′-x′y′z ′”. The mapping from the subject 37 to the voxel space was made so that the “coordinates expressed by the oblique coordinate system in the voxel space of the points after mapping” were the same.

Ideally, the mapping point P ′ obtained by the first mapping of the body cavity feature point P into the voxel space and the point P ″ corresponding to the body cavity feature point specified in step S1) may coincide with each other. Desirable, but in practice it is difficult to match exactly.
This is because “the spatial positional relationship between an arbitrary point on the orthogonal coordinate axis O-xyz and the oblique coordinate system on the subject 37” and “the orthogonal coordinate axis O′-x ′ corresponding to the arbitrary point anatomically. This is because the point in y′z ′ and the spatial positional relationship between the oblique coordinate system in the voxel space do not completely match due to various factors.
In the present embodiment, the first mapping and the first transformation formula are obtained from the coordinates of the body surface feature points having features on the skeleton. The duodenal papilla P, which is a feature point in the body cavity, This is because the table feature points are not always in the same positional relationship.

This is mainly because the X-ray three-dimensional helical CT apparatus 15 and the three-dimensional MRI apparatus 16 are normally imaged in the supine position and are different in body posture from the time of the ultrasonic endoscope 2 examination in the left lateral position. The various organs in the subject 37 can be displaced according to gravity.
Therefore, the mapping of the feature point P in the body cavity is equivalent to the feature point in the body cavity in the voxel space by combining the first mapping with the parallel movement by the vector P′P ”as the correction value. Matches the point P ". Further, other points of the object 37, for example, the center O ″ of the ultrasonic tomographic image, are more accurately anatomically matched by the second mapping.
In the next step S4-7, the matching circuit 51 extracts each directional component of the position vector OO "of the center O" of the ultrasonic tomographic image on the orthogonal coordinate axis O-xyz from the position / orientation data acquired in step S4-5. (x0, y0, z0), each angular component (ψ, θ, φ) of Euler angles indicating the orientation of the image position orientation detection coil 31 with respect to the orthogonal coordinate axis O-xyz, and a plurality of angular components on the orthogonal coordinate axis O-xyz. A second transformation formula that newly creates each directional component (xi, yi, zi) (i is a natural number from 1 to the total number of insertion shape detection coils 32) of each position vector of the insertion shape detection coil 32. To convert to position / orientation mapping data.

As shown in FIG. 8, in the first conversion formula, the center O ″ of the ultrasonic tomographic image is mapped to the point Q ′ on the voxel space. By using the second conversion formula newly created in this step, As shown in Fig. 9, the center O "of the ultrasonic tomogram is mapped to a point Q" in the voxel space. The vector Q'Q "indicating the difference between Q 'and Q" is due to the translation in the second map. Since it matches the correction amount, it is the same as the vector P'P ". That is, the following expression is established.
Q'Q "= P'P"
In the next step S4-8, the image index creation circuit 52 creates image index data. The insertion shape creation circuit 53 creates insertion shape data.
The synthesizing circuit 58 synthesizes the three-dimensional human body image data, the image index data, and the insertion shape data, and creates synthesized three-dimensional data.
The rotation conversion circuit 59 performs rotation processing on the combined three-dimensional data.

The three-dimensional guide image creation circuit A and the three-dimensional guide image creation circuit B each create three-dimensional guide image data.
Each of the above processes is as described above.
In the next step S4-9, the mixing circuit 61 generates the mixed data for display by arranging the ultrasonic tomographic image data and the three-dimensional guide image data.
The display circuit 62 converts this mixed data into an analog video signal.
The display device 14 displays the ultrasonic tomographic image, the three-dimensional guide image obtained by observing the subject 37 from the ventral side, and the three-dimensional guide image observed from the foot side, side by side, as shown in FIG. .
Each of the above processes is as described above.
In the next step S4-10, the control circuit 63 confirms whether or not the operator presses the scan control key 66 again during steps S4-4 to S4-9.

If the surgeon has pressed the scan control key 66 again, the control circuit 63 ends the above processing, and outputs a scan control signal for commanding the radial scan control OFF to the ultrasound observation apparatus 4. To do. The ultrasonic transducer array 29 ends the radial scanning.
If the surgeon has not pressed the scan control key 66 again, the process jumps to step S4-4.
In this manner, by repeating the processing described in steps S4-4 to S4-9, the ultrasonic transducer array 29 performs one radial scan, and the ultrasonic observation apparatus 4 converts the ultrasonic tomographic image data. Each time the tomographic image data is created and input from the ultrasonic observation device 4 to the mixing circuit 61, two new three-dimensional guide images are created and displayed on the display screen of the display device 14 together with the new ultrasonic tomographic image. Displayed while being updated in real time.

That is, as shown in FIG. 16, the ultrasonic tomographic image marker Mu on the image index data is linked with the movement of the radial scanning plane accompanying the manual operation of the flexible portion 22 and the rigid portion 21 of the operator. The tip direction marker Md, the 6 o'clock direction marker Mt, the insertion shape marker Ms on the insertion shape data, and the coil position marker Mc move or deform on the three-dimensional human body image data.
This embodiment has the following effects.
In this embodiment, the ultrasonic endoscope 2 is provided with a fixed ultrasonic transducer array 29 for acquiring a signal for creating an ultrasonic tomographic image in the subject 37 on the insertion side into the body cavity. A hard part 21 and a flexible part 22 are provided on the front side of the hard part 21, and an ultrasonic observation device 4 that creates an ultrasonic tomographic image in the subject 37 from an echo signal acquired by the ultrasonic transducer 29 a is provided. The image position / orientation detection coil 31 is spatially fixed with respect to the rigid portion 21, and a plurality of insertion shape detection coils 32 are provided along the flexible portion 22 so as to contact the subject 37. A plurality of body surface detection coils 7 are provided, six degrees of freedom of the position and orientation of the image position / orientation detection coil 31, each position of the plurality of insertion shape detection coils 32, and the body surface detection coil 7. Detect position or orientation and output as position / orientation data An image index creating circuit that includes a communication antenna 6 and a position / orientation calculation device 5 and creates an ultrasonic tomographic image marker Mu that indicates the position and orientation of an ultrasonic tomographic image in the subject 37 created by the ultrasonic observation device 4. 52, and the combining circuit 58 generates the insertion shape on the distal end side of the flexible portion 22, the ultrasonic tomographic image marker Mu, and the three-dimensional human body image data based on the position / orientation data output from the position / orientation calculation device 5. The three-dimensional guide image creating circuits A and B for synthesizing and guiding the position and orientation of the flexible portion 22 and the ultrasonic tomographic image with respect to the subject 37 are provided.

Therefore, this embodiment can detect the insertion shape of the rigid portion 21 and the flexible portion 22 of the ultrasonic endoscope 2 and the direction of the ultrasonic tomographic image with less invasion of radiation exposure, and includes a three-dimensional guide including both. Images can be created.
In this embodiment, the image index creation circuit 52 creates image index data obtained by combining the ultrasonic tomographic image marker Mu with the blue tip direction marker Md and the yellow-green arrow-shaped 6 o'clock direction marker Mt. The synthesizing circuit 58 synthesizes the three-dimensional human body image data, the image index data, and the insertion shape data in the same voxel space, and the mixing circuit 61 combines the ultrasonic tomographic image data from the ultrasonic observation apparatus 4 and 3 The mixed data for display for displaying the dimension guide image data side by side is created, the display circuit 62 converts the mixed data into an analog video signal, and the display device 14 uses the analog video signal to produce an ultrasonic tomographic image. And the three-dimensional guide image are arranged and displayed side by side.

Therefore, this embodiment can guide the positional relationship between the ultrasonic tomogram and the region of interest such as the pancreas, and can be used as a radial scanning surface of the ultrasonic endoscope 2 with respect to the body cavity wall such as the digestive tract. It is possible to guide the orientation and shape of the flexible portion 22 and the hard portion 21.
Therefore, the surgeon can visually grasp these relationships, and can easily perform diagnosis, treatment, etc. on the region of interest.
In this embodiment, the matching circuit 51 repeats the processing described in steps S4-4 to S4-9, and the mixing circuit 61 captures position / orientation data at the moment when the ultrasonic tomographic image data is captured. A new second transformation formula is created to express the second mapping by combining the first transformation formula and the translation by the vector P'P ", and the position of the center O" of the ultrasonic tomographic image on the orthogonal coordinate axis O-xyz Each direction component (x0, y0, z0) of the vector OO ", each angle component (ψ, θ, φ) of Euler angles indicating the orientation of the image position orientation detection coil 31 with respect to the orthogonal coordinate axis O-xyz, and the orthogonal coordinate axis Each directional component (xi, yi, zi) (i is a natural number from 1 to the total number of insertion shape detection coils 32) of each position vector of the plurality of insertion shape detection coils 32 in O-xyz・ Configuration and operation to repeat the process of converting to orientation map data .

Therefore, in this embodiment, even if a change in the body position of the subject 37 occurs during the examination with the ultrasonic endoscope 2, an ultrasonic tomographic image, as long as the positional relationship between the body surface feature point and the organ does not change. The flexible portion 22, the rigid portion 21, and the ultrasonic tomographic image marker Mu, the tip direction marker Md, the 6 o'clock direction marker Mt, and the insertion shape marker Ms on the three-dimensional guide image are more accurately matched anatomically. There is an effect.
The X-ray three-dimensional helical CT apparatus 15 and the three-dimensional MRI apparatus 16 are usually imaged in the supine position, and the body position is different from that in the ultrasonic endoscopy in the left supine position. The matching circuit 51 is configured and operates so as to create a second conversion expression for expressing the second mapping by combining the first mapping with the parallel movement by the vector P′P ”as the correction value.
Therefore, in the present embodiment, the organs in the subject 37 are displaced according to gravity at the time of ultrasonic endoscopy in the left lateral position as compared with the X-ray three-dimensional helical CT apparatus 15 and the three-dimensional MRI apparatus 16. However, a more accurate anatomical coincidence with the point of the subject 37, for example, the center O "of the ultrasonic tomographic image by the second mapping. Therefore, the three-dimensional guide image makes the ultrasonic tomographic image more accurate. Can guide.

  Further, in this embodiment, the three-dimensional guide image creating circuit A generates three-dimensional guide image data observed in the direction from the ventral side of the subject 37 on the subject head side on the right side and the subject foot side on the left side. Configure and act to create. The subject 37 is usually examined in the position of the left lateral position in the ultrasonic endoscopy.

  In this embodiment, since the 3D guide image is also displayed in the left-side position, the subject 37 and the 3D guide image can be easily compared, and the operator can easily understand the 3D guide image. Therefore, this embodiment can improve or appropriately support the operability during diagnosis and treatment by the operator.

  In addition, according to the present embodiment, since the 3D guide image creation circuit A and the 3D guide image creation circuit B create a 3D guide image in which the line of sight is set in different directions, an ultrasonic tomogram, a pancreas, etc. The positional relationship with the region of interest can be guided from a plurality of directions, and the ultrasonic tomographic image, the flexible portion 22 and the rigid portion 21 of the ultrasonic endoscope 2 with respect to the body cavity wall such as the digestive tract, etc. The orientation and shape of the can be guided from a plurality of directions, making it easy for the operator to understand.

(Modification)
In this embodiment, the ultrasonic endoscope 2 provided with the treatment instrument channel 46 and the body cavity contact probe 8 inserted through the treatment instrument channel 46 are provided. However, the configuration is not limited to this.
If the objective lens 25 is focused on the feature point in the body cavity via the optical observation window 24 and the rigid part 21 itself can be accurately brought into contact with the feature point in the body cavity without using the contact probe 8 in the body cavity, the rigid part 21 is obtained. The image position / orientation detection coil 31 fixed to the body cavity may be substituted for the body cavity detection coil 42 of the body cavity contact probe 8.
At this time, the image position / orientation detection coil 31 functions not only as an image position / orientation detection element but also as a body cavity detection element.
In the present embodiment, the electronic radial scanning ultrasonic endoscope 2 is used as the ultrasonic probe. However, as in the body cavity probe device disclosed in Japanese Patent Application Laid-Open No. 2004-113629, a mechanical device is used. It can be a scanning ultrasound endoscope, an electronic convex scanning ultrasound endoscope with a group of ultrasound transducers on one of the insertion axes, or a capsule ultrasound sonde. Is not limited. An ultrasonic probe without the optical observation window 24 may be used.

In this embodiment, the ultrasonic transducers are cut into strips in the rigid portion 21 of the ultrasonic endoscope 2 and arranged in an annular array around the insertion axis. It may be provided on the entire 360 ° circumference, or a part thereof may be missing. For example, the ultrasonic transducer array 29 may be formed in a portion extending over 270 ° or 180 °.
In this embodiment, the transmitting antenna 6 and the receiving coil are used as position detecting means and the position and orientation are detected and detected by the magnetic field. However, transmission and reception may be reversed. By detecting the position and orientation using a magnetic field, the position (orientation) detection means can be formed with a simple configuration, and the cost and size can be reduced.
However, the position (orientation) detection means is not limited to one using a magnetic field, and may be configured and operated to detect the position and orientation by acceleration or other means.
In the present embodiment, the origin O is set to a specific position on the transmission antenna 6. However, the origin O may be set to another place where the positional relationship with the transmission antenna 6 does not change.

In this embodiment, the image position / orientation detection coil 31 is fixedly provided on the rigid portion 21. However, if the position of the rigid portion 21 and the position is fixed (determined), it is not inside the rigid portion 21. Good.
In the present embodiment, each organ on the three-dimensional guide image data is displayed by being color-coded for each organ. However, the present invention is not limited to the color-coded (change display color) mode, and brightness, lightness, and saturation. Other modes may be adopted. For example, the luminance value may be changed for each organ.
Further, in this embodiment, the reference image data is configured and operated so as to use a plurality of two-dimensional CT images and two-dimensional MRI images captured by the X-ray three-dimensional helical CT apparatus 15 and the three-dimensional MRI apparatus 16. However, three-dimensional image data acquired in advance using another modality such as PET (Positoron Emission Tomography) may be used. Alternatively, three-dimensional image data acquired in advance by a so-called extracorporeal body cavity probe device that irradiates ultrasonic waves from outside the body may be used.
In this embodiment, the image data captured from the subject 37 by the X-ray three-dimensional helical CT apparatus 15 or the like is used and used as the reference image data. Data may be used.

Further, in this embodiment, a body surface detection coil 7 comprising four coils wound in one axial direction is provided, and a plurality of body surface feature points are provided on the subject body surface by tape, belt, band, etc. The position / orientation data of the body surface feature points are obtained at the same time, but instead of one coil, for example, the detection coil 42 in the body cavity, prior to the examination by the ultrasonic endoscope 2 After the subject 37 is placed in the left-side position, the configuration and operation are performed in which the tip of the body cavity contact probe 8 is sequentially brought into contact with a plurality of body surface feature points to sequentially obtain the position / orientation data of the body surface feature points. May be.
In this embodiment, the position / orientation calculation means calculates the position of the body surface detection coil 7 as the position / orientation data. However, the direction of the winding axis may be calculated instead of the position. Further, both the position and the direction of the winding axis may be calculated. The number of body surface detection coils 7 can be reduced by increasing the degree of freedom that the position / orientation calculation device 5 calculates for one body surface detection coil 7, and the body surface detection coil 7 is attached to the subject 37. It is possible to reduce the burden on the operator and the subject 37 during fixation or during the ultrasonic endoscopy.

Further, in this embodiment, the body surface feature points are described as the xiphoid process of the abdominal body surface, the upper left anterior iliac spine, the upper right anterior iliac spine, and the lumbar spine spinous process, and the body cavity feature point is described as the duodenal papilla. However, the present invention is not limited to this example, but may be a feature point in the chest body surface, a chest body cavity, or another example. In general, the accuracy of the orientation of the ultrasonic tomographic image marker Mu is better when the body surface feature point is a point related to the skeleton.
Further, in this embodiment, the rotation instruction signal from the control circuit 63 is rotated by 90 degrees by the operator's mouse 12 and keyboard 13, and the instruction contents are observed from the foot side. The three-dimensional guide image creation circuit B creates the three-dimensional guide image data observed in the direction from the subject's foot side. However, the present invention is not limited to this example. The three-dimensional guide image may be rotated in real time with respect to the input at an arbitrary angle.

Next, a second embodiment of the present invention will be described. The configuration of the present embodiment is the same as that of the first embodiment. However, only the operation of the three-dimensional guide image creation circuit B is different from the first embodiment.
Next, the operation of this embodiment will be described.
As described above, the present embodiment differs from the first embodiment only in the operation of the three-dimensional guide image creation circuit B.
In Example 1, as shown in FIG. 15, the three-dimensional guide image creation circuit B creates three-dimensional guide image data observed in the direction from the subject's foot side, and outputs it to the mixing circuit 61.
Then, along with the movement of the radial scanning surface accompanying manual operation of the flexible portion 22 and the rigid portion 21 by the operator, the ultrasonic tomographic image marker Mu, the tip direction marker Md, and the 6 o'clock direction marker Mt on the image index data. Then, the insertion shape marker Ms and the coil position marker Mc on the insertion shape data are moved or deformed on the three-dimensional human body image data.

In this embodiment, as shown in FIG. 22, the three-dimensional guide image creating circuit B uses the normal line of the ultrasonic tomographic image marker Mu as the observation line of sight, that is, the screen method of the display device 14 based on the position / orientation mapping data. A guide image is created in which the screen faces the screen so as to match the line, and the 6 o'clock direction marker Mt is set to face downward in the screen of the display device 14.
The three-dimensional guide image data in FIG. 22 includes the ultrasonic tomographic image marker Mu on the image index data and the tip direction along with the movement of the radial scanning plane accompanying the manual operation of the flexible portion 22 and the rigid portion 21 of the operator. The marker Md, the 6 o'clock direction marker Mt, the insertion shape marker Ms on the insertion shape data, and the coil position marker Mc are fixed on the screen of the display device 14, and the three-dimensional human body image data moves on the screen of the display device 14. I will do it.
In the three-dimensional guide image data of FIG. 22, the ultrasonic tomographic image marker Mu in the image index data is made translucent, the 6 o'clock direction marker Mt and the tip direction marker Md of the image index data, and the insertion shape of the insertion shape data The marker Ms and the coil position marker Mc are seen through.

For other organs, the ultrasonic tomographic image marker Mu is made opaque so that the portion on the back side of the ultrasonic tomographic image marker Mu cannot be seen.
Other operations are the same as those of the first embodiment.
This embodiment has the following effects.
According to the present embodiment, the three-dimensional guide image creation circuit B matches the normal line of the ultrasonic tomographic image marker Mu with the observation line of sight, that is, the screen normal line of the display device 14 based on the position / orientation mapping data. The 3D guide image and the display device are configured and operated so as to create a 3D guide image that faces the screen and is set so that the 6 o'clock direction marker Mt faces the lower direction of the screen of the display device 14. The direction coincides with the ultrasonic tomographic image displayed on the screen of 14 in real time. Therefore, it is easy for the surgeon to compare the two, and to interpret the ultrasonic tomogram anatomically.
Other effects are the same as those of the first embodiment.
(Modification)
As a modification of the present embodiment, the modification described in the first embodiment can be applied.

Next, a third embodiment of the present invention will be described.
The configuration of the present embodiment is the same as that of the second embodiment. This embodiment is different from the second embodiment only in the operation of the three-dimensional guide image creation circuit B.
Next, the operation of this embodiment will be described.

As described above, the operation of this embodiment differs from that of Embodiment 2 only in the operation of the three-dimensional guide image creation circuit B.
In the second embodiment, as shown in FIG. 22, the three-dimensional guide image creation circuit B makes the ultrasonic tomographic image marker Mu of the image index data translucent, and the 6 o'clock direction marker Mt of the image index data and the tip direction. The marker Md, the insertion shape marker Ms of the insertion shape data, and the coil position marker Mc are seen through, and the ultrasonic tomographic image marker Mu is made opaque to other organs, and the back side of the ultrasonic tomographic image marker Mu. The three-dimensional guide image data in which the portion was made invisible was created and output to the mixing circuit 61.

In this embodiment, as shown in FIG. 23, the three-dimensional guide image creation circuit B makes the ultrasonic tomographic image marker Mu in the image index data semi-transparent. In addition, the three-dimensional guide image creation circuit B not only includes the 6 o'clock direction marker Mt and the tip direction marker Md of the image index data, the insertion shape marker Ms and the coil position marker Mc of the insertion shape data, but also the superposition of other organs. The back side portion of the ultrasonic tomographic image marker Mu is also seen through, and the luminance of the portions on the front side and the back side of the ultrasonic tomographic image marker Mu is changed to create three-dimensional guide image data, and to the mixing circuit 61. Output.
In the case of the pancreas, the portion on the front side (front side) from the ultrasonic tomographic image marker Mu is created in dark green, and the portion on the back side is created in light green. In the case of a blood vessel, the portion on the front side (front side) from the ultrasonic tomographic image marker Mu is created in dark red, and the portion on the back side is created in light red.
In FIG. 23, each marker and each organ that are behind the ultrasonic tomographic image marker Mu and overlap the ultrasonic tomographic image marker Mu are indicated by broken lines.
Other operations are the same as those in the second embodiment.

This embodiment has the following effects.
According to the present embodiment, the three-dimensional guide image creation circuit B makes the ultrasonic tomographic image marker Mu in the image index data translucent, and inserts the 6 o'clock direction marker Mt and the tip direction marker Md of the image index data. In addition to the shape data insertion shape marker Ms and the coil position marker Mc, the back side portion of the ultrasonic tomographic image marker Mu of other organs can be seen through, and the front side and the back side of the ultrasonic tomographic image marker Mu, respectively. The configuration and the operation of creating the three-dimensional guide image data by changing the luminance of the portion in the above.
Therefore, it is easy for the surgeon to know how to move the flexible portion 22 and the rigid portion 21 further so that the region of interest such as the affected portion can be displayed on the ultrasonic tomographic image. It is easy to operate the second flexible portion 22 and the hard portion 21.

In particular, an organ that is soft and easy to move within the subject 37, such as the gallbladder, may appear on the ultrasonic tomographic image marker Mu, but not on the ultrasonic tomographic image. The three-dimensional guide image of the present embodiment is a mark that the surgeon moves the rigid portion 21 and the flexible portion 22 a little more, so that it can be drawn on the ultrasonic tomogram of the gallbladder. The sonic endoscope 2 is easy to operate with the rigid portion 21 and the flexible portion 22.
Other effects are the same as those of the second embodiment.
(Modification)
In this embodiment, the ultrasonic tomographic image marker Mu in the image index data is made translucent, the 6 o'clock direction marker Mt and the tip direction marker Md of the image index data, the insertion shape marker Ms of the insertion shape data, and the coil position marker. Although it was configured and operated so that not only Mc but also the back side portion of the ultrasonic tomographic image marker Mu of other organs could be seen through, as a modified example, the operator changed the transparency from the mouse 12 or the keyboard 13. You may make it change freely by selection input.
As another modification, the modification of the second embodiment can be applied.

Next, a fourth embodiment of the present invention will be described. This embodiment has the same configuration as that of the third embodiment. This embodiment differs from the third embodiment only in the operation of the three-dimensional guide image creation circuit B.
Next, the operation of this embodiment will be described.
As described above, the operation of the present embodiment is different from the third embodiment only in the operation of the three-dimensional guide image creation circuit B.
In the third embodiment, as shown in FIG. 23, the three-dimensional guide image creation circuit B makes the ultrasonic tomographic image marker Mu of the image index data translucent, and the 6 o'clock direction marker Mt and the tip direction of the image index data. In addition to the marker Md, the insertion shape marker Ms of the insertion shape data, and the coil position marker Mc, the back side portion of the ultrasonic tomographic image marker Mu of other organs can be seen through, respectively. Three-dimensional guide image data was generated by changing the luminance of the portions on the front side and the back side of Mu and output to the mixing circuit 61.

In the case of the pancreas, the portion on the front side (near side) from the ultrasonic tomographic image marker Mu was created in dark green, and the portion on the back side in light green. In the case of a blood vessel, the portion on the front side (near side) from the ultrasonic tomographic image marker Mu is created in dark red and the portion on the back side is created in light red.
In the present embodiment, as shown in FIG. 24, the three-dimensional guide image creation circuit B has a distal end side of the flexible portion 22 in two regions divided by the ultrasonic tomographic image marker Mu in the image index data, that is, The front side of the screen of the display device 14 is not displayed, and three-dimensional guide image data in which the luminance of the portion on the ultrasonic tomographic image marker Mu and the portion on the back side is changed is created and output to the mixing circuit 61.
In the case of the pancreas, the portion on the ultrasonic tomographic image marker Mu is created in dark green, and the portion on the back side is created in light green. In the case of a blood vessel, the portion on the ultrasonic tomographic image marker Mu is created in dark red, and the portion on the back side is created in light red.

Other operations are the same as those in the third embodiment.
This embodiment has the following effects.
In the present embodiment, the three-dimensional guide image creation circuit B has the two regions divided by the ultrasonic tomographic image marker Mu in the image index data, that is, on the distal end side of the flexible portion 22, that is, in front of the screen of the display device 14. The side is not displayed, and the three-dimensional guide image data in which the luminance of the portion on the ultrasonic tomographic image marker Mu and the portion on the back side is changed is created and operated.
For this reason, according to the present embodiment, an ultrasonic wave displayed in real time side by side on the screen of the three-dimensional guide image and the display device 14 without causing the organ on the near side to obstruct the operator's observation of the three-dimensional guide image. It is easier to compare with tomographic images and to facilitate anatomical interpretation of ultrasonic tomographic images.
Other effects are the same as those of the third embodiment.
(Modification)
The modification of the third embodiment can be applied to the modification of the present embodiment.

Next, a fifth embodiment of the present invention will be described. The configuration of the present embodiment is the same as that of the first embodiment. This embodiment differs from the first embodiment only in the operation of the three-dimensional guide image creation circuit B.
Next, the operation of this embodiment will be described.
As described above, the operation of this embodiment is different from the first embodiment only in the operation of the three-dimensional guide image creation circuit B.
In the first embodiment, as shown in FIG. 15, the three-dimensional guide image creation circuit B makes the ultrasonic tomographic image marker Mu in the image index data translucent, and the 6 o'clock direction marker Mt and the tip of the image index data. The direction marker Md, the insertion shape marker Ms of the insertion shape data, and the coil position marker Mc are made transparent, and the ultrasonic tomographic image marker Mu is made opaque for other organs. Three-dimensional guide image data in which the back side portion was not visible was created and output to the mixing circuit 61.

In this embodiment, as shown in FIG. 25, the three-dimensional guide image creation circuit B makes the ultrasonic tomographic image marker Mu in the image index data translucent, and the 6 o'clock direction marker Mt and the tip direction of the image index data. In addition to the marker Md, the insertion shape marker Ms of the insertion shape data, and the coil position marker Mc, the back side portion of the ultrasonic tomographic image marker Mu of other organs can be seen through, respectively. Three-dimensional guide image data is created by changing the luminance of the portions on the front side and the back side of Mu and output to the mixing circuit 61.
In the case of the pancreas, the portion on the tip direction marker Md side from the ultrasonic tomographic image marker Mu is created in dark green, and the portion on the opposite side is created in light green. In the case of a blood vessel, the portion on the tip direction marker Md side from the ultrasonic tomographic image marker Mu is created in dark red, and the portion on the opposite side is created in light red.
Other operations are the same as those of the first embodiment.
This embodiment has the following effects.

According to the present embodiment, the three-dimensional guide image creation circuit B makes the ultrasonic tomographic image marker Mu of the image index data translucent, and inserts the 6 o'clock direction marker Mt and the tip direction marker Md of the image index data. In addition to the shape data insertion shape marker Ms and the coil position marker Mc, the back side of the ultrasonic tomographic image marker Mu of other organs can be seen through, and the front side and the back side of the ultrasonic tomographic image marker Mu, respectively. The three-dimensional guide image data is created and operated by changing the luminance of the portion in the area.
Therefore, it is easy for the operator to know how to move the flexible portion 22 and the rigid portion 21 further and display the region of interest such as the affected area on the ultrasonic tomographic image, and operate the ultrasonic endoscope 2. Cheap.
In particular, an organ that is soft and easy to move within the subject 37, such as the gallbladder, may appear on the ultrasonic tomographic image marker Mu, but not on the ultrasonic tomographic image. The three-dimensional guide image of this embodiment becomes a mark that can be drawn on an ultrasonic tomographic image of the gallbladder if the surgeon moves the rigid portion 21 and the flexible portion 22 a little more. It is easy to operate the endoscope 2.
Other effects are the same as those of the first embodiment.

(Modification)
In this embodiment, the ultrasonic tomographic image marker Mu in the image index data is made translucent, the 6 o'clock direction marker Mt and the tip direction marker Md of the image index data, the insertion shape marker Ms of the insertion shape data, and the coil position marker. Although it was configured and operated so that not only Mc but also the back side of the ultrasonic tomographic image marker Mu of other organs could be seen through, as a modified example, the operator can freely set the transparency with the mouse 12 or the keyboard 13. You may make it change to.
As another modification, the modification of the first embodiment can be applied.

Next, a sixth embodiment of the present invention will be described. Only portions different from the first embodiment will be described.
In the image processing apparatus 11 of the first embodiment, the rigid portion 21 is provided with the image position / orientation detection coil 31 fixed in the very vicinity of the center of the ring of the ultrasonic transducer array 29.
In this embodiment, the rigid portion 21 is provided with an image position / orientation detection coil 31 fixed in the very vicinity of the CCD camera 26.
The direction in which the image position / orientation detection coil 31 is fixed is the same as in the first embodiment. The optical axis of the CCD camera 26 is oriented at a known angle with respect to V in the plane including V and V _{12 in} FIG.
FIG. 26 shows the image processing apparatus 11 of the present embodiment. In the image processing apparatus 11 according to the first embodiment, the mixing circuit 61 is connected to the ultrasonic observation apparatus 4. In the present embodiment, the mixing circuit 61 is connected to the optical observation device 3 instead of the ultrasonic observation device 4.

Other configurations are the same as those of the first embodiment.
Next, the operation of this embodiment will be described.
In the image processing apparatus 11 according to the first embodiment, the operator selects the X-ray three-dimensional helical CT apparatus 15 as an acquisition destination, and the communication circuit 54 captures a plurality of two-dimensional CT images as reference image data, which is shown in FIG. The operation in the case where such reference image data is stored in the reference image storage unit 55 has been described. For example, due to the action of an X-ray contrast medium, blood vessels such as the aorta and superior mesenteric vein have high brightness, organs containing many peripheral blood vessels such as pancreas have medium brightness, and duodenum etc. have low brightness. Was contrasted.
In this embodiment, the chest, particularly the trachea, bronchus, and tracheal branch, is imaged non-contrastally with the X-ray three-dimensional helical CT apparatus 15. An example in which the ultrasonic endoscope 2 is inserted into the trachea branch a at a location branched to b will be described.

The optical observation device 3 creates optical image data by setting the 12 o'clock direction (upward direction) of the optical image to a direction opposite to the direction in which V ₁₂ is projected onto the plane including V and V _{12 in} FIG.
The three-dimensional human body image creation circuit 57 extracts voxels (mainly trachea, bronchi, and tracheal branch walls) having high luminance values from the interpolation circuit 56 and colors them. Next, the three-dimensional human body image creation circuit 57 fills the extracted voxels into the voxel space of the synthesis memory 58a of the synthesis circuit 58 as three-dimensional human body image data.
At this time, the three-dimensional human body image creation circuit 57 fills the extracted voxel space address in the interpolation memory 56a and the voxel space address in the synthesis memory so as to be the same. Three-dimensional human body image data is extracted from the high-luminance trachea wall, bronchial wall, and tracheal branch wall, and each wall is colored with skin color. It is the observed three-dimensional data.

The image index creation circuit 52 includes each direction component (x0, y0, z0) of the position vector OO "of the position O" of the image position / orientation detection coil 31 on the orthogonal coordinate axis O-xyz, and the image position relative to the orthogonal coordinate axis O-xyz. Image index data is created from position / orientation mapping data having a total of six degrees of freedom with each of the Euler angle components (ψ, θ, φ) indicating the orientation of the orientation detection coil 31, and is output to the synthesis circuit 58.
The image index data is an orthogonal coordinate axis O′-x′y ′ obtained by combining an orange optical image field direction marker indicating the direction of the optical axis and a yellow-green optical image up direction marker indicating the 12 o'clock direction of the optical image. Image data on z ′.
Similarly to the first embodiment, the insertion shape creation circuit 53 includes a plurality of directional components (x0, y0, z0) of the position vector OO ″ of the position O ″ of the image position / orientation detection coil 31 and a plurality of orthogonal coordinate axes O-xyz. Insertion shape data is created from position / orientation mapping data of each of the position vectors of each of the insertion shape detection coils 32 and the direction components (xi, yi, zi), and is output to the synthesis circuit 58.

This is shown in FIG. The insertion shape data includes a string-like insertion shape marker Ms obtained by interpolating the positions of the image position / orientation detection coil 31 and the plurality of insertion shape detection coils 32 in order, and a coil position marker Mc indicating each coil position. Is the image data on the orthogonal coordinate axis O′-x′y′z ′.
The synthesis circuit 58 fills the image index data and the insertion shape data in the voxel space in the synthesis memory 58a. In this manner, the synthesis circuit 58 embeds the three-dimensional human body image data, the image index data, and the insertion shape data in the same synthesis memory 58a in the same voxel space, thereby combining a set of synthesis. It is synthesized as 3D data.
The rotation conversion circuit 59 reads the combined three-dimensional data and performs a rotation process on the combined three-dimensional data in accordance with the rotation instruction signal from the control circuit 63.

The three-dimensional guide image creation circuit A performs rendering processing such as hidden surface removal and shading on the synthesized three-dimensional data, and creates three-dimensional guide image data that can be output to the screen. The default direction of the three-dimensional guide image data is the direction from the ventral side of the human body.
Therefore, the three-dimensional guide image creation circuit A creates three-dimensional guide image data observed in the direction from the ventral side of the subject 37. The three-dimensional guide image creation circuit A outputs the three-dimensional guide image data observed from the subject's ventral side to the mixing circuit 61. This three-dimensional guide image data is shown in FIG. The right side of FIG. 27 is the subject head side, and the left side is the subject foot side.
In the three-dimensional guide image data of FIG. 27, the wall of the bronchi and the walls of the trachea branch a and the trachea branch b are made translucent, and the optical image field direction marker and the optical image up of the image index data are displayed. The direction marker, the insertion shape marker Ms of the insertion shape data, and the coil position marker Mc are made visible.

The three-dimensional guide image creation circuit B performs rendering processing such as hidden surface removal and shading on the combined three-dimensional data subjected to the rotation processing, and creates three-dimensional guide image data that can be output to the screen.
In this embodiment, as an example, the rotation instruction signal from the control circuit 63 rotates the three-dimensional guide image data by 90 degrees according to the input from the mouse 12 and the keyboard 13 by the operator, and the instruction content is observed from the foot side. It shall have been.
Accordingly, the 3D guide image creation circuit B creates 3D guide image data observed in the direction from the subject's foot side. The three-dimensional guide image creation circuit B outputs the three-dimensional guide image data observed from the subject foot side to the mixing circuit 61. This three-dimensional guide image data is shown in FIG. The right side of FIG. 28 is the subject right side, and the left side is the subject left side.

In the three-dimensional guide image data of FIG. 28, the wall of the bronchus and the walls of the trachea branch a and the trachea branch b are made translucent, and the optical image field direction marker and the optical image up of the image index data are displayed. The direction marker, the insertion shape marker Ms of the insertion shape data, and the coil position marker Mc are made visible.
The mixing circuit 61 includes optical image data from the optical observation device 3, three-dimensional guide image data obtained by observing the subject 37 from the three-dimensional guide image creation circuit A from the ventral side, and three-dimensional guide image creation circuit B. The mixed data for display is created by arranging the three-dimensional guide image data obtained by observing the subject 37 from the foot side.
The display circuit 62 converts this mixed data into an analog video signal.
The display device 14 displays the optical image, the three-dimensional guide image obtained by observing the subject 37 from the foot side, and the three-dimensional guide image observed from the abdomen side by side based on the analog video signal.

As shown in FIG. 29, the display device 14 displays the wall of the bronchus and the wall of the trachea branch expressed on the three-dimensional guide image in skin color.
In this embodiment, the optical image is processed as a real-time image.
As in the first embodiment, in this embodiment, two new three-dimensional guide images are created and displayed on the display screen of the display device 14 while being updated in real time together with new optical images. That is, as shown in FIG. 29, the optical image field direction marker of the image index data and the optical image up direction are interlocked with the movement of the optical axis accompanying the manual operation of the flexible portion 22 and the rigid portion 21 of the operator. The marker, the insertion shape marker Ms of the insertion shape data, and the coil position marker Mc move or deform on the three-dimensional human body image data.
Other operations are the same as those of the first embodiment.
This embodiment has the following effects.

According to the present embodiment, the bronchial wall and the walls of the trachea branch a and the trachea branch b beyond that are translucent, and the optical image field direction marker and the optical image up direction marker of the image index data Then, three-dimensional guide image data is created so that the insertion shape marker Ms and the coil position marker Mc of the insertion shape data can be seen, and the mixing circuit 61 and the display device 14 observe the optical image and the subject 37 from the ventral side. The three-dimensional guide image and the three-dimensional guide image observed from the foot side are arranged and displayed side by side.
Therefore, this embodiment prevents the ultrasonic endoscope 2 (or an endoscope as described in the following modification) from being erroneously inserted into the trachea branch b when inserted into the trachea branch a. be able to.
Other effects are the same as those of the first embodiment.

  Here, the case of insertion into the deep part of the bronchi has been described. However, in other cases, the optical image field direction marker of the image index data, the optical image up direction marker, the insertion shape data insertion shape marker Ms, and the coil position marker are used. Since the three-dimensional guide image data synthesized so that Mc can be seen is created, the surgeon can insert a body cavity probe into the body cavity and perform smooth diagnosis and treatment. Therefore, it is possible to realize an in-vivo probe device that allows an operator to easily perform diagnosis and treatment smoothly.

(Modification)
According to the present embodiment, as in the first embodiment, an electronic radial scanning type provided with an optical observation system (an optical observation window 24, an objective lens 25, a CCD camera 26, and an illumination light irradiation window not shown) as a body cavity probe. However, instead of the ultrasonic endoscope 2, an endoscope provided with an optical observation system may be used as a body cavity probe.
Other modifications can be applied to the modification in the first embodiment.
It should be noted that embodiments configured by partially combining the above-described embodiments and the like also belong to the present invention. Further, the block configuration of the image processing apparatus 11 shown in FIG. 4 and the like may be changed.

[Appendix]
1. An intracavity probe inserted into the body cavity of the subject;
An insertion shape creation means for creating an insertion shape of the probe in the body cavity;
3D image creation means for creating a 3D image of the human body from 3D data of the human body;
Synthesis means for synthesizing the insertion shape and the three-dimensional image;
In a body cavity probe device comprising:
The intra-body-cavity probe includes a rigid portion provided with fixed image signal acquisition means for acquiring a signal for creating an image in the subject on the insertion side into the body cavity, and a front side of the rigid portion. A flexible portion,
Image creating means for creating a real-time image in the subject from the signal obtained by the image signal obtaining means;
An image position / orientation detecting element whose position is fixed with respect to the hard part;
A plurality of insertion shape detecting elements provided along the flexible portion;
An analyte detection element capable of contacting the analyte;
6 degrees of freedom between the position and orientation of the image position / orientation detecting element, each position of the plurality of insertion shape detecting elements, and the position or orientation of the subject detecting element are detected as detection values. Detecting means for outputting;
Image index creating means for creating an image index indicating the position and orientation of the real-time image in the subject created by the image creating means;
Comprising
The synthesizing unit synthesizes the insertion shape, the image index, and the three-dimensional image based on the detection value output from the detection unit, and the position and orientation of the flexible part and the real-time image with respect to the subject An intra-body-cavity probe apparatus characterized by creating a three-dimensional guide image for guiding

(Effects of Appendix 1)
The insertion shape of the body cavity probe and the direction of the real-time image can be detected with less invasion, and a guide image including both can be created.
Further, the positional relationship between the real-time image and the region of interest such as the pancreas can be guided, and the real-time image, the probe in the body cavity, the flexible part 22, the rigid part 21, and the body cavity wall such as the digestive tract and trachea Can be guided in what orientation and shape.
In addition, since a subject detection element that can contact the subject is provided, even if a change in the posture of the subject occurs during the examination with the body cavity probe, the positional relationship between the body surface feature point and the organ does not change. In addition, the real-time image or the flexible part or the rigid part and the image index or the insertion shape on the three-dimensional image of the human body can be more accurately matched anatomically.

(Effects of Appendix 8)
The corrected X-ray three-dimensional helical CT apparatus and the three-dimensional MRI apparatus are normally set to image in the supine position, and are different in posture from the endoscopic examination in the left supine position. When creating a guide image based on the detection value of the position or orientation of the body surface detection element in the guide image, the guide image was corrected using the detection value of the position of the body cavity detection element. Compared to X-ray 3D helical CT and 3D MRI, the mapping of points in the subject is more anatomically accurate even if the organ is displaced according to gravity during endoscopic examination in the left lateral position Make a coincidence. Therefore, the guide image can more accurately guide the real-time image.
In addition, since a subject detection element that can contact the subject is provided, even if a change in the posture of the subject occurs during the examination with the body cavity probe, the positional relationship between the body surface feature point and the organ does not change. In addition, the real-time image or the flexible part or the rigid part and the image index or the insertion shape on the three-dimensional image of the human body can be more accurately matched anatomically.

9. 9. The intra-body-cavity probe apparatus according to appendix 8, wherein the correction process by the correction unit is a parallel movement process in the three-dimensional data.
10. Image index creating means for creating an image index indicating the position and orientation of the real-time image in the subject created by the image creating means;
The guide image creating means is a guide that synthesizes the image index based on the position and orientation of the image position / orientation detecting element output from the detecting means and the detected value of the position or orientation of the body surface detecting element. Create an image,
The correction processing by the correction means creates the guide image by synthesizing the image index at the position translated in the three-dimensional data or the guide image based on the detection value of the position of the body cavity detecting element. 9. The body cavity probe device according to appendix 8, wherein

11. The body cavity probe has a tubular channel;
Contact means incorporating and fixing the body cavity detection element,
9. The intra-body-cavity probe apparatus according to appendix 8, wherein the contact means is inserted into the channel and contacts a predetermined position in the body cavity of the subject.

12 The image position / orientation detecting element also serves as the body cavity detecting element,
9. The intra-body-cavity probe apparatus according to appendix 8, wherein the correction unit performs correction processing on the guide image based on a detection value of the position or orientation of the image position / orientation detection element.

13. The guide image creating means includes an extracting means for extracting an organ or a blood vessel from three-dimensional data imaged from the subject;
The guide image creating means creates a three-dimensional image representing the shape and arrangement of the organ or the vessel from the organ or the vessel of the subject extracted by the extracting means, and the three-dimensional image 9. The body cavity probe device according to appendix 8, wherein the guide image is created based on the guide image.

14 The image signal acquisition means is an image sensor that images the inside of the subject and outputs a video signal,
9. The intra-body-cavity probe apparatus according to appendix 8, wherein the image creating unit creates an optical image as the real-time image from the video signal.

15. The image signal acquisition means is an ultrasonic transducer that transmits and receives ultrasonic waves in the subject to output echo signals,
9. The intra-body-cavity probe apparatus according to appendix 8, wherein the image creating unit creates an ultrasonic tomographic image as the real-time image from the echo signal.

16. The image position and orientation detection element and the subject detection element are a magnetic field generator or a magnetic field detector,
The intra-body-cavity probe apparatus according to appendix 8, wherein the detection means performs the detection using a magnetic field.

17. An intracorporeal probe provided with an image signal acquisition means that is inserted into the body cavity of the subject and acquires a signal for creating an image in the subject at the distal end on the insertion side into the body cavity;
3D image creation means for creating a 3D image of the human body from 3D data of the human body;
Guide image creating means for creating a guide image for guiding the position or orientation of the real-time image in the subject with respect to the subject based on the three-dimensional image;
In a body cavity probe device comprising:
Image creating means for creating a real-time image in the subject from the signal obtained by the image signal obtaining means;
An image position / orientation detection element having a fixed positional relationship with the image signal acquisition means;
An analyte detection element capable of contacting the analyte;
Detecting means for detecting the position and orientation of the image position / orientation detecting element and the position or orientation of the object detecting element and outputting as a detection value;
Image index creating means for creating an image index indicating the position and orientation of the real-time image in the subject created by the image creating means;
Display means for displaying the real-time image in the subject created by the image creation means and the guide image created by the guide image creation means in a comparable manner;
Comprising
The display means displays the guide image such that the head side of the three-dimensional image of the human body is on the right, the foot side is on the left, and the ventral side is in a normal direction in front of the screen of the display means. Body cavity probe device.

(Effect of Supplementary Note 17)
The subject is usually examined in a left-right position in endoscopy. Since the guide image is also displayed in the left lateral position, it is easy to compare the subject and the guide image, and the surgeon can easily understand the guide image.
In addition, since a subject detection element that can contact the subject is provided, even if a change in the posture of the subject occurs during the examination with the body cavity probe, the positional relationship between the body surface feature point and the organ does not change. In addition, the real-time image or the flexible part or the rigid part and the image index or the insertion shape on the three-dimensional image of the human body can be more accurately matched anatomically.

18. The guide image creating means includes an extracting means for extracting an organ or a blood vessel from three-dimensional data imaged from the subject;
The guide image creating unit creates a three-dimensional image representing the shape and arrangement of the organ or the vessel from the organ or the vessel of the subject extracted by the extracting unit, and the three-dimensional image 18. The body cavity probe device according to appendix 17, wherein the guide image is created based on the guide image.

19. The image signal acquisition means is an image sensor that images the inside of the subject and outputs a video signal,
18. The body cavity probe device according to appendix 17, wherein the image creating means creates an optical image as the real-time image from the video signal.

20. The image signal acquisition means is an ultrasonic transducer that transmits and receives ultrasonic waves in the subject to output echo signals,
The in-vivo probe device according to appendix 17, wherein the image creating means creates an ultrasonic tomographic image as the real-time image from the echo signal.

21. The image position and orientation detection element and the subject detection element are a magnetic field generator or a magnetic field detector,
18. The body cavity probe device according to appendix 17, wherein the detection means performs the detection using a magnetic field.

22. An intracorporeal probe provided with an image signal acquisition means that is inserted into the body cavity of the subject and acquires a signal for creating an image in the subject at the distal end on the insertion side into the body cavity;
3D image creation means for creating a 3D image of the human body from 3D data of the human body;
Guide image creating means for creating a guide image for guiding the position or orientation of the real-time image in the subject with respect to the subject based on the three-dimensional image;
In a body cavity probe device comprising:
Image creating means for creating a real-time image in the subject from the signal obtained by the image signal obtaining means;
An image position / orientation detection element having a fixed positional relationship with the image signal acquisition means;
An analyte detection element capable of contacting the analyte;
Detecting means for detecting the position and orientation of the image position / orientation detecting element and the position or orientation of the object detecting element and outputting as a detection value;
Image index creating means for creating an image index indicating the position and orientation of the real-time image in the subject created by the image creating means;
Comprising
The guide image creating means synthesizes the three-dimensional image and the image index based on the detection value output from the detection means, and creates a guide image in which the line of sight is set in at least two different directions. A body cavity probe device.

(Effects of Supplementary Note 22)
Since the guide images were created with the line of sight set in two different directions, the positional relationship between the real-time image and the region of interest such as the pancreas can be guided from multiple directions, and against the body cavity wall such as the digestive tract The orientation and shape of the real-time image and the flexible part and the rigid part of the body cavity probe can be guided from a plurality of directions, which is easy for the operator to understand.
In addition, since a subject detection element that can contact the subject is provided, even if a change in the posture of the subject occurs during the examination with the body cavity probe, the positional relationship between the body surface feature point and the organ does not change. In addition, the real-time image or the flexible part or the rigid part and the image index or the insertion shape on the three-dimensional image of the human body can be more accurately matched anatomically.

23. The guide image creating means creates a guide image in which the line of sight is set in at least two different directions;
And at least one of the two guide images is set in the direction from the ventral or dorsal side of the subject,
The in-vivo probe device according to appendix 22, wherein one line of sight of at least the two guide images is set in a direction from the head side or the foot side of the subject.

24. The guide image creating means creates a guide image in which the line of sight is set in at least two different directions;
The body cavity probe device according to appendix 22, further comprising guide image rotation means for changing a line-of-sight direction of at least one of the at least two guide images.

25. The guide image creating means includes an extracting means for extracting an organ or a blood vessel from three-dimensional data imaged from the subject;
The guide image creation means creates a three-dimensional image representing the shape and arrangement of the organ or the vessel from the organ or the vessel of the subject extracted by the extraction means, and the three-dimensional image The intracavity probe apparatus according to any one of appendices 22 to 24, wherein the guide image is created based on the appendix.

26. The image signal acquisition means is an image sensor that images the inside of the subject and outputs a video signal,
25. The intra-body-cavity probe apparatus according to any one of appendices 22 to 24, wherein the image creation means creates an optical image as the real-time image from the video signal.

27. The image signal acquisition means is an ultrasonic transducer that transmits and receives ultrasonic waves in the subject to output echo signals,
25. The body cavity probe device according to any one of appendices 22 to 24, wherein the image creating means creates an ultrasonic tomographic image as the real-time image from the echo signal.

28. The image position and orientation detection element and the subject detection element are a magnetic field generator or a magnetic field detector,
25. The body cavity probe device according to any one of appendices 22 to 24, wherein the detection means performs the detection using a magnetic field.

29. An intracorporeal probe provided with an image signal acquisition means that is inserted into the body cavity of the subject and acquires a signal for creating an image in the subject at the distal end on the insertion side into the body cavity;
3D image creation means for creating a 3D image of the human body from 3D data of the human body;
Guide image creating means for creating a guide image for guiding the position or orientation of the real-time image in the subject with respect to the subject based on the three-dimensional image;
In a body cavity probe device comprising:
Image creating means for creating a real-time image in the subject from the signal obtained by the image signal obtaining means;
An image position / orientation detection element having a fixed positional relationship with the image signal acquisition means;
An analyte detection element capable of contacting the analyte;
Detecting means for detecting the position and orientation of the image position / orientation detecting element and the position or orientation of the object detecting element and outputting as a detection value;
Image index creating means for creating an image index indicating the position and orientation of the real-time image in the subject created by the image creating means;
Comprising
The guide image creating unit is configured to synthesize the three-dimensional image and the image index based on the detection value output from the detection unit, and set the line of sight to a direction that coincides with the normal line of the image index. An intra-body-cavity probe apparatus characterized by creating one guide image.
(Effects of Supplementary Note 29)
Since the directions of the first guide image and the real-time image coincide with each other, it is easy for the operator to compare the two images and to interpret the real-time image anatomically.
In addition, since a subject detection element that can contact the subject is provided, even if a change in the posture of the subject occurs during the examination with the body cavity probe, the positional relationship between the body surface feature point and the organ does not change. In addition, the real-time image or the flexible part or the rigid part and the image index or the insertion shape on the three-dimensional image of the human body can be more accurately matched anatomically.

30. The guide image creating means, in addition to the first guide image, the line of sight from the abdominal side or the back side of the subject, or
30. The body cavity probe device according to appendix 29, wherein the second guide image set in a direction from the head side or the foot side of the subject is created.

31. Display means for displaying the real-time image in the subject created by the image creation means and the first guide image created by the guide image creation means in a comparable manner;
The display means can match the first guide image with the real-time image in the subject by matching the normal line of the screen with the normal line of the image index synthesized with the first guide image. The in-body-cavity probe device according to appendix 29, characterized by:

32. Display means for displaying the real-time image in the subject created by the image creation means and the first guide image created by the guide image creation means in a comparable manner;
Item 32. The supplementary note 31, wherein the display unit displays the real-time image in the subject and the first guide image created by the guide image creation unit so that the directions of the images coincide with each other. Body cavity probe device.

33. 33. The intracavity according to appendix 31 or 32, wherein the guide image creating means creates the guide image while hiding one of the two regions in which the three-dimensional image is divided by the image index. Probe device.

34. The guide image creating means creates the guide image by changing the luminance, brightness, saturation, or display color of a portion on the image index of the three-dimensional image from the other display portions. The in-body-cavity probe device according to Supplementary Note 31 or 32.

35. The guide image creating means, in addition to the first guide image, the line of sight from the abdominal side or the back side of the subject, or
Create a second guide image set in the direction from the head or foot of the subject,
33. The body cavity probe device according to appendix 31 or 32, wherein the display means displays the first guide image and the second guide image simultaneously, sequentially or by switching.

36. The guide image creating means includes an extracting means for extracting an organ or a blood vessel from three-dimensional data imaged from the subject;
The guide image creation means creates a three-dimensional image representing the shape and arrangement of the organ or the vessel from the organ or the vessel of the subject extracted by the extraction means, and the three-dimensional image 36. The body cavity probe device according to any one of appendices 29 to 35, wherein the guide image is created based on

37. The image signal acquisition means is an image sensor that images the inside of the subject and outputs a video signal,
36. The intra-body-cavity probe apparatus according to any one of appendices 29 to 35, wherein the image creation means creates an optical image as the real-time image from the video signal.

38. The image signal acquisition means is an ultrasonic transducer that transmits and receives ultrasonic waves in the subject to output echo signals,
36. The in-vivo probe device according to any one of appendices 29 to 35, wherein the image creating means creates an ultrasonic tomographic image as the real-time image from the echo signal.

39. The image position and orientation detection element and the subject detection element are a magnetic field generator or a magnetic field detector,
36. The body cavity probe device according to any one of appendices 29 to 35, wherein the detection means performs the detection using a magnetic field.

40. An intracorporeal probe provided with an image signal acquisition means that is inserted into the body cavity of the subject and acquires a signal for creating an image in the subject at the distal end on the insertion side into the body cavity;
3D image creation means for creating a 3D image of the human body from 3D data of the human body;
Guide image creating means for creating a guide image for guiding the position or orientation of the real-time image in the subject with respect to the subject based on the three-dimensional image;
In a body cavity probe device comprising:
Image creating means for creating a real-time image in the subject from the signal obtained by the image signal obtaining means;
An image position / orientation detection element having a fixed positional relationship with the image signal acquisition means;
An analyte detection element capable of contacting the analyte;
Detecting means for detecting the position and orientation of the image position / orientation detecting element and the position or orientation of the object detecting element and outputting as a detection value;
Image index creating means for creating an image index indicating the position and orientation of the real-time image in the subject created by the image creating means;
Comprising
The guide image creating means combines the three-dimensional image and the image index based on the detection value output from the detection means, and displays two regions in which the three-dimensional image is divided by the image index. An intra-body-cavity probe apparatus, wherein the guide image is created by changing

(Effects of Supplementary Note 40)
In order to create a guide image by changing the display mode, the surgeon can easily understand how the region of interest such as the affected part can be displayed on the real-time image by further moving the flexible part and the hard part. It is easy to operate the flexible part and the hard part.
In particular, organs such as the gallbladder that are soft and easy to move inside the subject may appear on the image index, but may not appear on the real-time image. And the flexible part is a mark that the gallbladder can be drawn on the real-time image, so that the operator can easily operate the rigid part and the flexible part of the body cavity probe.
In addition, since a subject detection element that can contact the subject is provided, even if a change in the posture of the subject occurs during the examination with the body cavity probe, the positional relationship between the body surface feature point and the organ does not change. In addition, the real-time image or the flexible part or the rigid part and the image index or the insertion shape on the three-dimensional image of the human body can be more accurately matched anatomically.

41. 41. The intra-body-cavity probe device according to appendix 40, wherein the image index creating means creates the image index so that the image index is transmissive.
42. The guide image creating means creates the guide image by changing the brightness, brightness, saturation, or display color of the two regions in which the three-dimensional image is divided by the image index. 40. The body cavity probe device according to 40 or 41. 43. The guide image creating means includes an extracting means for extracting an organ or a blood vessel from three-dimensional data imaged from the subject;
The guide image creating unit creates a three-dimensional image representing the shape and arrangement of the organ or the vessel from the organ or the vessel of the subject extracted by the extracting unit, and the three-dimensional image 43. The intra-body-cavity probe apparatus according to any one of appendices 40 to 42, wherein the guide image is created based on

44. The image signal acquisition means is an image sensor that images the inside of the subject and outputs a video signal,
43. The intra-body-cavity probe apparatus according to any one of appendices 40 to 42, wherein the image creation means creates an optical image as the real-time image from the video signal.

45. The image signal acquisition means is an ultrasonic transducer that transmits and receives ultrasonic waves in the subject to output echo signals,
43. The body cavity probe device according to any one of appendices 40 to 42, wherein the image creating means creates an ultrasonic tomographic image as the real-time image from the echo signal.

46. The image position and orientation detection element and the subject detection element are a magnetic field generator or a magnetic field detector,
43. The body cavity probe device according to any one of appendices 40 to 42, wherein the detection means performs the detection using a magnetic field.

47. An intracorporeal probe provided with an image signal acquisition means that is inserted into the body cavity of the subject and acquires a signal for creating an image in the subject at the distal end on the insertion side into the body cavity;
3D image creation means for creating a 3D image of the human body from 3D data of the human body;
Guide image creating means for creating a guide image for guiding the position or orientation of the real-time image in the subject with respect to the subject based on the three-dimensional image;
In a body cavity probe device comprising:
Image creating means for creating a real-time image in the subject from the signal obtained by the image signal obtaining means;
An image position / orientation detection element having a fixed positional relationship with the image signal acquisition means;
An analyte detection element capable of contacting the analyte;
Detecting means for detecting the position and orientation of the image position / orientation detecting element and the position or orientation of the object detecting element and outputting as a detection value;
Image index creating means for creating an image index indicating the position and orientation of the real-time image in the subject created by the image creating means;
Comprising
The guide image creating means synthesizes the three-dimensional image and the image index based on the detection value output from the detection means, and selects one of the two regions into which the three-dimensional image is divided by the image index. An intra-body-cavity probe apparatus, wherein the guide image is created in a non-display state.
(Effect of Supplementary Note 47)
Since the guide image was created by hiding one of the two areas divided by the image index, the contrast between the guide image and the real-time image can be further increased without causing the organ to obstruct the operator's observation of the guide image. It's easy to do.
In addition, since a subject detection element that can contact the subject is provided, even if a change in the posture of the subject occurs during the examination with the body cavity probe, the positional relationship between the body surface feature point and the organ does not change. In addition, the real-time image or the flexible part or the rigid part and the image index or the insertion shape on the three-dimensional image of the human body can be more accurately matched anatomically.

48. The guide image creating means creates the guide image by changing the luminance, brightness, saturation, or display color of a portion on the image index of the three-dimensional image from the other display portions. 48. The body cavity probe device according to appendix 47.
49. The guide image creating means includes an extracting means for extracting an organ or a blood vessel from three-dimensional data imaged from the subject;
The guide image creation means creates a three-dimensional image representing the shape and arrangement of the organ or the vessel from the organ or the vessel of the subject extracted by the extraction means, and the three-dimensional image 48. The body cavity probe device according to appendix 47, wherein the guide image is created based on

50. The image signal acquisition means is an image sensor that images the inside of the subject and outputs a video signal,
48. The intra-body-cavity probe apparatus according to appendix 47, wherein the image creating means creates an optical image as the real-time image from the video signal.
51. The image signal acquisition means is an ultrasonic transducer that transmits and receives ultrasonic waves in the subject to output echo signals,
48. The intra-body-cavity probe device according to appendix 47, wherein the image creating means creates an ultrasonic tomographic image as the real-time image from the echo signal.

52. The image position and orientation detection element and the subject detection element are a magnetic field generator or a magnetic field detector,
48. The body cavity probe device according to appendix 47, wherein the detection means performs the detection using a magnetic field.

  When displaying an image acquired by an image signal acquisition means provided at a hard part at the tip of an intracavity probe such as an ultrasonic endoscope that is flexibly inserted into the body cavity as a real-time image, the position and orientation of the real-time image Is displayed together with the insertion shape, making it easier for the operator to perform diagnosis and treatment.

FIG. 1 is an overall configuration diagram of a body cavity probe device according to a first embodiment of the present invention. FIG. 2 is a diagram schematically showing an example of use of a body surface detection coil. FIG. 3 is a side view showing the body cavity contact probe. FIG. 4 is a block diagram showing the configuration of the image processing apparatus. FIG. 5 is an explanatory diagram showing reference image data stored in a reference image storage unit. FIG. 6 is an explanatory diagram showing a voxel space. FIG. 7 is a diagram showing an orthogonal base in which an origin is set on a transmission antenna in order to represent position / orientation data. FIG. 8 is an explanatory diagram showing a state in which the center of the ultrasonic tomogram on the subject side is mapped to the voxel space. FIG. 9 is an explanatory diagram showing a state in which feature points in the body cavity on the subject side are mapped to the voxel space. FIG. 10 is an explanatory diagram showing how image index data is created by the image index creating circuit. FIG. 11 is an explanatory diagram showing a state in which insertion shape data created by the insertion shape creation circuit is created. FIG. 12 is an explanatory diagram showing three-dimensional human body image data. FIG. 13 is an explanatory diagram showing a state in which the image index data and the insertion shape data are embedded in the voxel space in the synthesis memory by the synthesis circuit. FIG. 14 is an explanatory diagram showing three-dimensional guide image data when observed from the ventral side of the subject. FIG. 15 is an explanatory view showing three-dimensional guide image data when observed from the foot side of the subject. FIG. 16 is a diagram showing a three-dimensional guide image and an ultrasonic tomographic image displayed on the display device. FIG. 17 is a flowchart showing the overall processing contents of this embodiment. 18 is a flowchart showing specific processing contents of body surface feature points and body cavity feature point designation processing on the reference image in FIG. FIG. 19 is a flowchart showing specific processing contents of the correction value calculation processing in FIG. FIG. 20 is an explanatory diagram of the processing in FIG. FIG. 21 is a flowchart showing specific processing contents of the ultrasonic tomographic image / three-dimensional guide image creation / display processing in FIG. FIG. 22 is an explanatory diagram showing three-dimensional guide image data according to the second embodiment of the present invention. FIG. 23 is an explanatory diagram showing three-dimensional guide image data according to the third embodiment of the present invention. FIG. 24 is an explanatory diagram showing three-dimensional guide image data according to the fourth embodiment of the present invention. FIG. 25 is an explanatory diagram showing three-dimensional guide image data according to the fifth embodiment of the present invention. FIG. 26 is a block diagram showing a configuration of an image processing apparatus in Embodiment 6 of the present invention. FIG. 27 is an explanatory diagram showing three-dimensional guide image data generated by the three-dimensional guide image creating circuit A. FIG. 28 is an explanatory diagram showing three-dimensional guide image data generated by the three-dimensional guide image creating circuit B. FIG. 29 is a diagram showing a three-dimensional guide image and an optical image displayed on the display device.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Body cavity probe apparatus 2 ... Ultrasound endoscope 3 ... Optical observation apparatus 4 ... Ultrasound observation apparatus 5 ... Position orientation calculation apparatus 6 ... Transmitting antenna 7 ... Body surface detection coil 8 ... Body cavity contact probe 9 ... A / D conversion unit section 11 ... image processing apparatus 14 ... display apparatus 15 ... X-ray three-dimensional helical CT apparatus 21 ... rigid section 22 ... flexible section 26 ... CCD camera 29 ... ultrasonic transducer array 31 ... for image position and orientation detection Coil 32 ... Coil for insertion shape detection 42 ... Coil for body cavity detection 51 ... Matching circuit 52 ... Image index creation circuit 53 ... Insertion shape creation circuit 55 ... Reference image storage unit 56 ... Interpolation circuit 57 ... Three-dimensional human body image creation circuit 58 ... Composition circuit 59 ... Rotation conversion circuit 60 ... 3D guide image creation circuits A and B
61 ... Mixing circuit 62 ... Display circuit 65 ... Body cavity feature point designation key 66 ... Scanning control key

Claims (10)

An intracorporeal probe provided with an image signal acquisition means that is inserted into the body cavity of the subject and acquires a signal for creating an image in the subject at the distal end on the insertion side into the body cavity;
Image creating means for creating a real-time image in the subject from the signal obtained by the image signal obtaining means;
Guide image creation means for creating a guide image for guiding the position or orientation of the real-time image in the subject relative to the subject from three-dimensional data of the human body;
In a body cavity probe device comprising:
An image position orientation detecting element whose position is fixed with respect to the image signal acquisition means;
A subject detection element including a body cavity detection element capable of contacting the subject body cavity;
Detecting means for detecting the position and orientation of the image position / orientation detecting element and the position of the in-body cavity detecting element, and outputting the detected value;
When the guide image creation means creates the guide image based on the position and orientation detection value of the image position / orientation detection element output from the detection means, the detection value of the position of the body cavity detection element is used. Correction means for performing correction processing on the guide image;
A body cavity probe device comprising: The subject detection element further includes a body surface detection element capable of contacting the subject body surface;
The detection means detects the position and orientation of the image position / orientation detection element, the position or orientation of the body surface detection element, and the position of the intra-body-cavity detection element, and outputs a detection value.
The guide image creating means creates the guide image based on the position and orientation of the image position / orientation detecting element output from the detecting means and the detected value of the position or orientation of the body surface detecting element;
The body cavity probe device according to claim 1, wherein   The intra-body-cavity probe apparatus according to claim 1, wherein the correction process by the correction unit is a parallel movement process in the three-dimensional data. Image index creating means for creating an image index indicating the position and orientation of the real-time image in the subject created by the image creating means;
The guide image creating means is a guide that synthesizes the image index based on the position and orientation of the image position / orientation detecting element output from the detecting means and the detected value of the position or orientation of the body surface detecting element. Create an image,
The correction processing by the correction means creates the guide image by synthesizing the image index at a position translated in the three-dimensional data or the guide image based on the detection value of the position of the body cavity detecting element. The intracavity probe device according to claim 1, wherein the processing is performed. The body cavity probe has a tubular channel;
Contact means incorporating and fixing the body cavity detection element,
2. The intra-body-cavity probe apparatus according to claim 1, wherein the contact unit is inserted into the channel and contacts a predetermined position in the body cavity of the subject. The image position / orientation detecting element also serves as the body cavity detecting element,
The intra-body-cavity probe apparatus according to claim 1, wherein the correction unit performs a correction process on the guide image based on a detection value of the position or orientation of the image position / orientation detection element. The guide image creating means includes an extracting means for extracting an organ or a blood vessel from three-dimensional data imaged from the subject;
The guide image creating unit creates a three-dimensional image representing the shape and arrangement of the organ or the vessel from the organ or the vessel of the subject extracted by the extracting unit, and the three-dimensional image The intracavity probe apparatus according to claim 1, wherein the guide image is created based on the information. The image signal acquisition means is an image sensor that images the inside of the subject and outputs a video signal,
The intra-body-cavity probe apparatus according to claim 1, wherein the image creating unit creates an optical image as the real-time image from the video signal. The image signal acquisition means is an ultrasonic transducer that transmits and receives ultrasonic waves in the subject to output echo signals,
The intra-body probe apparatus according to claim 1, wherein the image creating unit creates an ultrasonic tomographic image as the real-time image from the echo signal. The image position and orientation detection element and the subject detection element are a magnetic field generator or a magnetic field detector,
The intra-body-cavity probe apparatus according to claim 1, wherein the detection unit performs the detection using a magnetic field.