JP2003265408A - Endoscope guide device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To judge the position of the distal end of an endoscope simply and with high accuracy. <P>SOLUTION: In order to guide an endoscope, a virtual endoscope image of a subject to be tested and information on the position and attitude of the distal end of the endoscope are stored in a database. When the actual soft endoscope image of the subject to be tested is obtained, the actual endoscope image is compared with the virtual endoscope image in the database to determina the virtual endoscope image having the highest similarity to the endscope image. According to the information on the determined virtual endoscope image, the position and attitude of the distal end of the soft endoscope corresponding to the actual endoscope image is determined. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to navigation of a flexible endoscope that bends freely.

[0002]

2. Description of the Related Art During an operation such as neurosurgery, it is possible to grasp a positional relationship between an operation site of a patient and a tomographic image (three-dimensional image) such as computer-coupled tomography (CT) and magnetic resonance imaging (MRI). As described above, a navigation device that supports an operator is used. As a result, the operator can recognize the relative positional relationship between the operation position and the medical image captured at the time of diagnosis.

A mechanical method (for example, Kosugi et al., An Articulated Neurosurgical Na) is mainly used for three-dimensional position measurement of an operation part using this navigation device.
vigation System Using MRI and CT Images (IEEE Tran
sactions of Biomedical Engineering, Vol. 35, No.
2, February 1988), and U.S. Pat.No. 5050608 (E. Wat.
anabe, Y. Kosugi and S. Manaka: System for indicat
ing a position to beoperated in a patient's body)
And a stereo vision method (for example, Japanese Patent No. 3152810;
Kosugi, Watanabe, Suzuki, Kawakami, Nakagawa: optical 3D position detection device). In the mechanical method, the tip position coordinates are obtained by a multi-joint arm navigation device having a rotary encoder. Further, the stereo vision method is for recognizing a bright spot image set on a probe by a plurality of CCD cameras and calculating a three-dimensional position of the probe based on the principle of triangulation.

[0004]

However, when attempting to measure the position of the endoscope tip with these devices, in the mechanical system, the positions of the arm and the endoscope tip are mechanically kept rigid. It is necessary to connect in a slanted shape, and such a usage condition is not satisfied with a fiber-type flexible endoscope that can be freely changed in shape. In addition, in the stereo vision system, even if the light emitting unit is attached to the tip of the endoscope placed inside the body, the opaque living tissue exists between the CCD camera and the outside of the body. Cannot be determined.

In order to measure the position of the endoscope distal end in a surgical device using a flexible endoscope, 1) the problem of how to generate and register an image corresponding to the three-dimensional position in the body 2) The problem of how to match the registered image with the image observed from the endoscope. 3) A human interface that gives the operator a proper direction in which the endoscope should move in the surgical environment. There is a problem of how to configure. In addition, 4) it is necessary to develop a calculation algorithm and an accessory device for performing the position measurement of the endoscope distal end in a simple and accurate manner in real time during surgery.

An object of the present invention is to make it possible to determine the position of the distal end portion of an endoscope easily and with high accuracy in a surgery or the like using a flexible endoscope that bends freely.

[0007]

An endoscopic guidance device according to the present invention comprises a storage means for storing a database of a virtual endoscopic image of an examination object and information on the position and orientation of the tip of the endoscope, and an examination. Input means for acquiring a real flexible endoscopic image of the object, and comparing the real endoscopic image with the virtual endoscopic image in the database, the virtual endoscopic image having the highest degree of similarity with the endoscopic image. The comparison means determines the mirror image, and the determination means determines the position and orientation of the flexible endoscope tip portion corresponding to the real endoscopic image from the information of the determined virtual endoscopic image. Here, the real flexible endoscope image is an image observed by a real flexible endoscope, and the real endoscope image is an image observed by a real endoscope.

In the endoscope guiding apparatus, for example, the virtual endoscopic image is an image generated based on the three-dimensional image data of the inspection target.

In the endoscope guiding apparatus, for example, the virtual endoscopic image and the information on the position and posture of the endoscope tip are obtained by a rigid endoscope, and the image is obtained. This is information on the position and posture of the tip of the rigid endoscope at that time.

In the above endoscope guiding apparatus, for example, the position and orientation of the endoscope distal end portion determined by the determining means is further superimposed on the image of the inspection object displayed in a two-dimensional sectional view or a three-dimensional display. Display means for displaying on the screen of the display device.

In the above endoscope guiding apparatus, for example, the display means may further display in the body that is not displayed in the real endoscopic image or the virtual endoscopic image having the highest similarity to the real endoscopic image. The affected area is displayed superimposed.

The above-mentioned endoscope guide device further comprises, for example, an image generating means for calculating a virtual endoscopic image based on the three-dimensional image data of the object to be inspected, and this image generating means is set to a predetermined value. Based on the position and orientation of the tip of the endoscope, interpolation calculation and rendering calculation of three-dimensional image data are executed based on the position and orientation to acquire a virtual endoscopic image. A large-scale virtual endoscopic image database can be constructed by setting and calculating the positions and postures of a large number of endoscope tips.

In the above endoscope guiding apparatus, for example, the comparing means compares the pixel value of the real endoscopic image and the pixel value of the virtual endoscopic image at the pixel position (i, j) with the real endoscope. When the virtual endoscopic image distributed on the smallest area is plotted from the database when plotted on a two-dimensional graph having the pixel value of the mirror image and the pixel position of the virtual endoscopic image as coordinate axes,
The virtual endoscopic image having the highest degree of similarity is set.

In the endoscope guiding apparatus, for example, the comparing means searches the virtual endoscopic image database for a virtual endoscopic image having the maximum mutual information, and the virtual endoscopic image having the highest degree of similarity is searched. Use as a mirror image.

In the endoscope guiding apparatus, for example, the comparing means uses rotation-invariant complex image associative processing.

In the endoscope guiding apparatus, for example, the comparing means uses the degree of similarity based on the final matching score after executing the adaptive nonlinear mapping.

In the above endoscope guiding apparatus, for example, when the endoscope is advanced, the comparing means always sets the search range to the vicinity of the search range of the previous time and sets the search range of the endoscopes of the previous time and this time. Based on the search radius corresponding to the difference in the insertion distance, the search for the virtual endoscopic image in the database is performed.

In the endoscope guiding apparatus, for example, the comparing means converts the real endoscopic image and the virtual endoscopic image into polar coordinate systems, respectively, and thereafter, the virtual endoscopic image having the highest degree of similarity. Choose.

In the endoscope guiding apparatus, for example, the comparing means sets the maximum value of the pixel intensity of each image of the real endoscopic image and the virtual endoscopic image to 1 in advance in the similarity evaluation. After normalization, the sum of squares of the intensity difference from the corresponding pixel is used.

In the above endoscope guiding apparatus, for example, the determining means may be arranged such that the estimated position of the endoscope tip obtained by the comparing means is a time series of the endoscope tip position estimated in the past. If the position is significantly deviated from the current predicted value extrapolated when arranged, the estimated position is not adopted. The various constituent elements of the endoscope guiding apparatus described above can be combined as much as possible.

In the endoscopic guidance method according to the present invention, the virtual endoscopic image to be inspected and the information on the position and orientation of the endoscope tip are stored in the database. When the real flexible endoscopic image of the inspection target is acquired, the real endoscopic image is compared with the virtual endoscopic image in the database, and the virtual endoscopic image having the highest similarity to the endoscopic image is obtained. To decide. Then, the position and orientation of the tip portion of the flexible endoscope corresponding to the real endoscopic image is determined from the information of the determined virtual endoscopic image.

In the above endoscope guiding method, it is preferable that the determined position and orientation of the endoscope distal end portion be
The image of the inspection target displayed in the two-dimensional cross-section or the three-dimensional display is superimposed and displayed on the screen of the display device.

In the above endoscope guiding method, it is preferable that the affected area in the body not displayed in the image is further superimposed on the real endoscopic image or the virtual endoscopic image having the highest degree of similarity. indicate. It should be noted that various components of the above-mentioned endoscope guiding device can be combined as components of the endoscope guiding method.

[0024]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the accompanying drawings. The present invention relates to navigation of a freely refracting endoscope. In recent years, surgery using an endoscope has been performed in many medical fields including the field of neurosurgery, but the operator may sometimes lose sight of where the tip of the endoscope is located in the patient. Therefore,
We have noticed that humans can move freely in the familiar city without hesitation without using a special position detecting device such as GPS. This is due to the sense of "I've seen this scene" and the function of the human brain that can associatively retrieve location information from the scene. In the present invention, the function of retrieving position information from such an image impression is realized by a computer algorithm. Here, the data of the relationship between the position and the image obtained by volume rendering from a group of tomographic images such as computer-aided tomography (CT) and magnetic resonance imaging (MRI) and image display by a surgical navigation system is accumulated. In particular, the three-dimensional position information of the endoscope tip operated by a practitioner such as surgery is estimated from the images registered in the past and provided to the operator.

First Embodiment of the Invention FIG. 1 shows the configuration of an endoscope guiding apparatus according to the first embodiment of the invention. This device is connected to a flexible endoscope (not shown) that bends freely, and assists the operator in grasping the positional relationship between the surgical operation site of the patient and the tomographic image (three-dimensional image). More specifically, in this apparatus, the network controller 10 externally reads a diagnostic image such as a three-dimensional image such as a CT image or an MRI image, or an image (endoscopic image) from a real endoscope. These diagnostic images are stored in the hard disk 12. The CPU 14 executes computer processing according to an endoscope guidance program based on an instruction from the input device 16 such as a keyboard and a mouse. Here, the endoscope guidance program stored in the hard disk (recording medium) in the hard disk device 12 is stored in the memory 1 when it is executed.
Loaded in 8. The image of the processing result is displayed on the display monitor 22 via the video controller 20.

FIG. 2 shows functional blocks in the endoscope guidance program. In the virtual endoscopic image database calculation step (S10), three-dimensional image data such as MRI and CT of an inspection target is input, a virtual endoscopic image is calculated based on the input three-dimensional image data, and a hard disk device It is stored in the database in 12. When endoscopic image data to be inspected is input during surgery or the like, in the comparison step (S12), the virtual endoscopic image in the database and the image to be inspected by the real endoscope (actual endoscopic The current position and orientation of the endoscope distal end portion is determined in real time by comparison with a mirror image). Next, in a display step (S14), the current position and orientation of the endoscope distal end portion obtained in the comparison step is displayed by being superimposed on the MRI image or CT image displayed on the screen of the monitor 22. To do.

In the calculation of the virtual endoscopic image, the M
It is necessary to calculate a virtual endoscopic image based on three-dimensional image data such as RI and CT. For this, a method of volume rendering 3D image data is known, and that method may be used. For details of the method, see, for example, Rubin, Perspective Volume Rendering of CTand.
MRI Images: Applications for Endoscopic Imaging
(Radiology, Volume 199, Pages 321 to 330, 199
6 years). Briefly, the three-dimensional image data such as MRI and CT to be inspected is three-dimensional volume data. When the endoscope is inserted into the examination target, the image that can be observed with the camera at the tip of the endoscope is the observation point set at this point, the light flux is diverged from that point, and it follows the straight path of each light. Simulation can be performed by calculating the color and the transparency of the volume element to be inspected.
For example, when the value in each volume element of MRI is small,
It is generally the air or bone region. When the value in each volume element of MRI is large, it is generally a region rich in water such as soft tissue. Therefore, if the value in each volume element of the MRI is small, the transparency is increased, and as the value in each volume element of the MRI is increased, the transparency is lowered, so that the air region and the soft tissue region can be distinguished with good contrast and three-dimensional display can be performed. .

Virtual endoscopic image database calculation (FIG. 2,
In S10), the observation point (viewpoint) and the observation attitude (virtual camera direction) are variously changed to obtain an image (virtual endoscopic image) simulating a large number of endoscopic images,
The virtual endoscopic image and viewpoint information are stored in the hard disk device 1
It is stored in the database in 2. The process is shown in more detail as a flow chart in FIG. First, CT in DICOM format, which is commonly used in hospital networks
Alternatively, the MRI image is read (S100). Next, the opacity and the color table for the voxel value are defined (step S102). Regarding the opacity with respect to the voxel value, the opacity below a predetermined threshold is zero (completely transparent), and the opacity above the threshold is 1 (completely opaque).
Then, the rendering speed increased. Next, the interpolation calculation method is set (step S104). Three-dimensional linear interpolation was used as the interpolation calculation method. Next, the shadow processing parameters are set (step S106), and the rendering method is selected (step S108). The shading parameters were adjusted by trial and error with multiple values. Moreover, the volume rendering by the ray casting method was adopted as the rendering method. It should be noted that high-speed calculation could be achieved by increasing the calculation step size on the ray as large as possible within the range where artifacts were not included in the display. Next, the viewpoint (camera) position, the center of rotation, and vertically upward viewpoint information are set (step S110). The viewpoint information is determined based on the volume rendering display of the entire target image by parallel projection with the camera position outside the body. next,
The viewing angle is set (step S112). As the viewing angle, the value of the viewing angle of the endoscope used at the same time was input.
Next, rendering is executed to acquire a virtual endoscopic image (step S114). Finally, the virtual endoscopic image and the viewpoint information are saved in the database file (step S116). The viewpoint information (observation point and observation attitude) is changed variously, and the processes of steps S110 to S116 are repeated to obtain an image (virtual endoscopic image) simulating a large number of endoscopic images.

FIG. 4 specifically shows the setting of the viewpoint information in the process (S10 in FIG. 2) for generating the virtual endoscopic image database. This process corresponds to steps S110 and S112 in FIG. Here, it is necessary to install virtual cameras at various positions and postures. For camera position,
1) First, an area of a camera position that is assumed when searching with a real endoscope is determined (S120), and then, all points that move in this direction in three axis directions in steps of 10 mm are adopted as camera positions. (S122). next,
Regarding the posture of the camera, 2) set the rotation of the camera image every 5 degrees, 3) move the edge position of the camera image in parallel by 1/10 of the image size, and 4) further enlarge the image by the virtual camera. 20% reduction rate from 50% to 200%
Are changed one by one (S124). These parameters 1) to 4) are independently changed (S126) to give a large number of camera positions and postures. A large-scale virtual endoscopic image database is generated by generating a rendering image based on this. When calculating the virtual endoscope image, it is preferable that the viewing angle of the virtual endoscope and the relative arrangement of the camera rotation center of the virtual endoscope and the camera position are the same as those of the real endoscope.

If such a database is used, in the image comparison step (FIG. 2, S12), the calculated virtual endoscopic image is compared with the image observed by a real endoscope (real endoscopic image). Then, the virtual endoscopic image having the highest degree of coincidence with the real endoscopic image can be searched from the inside of the database. In the image comparison step (S12 in FIG. 2), the degree of matching between the images stored in the virtual endoscopic image database and the current endoscopic image is checked, and the images are stored in the virtual endoscopic image database having the best match. The image is determined and the corresponding virtual camera position and orientation are obtained. As a result, the tip of the endoscope at this point is the target of inspection (for example, in the human brain)
It is possible to know in which position and in which direction of the. By superimposing and displaying this information on the two-dimensional or three-dimensional MRI image or CT image of the monitor 22, the endoscopic surgeon can continue complicated surgery without making a mistake in the determination of the position and direction. it can.

The specific method is shown in the flowchart of FIG. First, the virtual endoscopic image is displayed in gray scale off-line, and the change in shading of the virtual endoscopic image is adjusted so as to match the real endoscopic image captured at the same place as much as possible (S130). Specifically, if the MRI or CT imaging is performed while the endoscope is inside the body, and the real endoscopic image and the virtual endoscopic image of the same site are acquired, the above color adjustment can be performed interactively. Become. If the function for converting the pixel intensity of the virtual endoscope into the gray scale is interactively defined, the shading of the virtual endoscope image can be arbitrarily changed. Next, the virtual endoscopic image having the maximum correlation coefficient is searched from the virtual endoscopic image database (S132). Next, information on the position and orientation of the endoscope tip with respect to the obtained virtual endoscopic image is output (S
134).

More specifically, a method of calculating a correlation coefficient between images, which is usually used, is suitable for determining the degree of coincidence. With virtual endoscopy and real endoscopic images,
Since the correlation of color information is not generally high, it is desirable to perform the correlation calculation after converting a color image into a grayscale image. Therefore, the virtual endoscopic image is gray-scaled off-line, and the change in shading of the virtual endoscopic image is adjusted to match the real endoscopic image taken at the same place as much as possible. By defining a function that converts the pixel intensity of the virtual endoscope into gray scale, the shading can be defined arbitrarily. Color image (R, G, B color intensity is I
_R , I _G , I _B ) to grayscale image (intensity I)
The method of converting to can be calculated by the following equation (1), for example.

[Equation 1]

Next, the correlation calculation between the virtual endoscopic image and the endoscopic image is performed. For the endoscopic image pixel value A (i, j) and the virtual endoscopic image pixel value B (i, j) at the pixel position (i, j), the correlation coefficient r is defined by the following equation (2). It

[Equation 2] Here, U and V are vectors and are obtained by arranging pixel values of the endoscopic image and the virtual endoscopic image, and are specifically defined by the following equations (3) and (4).

[Equation 3]               U = (A (1,1), A (1,2), ..., A (2,1), A (2,2), ..., A (m, n))                           (3)

V = (B (1,1), B (1,2), ..., B (2,1), B (2,2), ..., B (m, n)) (4) U · V represents the inner product of the vectors U and V, and | U | represents the magnitude of the vector U. The correlation coefficient r is the vector U
Although the average value of V and V is not subtracted from the normal correlation coefficient, the above equation (2) is preferable when evaluating the degree of coincidence between two images including the DC component. If the root mean square of residuals is used for the evaluation function,
It is sufficient to set U | = | V | = 1. If this is not done, pixel differences between images with different scales will be evaluated, and no meaning can be found.

By the way, since the virtual endoscopic image and the real endoscopic image are different images, the pixel intensities are generally not proportional to each other. To calculate the correlation coefficient, the corresponding pixel intensities in the two images are
If they are proportional to each other, the correlation becomes maximum, so it is necessary to perform preprocessing so as to make them proportional. Such problems can be dealt with by an empirical method for each patient and each site.

FIG. 6 shows an example of the process of determining the image stored in the virtual endoscopic image database that best matches. The left side is the current endoscopic image 30 and the right side is the best matching virtual endoscopic image 32 in the database.

FIG. 7 shows the display step (FIG. 2, S14).
3 is a result of superimposing the obtained three-dimensional position of the endoscope distal end portion and the orientation of the camera at the present time on the orthogonal three-plane images 40a, 40b, 40c as an arrow. The size of the arrow is indicated by the projection length on three planes. For example, arrow 4
The position of the endoscope tip is made to correspond to the starting points of 2a, 42b, and 42c, and the arrow vector is taken as the direction vector of the camera. As a result, the treating doctor can correctly recognize the position of the distal end of the endoscope and can perform endoscopic surgery at the correct position. FIG. 8 shows a case where the same display is superimposed and displayed on the three-dimensional rendering image. Volume rendering or surface rendering can be used as the three-dimensional rendering image. The use of volume rendering has an advantage that contour extraction for each tissue is unnecessary.

Embodiment 2 of the Invention FIG. 9 shows an endoscope guidance program according to the second embodiment of the invention. Here, after the process of FIG. 2, the affected area (tumor, etc.) under the surface layer of the body that cannot be seen by the endoscope is further displayed on the virtual endoscopic image by volume rendering, and the surface layer is semi-transparently displayed. Is aligned (S16). As a result, the affected area (tumor, etc.) under the surface layer of the body can be additionally displayed on the endoscopic image, and accurate surgery can be performed under the endoscope. The condition for displaying such a tumor in the body by volume rendering is that the MRI image intensities of the normal tissue and the tumor tissue are greatly different, and it is known that this condition is satisfied in many cases. .

As an example, FIG. 10 shows an example in which the position of the affected area (here, tumor) under the surface layer in the body is superimposed on the image shown in FIG. The affected area under the surface layer of the body is displayed on the virtual endoscopic image 32 by volume rendering so that the surface layer is displayed semitransparently and is superimposed. In addition, endoscopic image 3
The affected part 34 under the surface layer of the body is additionally displayed at 0.

Third Embodiment of the Invention First Embodiment
Then, the virtual endoscopic image was converted into gray scale (FIG. 5, S120). By the way, the gray scale display of the real endoscopic image is determined by the color of the tissue surface layer (ratio of R, G, B and the scaling value, that is, brightness), and the color is determined by the blood flow state and metabolic state of the tissue surface layer of the patient. On the other hand, CT or MR
The color when a virtual endoscopic image is constructed from the I image is CT
It is determined by the corresponding function of the value or MRI intensity and the display color. This correspondence function can be set arbitrarily. From this, when the patient is changed, the color of the tissue surface layer has individual differences. Therefore, it is generally inappropriate to use the offline measurement result of another patient by the method described in the third embodiment of the invention. I know there is. It can also be understood that it is generally inappropriate to use the offline measurement result of another tissue of the same patient because the blood flow state and the metabolic state of the tissue surface layer change if the tissue is different even in the same patient.

Because of such circumstances, ideally, the grayscale shading or color matching of the real endoscopic image and the virtual endoscopic image should be adaptively and automatically calculated for each image. . However, since this is complicated, the processing of the present embodiment that avoids this will be described. Here, a diagram called a joint histogram or a two-dimensional histogram is used, but this method is known as a method for aligning different types of images. CRC Press in 2001
Book, Medical Image Registratio
n, pages 57-59.

FIG. 11 shows the image comparison process in this embodiment. Specifically, as shown in FIG. 12A, the pixel value A (i, j) of the real endoscopic image and the pixel value B (i, j of the virtual endoscopic image at the pixel position (i, j). j), the point (A (i, j), B
(i, j)) and then two-dimensionally plot on the graph (S1
31), a virtual endoscopic image distributed on the smallest area is searched from the database (S133). Then, information on the position and orientation of the camera corresponding to the obtained virtual endoscopic image is output (S134). The method of determining the smallest area includes a method of interactively performed by an operator, a method of drawing a closed curve including plotted points, and a calculation of a correlation coefficient by an inner product operation of vectors described in the first embodiment of the invention. . The distribution on the smallest area means that
It shows that there is a strong correlation between the pixel value displayed in the real endoscopic image and the pixel value displayed in the virtual endoscopic image.
Strong association includes so-called positive correlation and strong negative correlation, but also includes more complex cases (Fig. 1).
2 (b)). Even if such a complicated relationship is strong, the calculation of the correlation coefficient by the vector inner product operation described in the first embodiment of the invention is effective. In any case, distribution on the smallest area means that the two images are very likely to be the same tissue viewed from the same camera position.

Fourth Embodiment of the Invention Embodiment 3 of the invention
As a method that further expands the processing using the correspondence function of
It is conceivable to use a method of evaluating the mutual information amount in the image comparison processing. The flow chart is shown in FIG. Registration of different images using mutual information (registration)
With regard to 60 to 60 of the documents cited in the third embodiment of the invention,
It is described on page 61. First, the mutual information amount is calculated using the pixel values of the real endoscopic image A (i, j) and the virtual endoscopic image B (i, j) at the pixel position (i, j) (S131 ').
Mutual information qualitatively gives how well one image information can explain the other image information,
For example, the mutual information I (A,
When B) becomes maximum, it is determined that the two images can be aligned.

[Equation 5] (5) Here, p ^T _AB (a, b) is a joint probability distribution function of images A and B, and p ^T _A (a) and p ^T _B (b) are marginal probability distribution functions. The definition of the mutual information is not limited to the above, and various other proposals have been made, and thus they may be used. Next, a virtual endoscopic image having the maximum mutual information amount is searched from the virtual endoscopic image database (S1).
33 '). Then, information on the position and orientation of the camera corresponding to the obtained virtual endoscopic image is output (S134).

Fifth Embodiment of the Invention When the correlation coefficient, squared error, mutual information, etc. are used for the similarity evaluation as described above, the virtual endoscopic image having the maximum similarity must exist in the virtual endoscopic image database. Therefore, the scale of the database is unavoidable. Generally, such processing needs to be calculated off-line,
It may be difficult to apply to emergency patients who require emergency surgery. Further, it is possible that the estimation of the optimum virtual endoscopic image may fail when noise is mixed in the endoscopic image. Therefore, in the embodiment of the present invention, by adopting the associative processing which is one of the brain functions, the database is downsized and the noise resistance performance is improved. In order to detect its position from the image obtained by the endoscope by associative processing, the endoscope is expected to rotate at an arbitrary angle during insertion. Is required to be realized.

The following is the Rotation-Invariant Image Association for En published by Aoki et al. In June 2001.
doscopic Positional Identification Using Complex-V
alued Associative Memories (Springer, Proceedings
of the 6th International Work-Conference on Artifi
cial and Natural Neural Networks Granada, Part I
I, pp.369-376), the outline of the rotation-invariant image associative processing for endoscope position detection using a complex number type associative memory is described. The process using associative memory is an inscription process that stores necessary patterns in advance (Phase 1)
And an incomplete pattern of the memory pattern (a pattern in which noise is mixed or some information of the pattern is missing) is given after that, the memory pattern that is more complete than the incomplete pattern is associated and output. The processing is divided into two processing steps (phase 2).

The phase 1 inscription process is a process of determining the connection weight between neurons from the pattern to be stored, and the information of the storage pattern is stored in the connection weight between neurons in a multi-distributed manner. In the recall process of phase 2, each neuron updates its state one after another according to a predetermined state transition rule. It is possible to easily handle multi-valued pattern information by applying a complex number type associative memory model that is an extension of the real number type. In particular, a grayscale image associative system is realized by combining a complex number type associative memory model (CAMM) and two-dimensional discrete Fourier transform processing (2-D DFT).

With reference to FIG. 14, an outline of a method of realizing an image associative system that is invariant to a rotating operation will be described. This system handles a circular image as shown in FIG. The following description is common to the inscription process and the recall process.

(A) First, the circular image shown in FIG. 14 (a) is converted into a rectangular image as shown in FIG. 14 (b). Then, the rotation operation in the θ direction around 0 in the circular image is replaced with the shift operation only in the n direction (horizontal direction) in the rectangular image.

(B) Two-dimensional discrete Fourier transform processing (2-D DFT) is performed on the figure converted into the format shown in (b) of FIG. The result of the Fourier transform is given as shown in FIG. Here, the four corner portions represent low frequency spatial frequency components, and the central portion represents high frequency spatial frequency components.

(C) Let each component of the Fourier transform of (c) of FIG. 14 be ν (m ', n'), ν (m ', n') = | ν (m ', n') | exp ( α
It can be represented as (m ', n')). The following formula (6),
Generate the matrix {χ _phase (m ', n')} represented by only the phase information by performing the process of multiplexing the amplitude information of ν (m ', n') with the phase information using (7). . In equation (6), | ν _max
(m ', n') | represents the maximum value in each element | ν (m ', n') |.
Equation (7) defines χ _phase (m ', n'). Thus, by converting the image information into the χ _phase (m ', n') format, the image information can be expressed by the complex neuron shown in FIG.

Γ (m ', n') = cos ^-1 (| ν (m ', n') | / | ν _max (m ', n') |) (6)

(7) χ _phase (m ', n') = exp {i (α (m ', n') + γ (m ', n'))} (7)

(D) As described above, the information of the image is χ above.
_It is converted into the form of _phase (m ', n') and given to the complex number type associative memory model (CAMM).

(E) Next, phase quantization of χ _phase (m ', n') is performed.

(F) Normally, if the CAMM is constructed using all the components of χ _phase (m ', n'), the number of neurons becomes very large, so that the deterioration of the image quality is not noticeable as shown in FIG. As shown in (c), C
Configure the AMM. At that time, the CAMM is constructed for each column in the low frequency region of χ _phase (m ', n'). For example, the matrix {χ _phase
If the columns 0 to X and N−X to N−1 of (m ′, n ′)} are used, the number of CAMMs created is 2X + 1. The connection weight between the neurons of the CAMM is determined.

The following is a description of the recall process. (G) Here, pay attention to the following property of the two-dimensional discrete Fourier transform (2-D DFT). The two-dimensional discrete Fourier transform of the image shifted by q in the n direction only is expressed as follows.

## EQU00008 ## If ν (m ', n') = DFT {u (m, n)}, u (m, n) is shifted by q in the n direction only. The dimensional discrete Fourier transform is given by

Equation 9] DFT {u (m, nq) } = ν (m ', n') exp (-iθ N n'q) Here θ _N = 2π / _N In other words, when viewed in the complex region side exp (- The term is θ _N n'q) is only added. (H) As described in (F), if a CAMM is constructed for each column of the matrix {χ _phase (m ', n')}, the value of n'is the same in one CAMM, so the effect of rotation Term exp (-i θ _N n '
The value of q) is the same in each neuron. That is, the influence of the image rotation operation is exp (-i
sθ _N ) terms are only added.

(I) On the other hand, CAMM has a property of absorbing the influence of exp (-isθ _N ) (a property that is invariant with respect to phase shift).
With. That is, when the memorization of the state pattern x ₁ as equilibrium CAMM, the state x ₁ S.theta _N (s is an arbitrary integer) states exp (-isθ _N) obtained by shifting only the phase x ₁ also C
The AMM is in equilibrium. (However, in this case, the quantized number K of the complex neuron is set to be an integral multiple of N.) In other words, in CAMM, x ₁ is also exp (-is θ _N ) x.
_{Even 1} is memorized equally. Thus, a rotation-invariant associative memory system can be realized by using the properties (H) and (I).

(J) The remaining problem is how to reconstruct the recalled image from the results recalled by each CAMM. Even if the result recalled by each CAMM is directly converted into the real number domain by performing the inverse two-dimensional discrete Fourier transform (2-D IDFT), a correct recalled image cannot be obtained. From the result recalled by each CAMM, it is necessary to correct the phase component of the influence term exp (-isθ _N ) due to rotation.

Therefore, a neuron for detecting the above correction amount is newly embedded in each CAMM (thereby the number of neurons in each CAMM becomes R + 1).
This neuron is stored in the CAMM at a predetermined phase (this is the reference phase), the phase of this neuron is checked each time the recall process is completed, and the correction amount is detected from the deviation from the reference phase. I shall.

(K) A matrix ν _a (m ′, n ′) of the form shown in the figure below is created from the result of the CAMM-corrected recall. ν
0 is added to each column from _a (m ', X + 1) to v _a (m', NX-1). (L) From this matrix {ν _a (m ', n')}, the recalled image {u _a (m, n)} is obtained using the following equation (10).

[Equation 10]

In the first embodiment of the invention, the step of setting the rotation of the camera image every 5 degrees as the image to be stored in the virtual endoscope database is used. But,
By using the process described above, this rotation becomes unnecessary. As a result, there is an effect that the size of the database becomes 5/360 = 1/72 when the database is rotated every 5 degrees.
This means that the calculation time for similarity evaluation is 1/72. Furthermore, there is an effect that even if noise is mixed in the image, the estimation can be correctly performed.

Sixth Embodiment of the Invention In the hybrid system of this embodiment, the rigid endoscopic image is interpolated by the MR rendering image without using the virtual endoscopic image based on the three-dimensional image data such as the MRI or CT of the examination target. In the case of a rigid endoscopic image, the mechanical method described as the conventional technique (for example, Kosugi et al, An Articulated Neurosurg
ical Navigation System Using MRI and CT Images (IE
EE Transactions on Biomedical Engineering, Vol. 35,
No. 2, February 1988), and US Pat.
Issue 08 (E. Watanabe, Y. Kosugi, and S. Manaka: Sys
tem for indicating a position to beoperated in ap
Atient's body). As shown in FIG. 15, the position coordinates and posture of the tip of the rigid endoscope can be obtained by a multi-joint arm navigation device having a rotary encoder. FIG. 16 shows a flow chart of processing in the hybrid system. First, a rigid endoscopic image of a place that is likely to become a landmark and three-dimensional positions and postures of the distal end of the rigid endoscope are acquired at a plurality of points in preparation for surgery and stored in a database (S20). Then, the operation is performed by replacing the endoscope with a soft optical fiber system (see FIG. 17). Here, a rigid endoscopic image having a high degree of similarity to the optical fiber endoscopic image obtained while advancing the optical fiber endoscope is selected from the database (S22). Then, in the vicinity of the point where the image having a high degree of similarity is selected, a local search is performed by evaluating the degree of similarity of different images (S2).
4). In this case, the rigid endoscope and the optical fiber type endoscopic image are the same type of image, and the similarity evaluation by the database is easy. Therefore, it is possible to accurately know the position of the tip of the optical fiber type endoscope at a required place as a landmark. Also, in the vicinity of these landmarks, similarity evaluation of different images is performed, but since local search is sufficient, the probability of false recognition of another similar place can be reduced, and safe navigation can be performed. . This method is useful because it is a tool to prevent mistakes in similar places in the first place, so mistakes cannot be tolerated.
Then, information on the position and orientation of the camera with respect to the obtained virtual endoscopic image is output (S26). Then, based on this result, the images are displayed in a superimposed manner as described with reference to FIGS.

Seventh Embodiment of the Invention As for the similarity, a method of using the similarity evaluation based on the final matching score after implementation of the adaptive non-linear mapping can be considered. The case where this processing is effective is the case where a sufficiently high degree of similarity cannot be obtained even if the degree of similarity is evaluated by any of the methods described above. If the tissue is locally distorted due to the insertion of the endoscope, it is necessary to evaluate the similarity by considering the spatial non-linear distortion included in the real endoscopic image. In particular,
While deforming the endoscopic image or the calculated virtual endoscopic image spatially by applying strain such as elastic deformation, a deformation pattern in which the degree of similarity increases is searched for. The details of such an adaptive non-linear mapping are described in Japanese Patent Publication No. 2860048 "Non-linear image conversion device and standard image calculation method", so that details will be omitted, but a brief description will be given with reference to FIGS. 18 and 19. .

FIG. 18 is a flow chart of a process using the similarity evaluation by the final matching score after implementation of the adaptive non-linear mapping. First, from the data of the first image, the first image is divided into a predetermined number to generate the data of the small area, and the data is stored as the data of the first image layer (S30). For example, in FIG.
It is divided into 4 × 4 (16) small areas as shown in FIG. Next, the data of the second image is saved (S32).
For example, the first image is a real endoscopic image and the second image is a virtual endoscopic image.

Then, with respect to the data of each small area of the first image, the movement vector is added to the position vector indicating the position of each small area, and the position is slid (S34).
The initial value of the movement vector D corresponding to each small area is 0 vector. Next, the movement amount (increment vector) having the highest similarity between each small area of the first image and each small area of the second image after the slide is searched for for each small area (S36). One of the methods described so far is used to evaluate the similarity. That is, a correlation coefficient, a squared error, or the like.

Next, the movement vector calculated separately for each small area is smoothed (S38). This is to smooth the change of the movement vector for each adjacent small area. However, the degree of smoothing is changed by the number of revolutions of an iterative loop that repeats the processing of steps S34 to S38. In this way, since the degree of smoothing is changed depending on the number of loop turns, this process is called adaptive non-linear mapping. Here, without smoothing much at first,
When the characteristics of each small area are emphasized and the degree of similarity improves locally, finally, the movement vector is smoothly changed in consideration of the continuity with the surroundings. in this way,
Due to the iteration, the calculation of the motion vector begins with the image after the previous motion within the iterative loop. That is, the increment vector of the movement amount is calculated every time the loop is repeated. Therefore, after the second iteration, after calculating the increment vector,
Calculation for obtaining the overall movement vector (step S40)
Will be required. If the similarity is sufficiently improved, that is, if the matching is successful (YES in S42), the movement vector for all pixels is calculated (S44). This allows
The operator can observe the real endoscopic image and the virtual endoscopic image displayed on the screen in consideration of the movement vector.

Eighth Embodiment of the Invention First Embodiment of the Invention
However, as described above, since the real endoscopic image and the virtual endoscopic image have a degree of freedom of rotation with respect to each other, the camera is rotated every 5 degrees to generate the virtual endoscopic image and construct the database. . Next, all the images in the database need to be evaluated in order to retrieve the virtual endoscopic image with the highest similarity. This is unavoidable because the relationship between the rotation angle of the camera and the degree of similarity between the real endoscopic image and the virtual endoscopic image is unknown in advance. An example of calculating the relationship between the degree of similarity and the rotation angle between the real endoscopic image and the virtual endoscopic image in FIG. 6 is shown below. The rotation was performed on the virtual endoscopic image.

FIG. 20 is a graph showing the result of evaluation of the degree of similarity in the Cartesian coordinate system. As the similarity, the maximum value of each image of the real endoscopic image and the virtual endoscopic image is standardized to 1 in advance, and then the sum of squares of the intensity difference (Erro
r) was used. Of course, the correlation value described in paragraph 0031 may be used. Since the degree of similarity is given by the sum of squares of pixel intensity values (denoted by Error on the vertical axis), the maximum degree of similarity is obtained when the minimum value is given. When the angle (Rotation) is changed, the maximum degree of similarity is given at an angle of 0 to 30 degrees, but there is a minimum value near 130 degrees, and optimizations such as normal division method and quadratic interpolation method are performed. It shows that the true angle may not be obtained when the search is speeded up using the method. In addition, the rotation angle is 0 to 30 degrees, which is a rather vague solution.

However, when the above two images are converted into polar coordinates and then the similarity is evaluated, the minimum value of the error (Error) does not have other local minimum values and the rotation angle (Rotation) as shown in the graph of FIG. ) It was found that it was clearly given as 0 degree. Therefore, it is shown that the true angle can be obtained when the search is optimized by using an optimization method such as a normal division method or a quadratic interpolation method. When the image before conversion (orthogonal coordinate system) is A (x, y) and the image after conversion to polar coordinate system is A ^* (r, theta), r = (X ² + Y ² ) ^1/2 , theta = Tan
Each component of the image A ^* (r, theta) after conversion into the polar coordinate system can be calculated from ⁻¹ (y / x). From the above, it was found that the evaluation can be speeded up by evaluating the similarity after converting the image into the polar coordinate system in advance.

Ninth Embodiment of the Invention Next, a method of speeding up the search of the virtual endoscopic image database will be described. Here, using the initial position information that introduced the endoscope into the body,
The virtual endoscopic image database is sufficiently narrowed down and searched. FIG. 22 shows a flow chart of the processing. That is, M
When acquiring the RI image or the CT image, the initial position of introducing the endoscope into the body is pasted with a microsphere marker containing a liquid on the body surface to determine the initial camera position of the virtual endoscopic image. (S50). When the endoscope is further advanced, the search range is always set near the previous search position (S52). At this time, the search in the virtual endoscope database is executed based on the search radius corresponding to the difference between the insertion distances of the endoscopes of the previous time and this time (S54). This makes the search faster and more accurate.

Tenth Embodiment of the Invention The endoscope tip position is estimated by the above-described various processes. Here, it is preferable to reject the solution when a position which is significantly deviated from the current predicted position value extrapolated when the endoscope tip positions estimated in the past are arranged in time series is tried. .
FIG. 23 shows a flow chart of the processing. The estimated value of the endoscope position is input (S60). On the other hand, the current predicted position value extrapolated when the endoscope tip positions estimated in the past are arranged in time series is calculated (S62) and compared with the estimated value of the endoscope position (S64). If the difference between the two values is smaller than the predetermined value, it is adopted, but if the difference between the two values is equal to or larger than the predetermined value, the estimated value is not adopted. This prevents a search error.

[0069]

According to the endoscopic guidance device and method according to the present invention, a virtual endoscopic image of an examination object and a database consisting of information on the position and orientation of the endoscope tip and a real endoscopic image are displayed in real time. By making a comparison with, it becomes possible to quickly and accurately guide the endoscope to the affected area given by the advanced image diagnostic technique. For example, in tumor resection surgery, it is possible to perform surgery with minimal residual tumor tissue, in a short time, and with minimal invasion to the patient. It is possible to prolong life and discharge earlier.

Further, three-dimensional image data (CT
Since the image is compared with the virtual endoscopic image based on the image, the diagnostic image such as the MRI image), the comparison with the real endoscopic image becomes easy.

Further, an image obtained by a rigid endoscope,
Since the position and orientation information of the tip of the rigid endoscope when the image is obtained is used, since the rigid endoscope and the optical fiber type endoscope image are the same type of image, it is easy to evaluate the degree of similarity using the database. Become.

Further, since the position and posture of the endoscope distal end portion are displayed by being superimposed on the image of the inspection object displayed in the two-dimensional sectional view or the three-dimensional display, the endoscopic surgery worker can check the position and the direction. You can continue complicated surgery without making a mistake.

Further, since the affected area (tumor, etc.) under the surface layer of the body can be superimposed and displayed also on the endoscopic image, it is possible to carry out an accurate operation under the endoscope.

Further, since the image generating means for calculating the virtual endoscopic image based on the three-dimensional image data of the inspection object is provided, the virtual endoscopic image at the positions and postures of a large number of endoscope tips is obtained. You can build a database.

Further, since the images of different types are aligned using a diagram called a joint histogram or a two-dimensional histogram, the virtual endoscopic image having the highest degree of similarity can be easily determined.

Further, since the comparing means uses mutual information, the virtual endoscopic image having the highest degree of similarity can be easily determined.

Further, since the invariant complex number associative processing is adopted for the rotation operation, the database of the virtual endoscopic image can be downsized and the noise resistance performance can be improved.

Further, since the comparing means uses the similarity based on the final matching score after the adaptive non-linear mapping, the virtual endoscopic image having the highest similarity can be easily determined.

Further, since the comparison means evaluates the degree of similarity after converting the image into the polar coordinate system, the evaluation can be speeded up.

Further, when the endoscope is moved forward, the search range is set close to the previous search range, and the search is performed in the database of virtual endoscopic images, so that the search can be sped up.

Further, preferably, when the estimated position of the endoscope tip is a position which deviates significantly from the predicted position value, the estimated position is not adopted, so that an incorrect estimation can be prevented.

[Brief description of drawings]

FIG. 1 is a block diagram of an endoscope guidance device.

[Fig. 2] Flow chart of the endoscope guidance program

[Fig. 3] Flow chart of virtual endoscopic image database calculation

FIG. 4 is a flowchart of viewpoint information setting processing in generating a virtual endoscopic image database.

FIG. 5 is a flow chart of search processing using grayscale conversion of an endoscopic image.

FIG. 6 is a diagram showing an example of processing in which an image stored in a virtual endoscopic image database that best matches is determined.

FIG. 7 is a diagram in which the three-dimensional position of the endoscope distal end portion and the orientation of the camera at the present time are displayed as arrows on an orthogonal three-plane image in an overlapping manner.

FIG. 8 is a diagram in which the three-dimensional position of the endoscope distal end portion and the direction of the camera at the current time are displayed as an arrow superimposed on an orthogonal three-plane image.

FIG. 9 is a flow chart of an endoscope guidance program.

FIG. 10 is a diagram in which tumor positions are superimposed and displayed.

FIG. 11 is a flowchart of image comparison processing.

FIG. 12: Two-dimensional graph of real endoscope pixel value and virtual endoscope pixel value

FIG. 13 is a flowchart of image comparison processing using mutual information.

FIG. 14 is a diagram for explaining the basic idea of a rotation-invariant associative memory system.

FIG. 15 is a diagram showing how to obtain an endoscopic image and its posture information using a rigid endoscope.

FIG. 16 is a flow chart of processing in a hybrid system

FIG. 17 is a diagram showing a situation in which an operation is performed by replacing with a fiber optic type soft endoscope.

FIG. 18 is a flowchart of a process using similarity evaluation based on a final matching score after implementation of the adaptive nonlinear mapping.

FIG. 19 is a diagram showing division into small areas.

FIG. 20 is a graph of the relationship between the degree of similarity and the rotation angle between the real endoscopic image and the virtual endoscopic image in the orthogonal coordinate system.

FIG. 21 is a graph of the relationship between the degree of similarity and the rotation angle between the real endoscopic image and the virtual endoscopic image in the polar coordinate system.

FIG. 22 is a flowchart of processing for speeding up the search of the virtual endoscopic image database.

FIG. 23 is a flowchart of processing that does not adopt a position that deviates significantly from the current predicted value.

[Explanation of symbols]

10 network controller, 12 hard disk, 14 CPU, 16 input device,
18 memories, 22 monitors.

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) A61B 6/03 377 A61B 6/12 6/12 G02B 23/26 C G01R 33/28 D G02B 23/26 H04N 7/18 M A61B 5/05 390 H04N 7/18 G01N 24/02 Y (72) Inventor Atsushi Nagata 4-8-1402 Wakabadai, Asahi-ku, Yokohama-shi, Kanagawa Prefecture (72) Yukio Kosugi Midori-ku, Yokohama-shi, Kanagawa Prefecture 4259 Nagatsutacho Graduate School of Science and Engineering, Tokyo Institute of Technology (72) Inventor Hidetoshi Watanabe 5-29-12-1001 Hongo, Bunkyo-ku, Tokyo (72) Inventor Hiroyuki Aoki 1220-2, Kusadacho, Hachioji, Tokyo F-term in technical college (reference) 2H040 BA04 BA14 BA21 CA11 DA03 DA12 DA14 GA02 GA11 4C061 AA23 BB02 CC06 HH51 JJ11 JJ17 LL01 NN05 NN07 SS21 TT06 TT07 WW03 WW04 WW06 W W13 YY03 YY12 YY13 YY14 4C093 AA22 CA17 CA18 CA50 EE30 FD07 FD20 FF12 FF17 FF33 FF43 FG01 4C096 AA18 AA20 AB50 AD14 AD15 AD19 AD23 DC14 DC18 DC28 DC31 DC36 DC37 DD08 DE02 DE06 DE08A 04FD01 A05 FC01 5C054

Claims (17)

[Claims] 1. A storage unit for storing a database including a virtual endoscopic image of an inspection target and information on the position and orientation of the endoscope tip, and an input unit for acquiring a real flexible endoscopic image of the inspection target, Comparing means for comparing the real endoscopic image with the virtual endoscopic image in the database to determine a virtual endoscopic image having the highest degree of similarity to the endoscopic image; and the determined virtual endoscopic image An endoscopic guidance device comprising: a determining unit that determines the position and orientation of the tip portion of the flexible endoscope corresponding to the real endoscopic image from information of the mirror image. 2. The virtual endoscopic image is a 3
The endoscopic guidance device according to claim 1, wherein the endoscopic guidance device is an image generated based on three-dimensional image data. 3. The virtual endoscopic image and the position and posture information of the endoscope tip are the image obtained by a rigid endoscope and the position of the tip of the rigid endoscope when the image is obtained. The endoscope guidance device according to claim 1, wherein the endoscope guidance device is information regarding the position and the posture. 4. A display for superimposing the position and orientation of the endoscope distal end portion determined by the determination means on the image of the inspection object displayed in a two-dimensional sectional view or three-dimensional display on the screen of the display device. And means.
The endoscope guiding device according to claim 3. 5. The display means further displays an actual endoscopic image or a virtual endoscopic image having the highest degree of similarity with the real endoscopic image by superimposing an affected part in the body which is not displayed in the image. The endoscope guiding apparatus according to any one of claims 1 to 4, which is characterized. 6. An image generating means for calculating a virtual endoscopic image based on three-dimensional image data of an inspection object is provided, and the image generating means is for each set position and orientation of the endoscope distal end. The endoscopic guidance device according to any one of claims 1 to 5, wherein the virtual endoscopic image is acquired by executing interpolation calculation and rendering calculation of the three-dimensional image data based on the above. 7. The comparing means compares the pixel value of the real endoscopic image and the pixel value of the virtual endoscopic image at the pixel position (i, j) with the pixel value of the real endoscopic image and the virtual endoscopic image. A characteristic is that a virtual endoscopic image distributed over the smallest area when it is plotted in a two-dimensional graph with the pixel position of the mirror image as the coordinate axis is searched from the database and the virtual endoscopic image with the highest degree of similarity is made. The endoscope guiding apparatus according to any one of claims 1 to 6. 8. The comparison means searches the virtual endoscopic image database for a virtual endoscopic image having the maximum mutual information amount, and sets it as the virtual endoscopic image having the highest degree of similarity. The endoscopic guidance device according to any one of claims 1 to 7. 9. The endoscope guiding apparatus according to claim 1, wherein said comparing means uses rotation-invariant complex image associative processing. 10. The endoscopic guidance according to claim 1, wherein the comparing means uses the degree of similarity based on the final matching score after performing the adaptive non-linear mapping. apparatus. 11. The comparison means, when the endoscope is advanced, the search range is always near the previous search range,
11. The search in the database of virtual endoscopic images is performed based on the search radius corresponding to the difference between the insertion distances of the endoscopes of the previous time and this time. Endoscope guidance device. 12. The comparison means selects the virtual endoscopic image having the highest degree of similarity after converting the real endoscopic image and the virtual endoscopic image into polar coordinate systems, respectively. The endoscope guiding device according to any one of claims 1 to 11. 13. The comparing means normalizes the maximum value of the pixel intensities of the images of the real endoscopic image and the virtual endoscopic image to 1 in advance in the similarity evaluation, and then, The endoscope guidance device according to any one of claims 1 to 12, wherein the sum of squares of the intensity difference between and is used. 14. The present determination means extrapolates the estimated position of the endoscope tip obtained by the comparison means when the estimated endoscope tip positions are arranged in time series. The endoscope guiding apparatus according to any one of claims 1 to 13, wherein the estimated position is not adopted when the position is significantly deviated from the predicted position value. 15. A virtual endoscopic image to be inspected and information on the position and orientation of the tip of the endoscope are stored in a database to obtain a real flexible endoscopic image to be inspected, and the real endoscope. The image is compared with the virtual endoscopic image in the database to determine the virtual endoscopic image having the highest degree of similarity with the endoscopic image, and from the information of the determined virtual endoscopic image, the real An endoscope guidance method for determining the position and orientation of the tip portion of a flexible endoscope corresponding to an endoscopic image. 16. The position and posture of the endoscope distal end portion that has been determined are displayed on the screen of the display device by being superimposed on the image of the inspection object that is displayed in a two-dimensional sectional view or a three-dimensional display. The method for guiding an endoscope according to claim 15. 17. The diseased part in the body which is not displayed in the image is superimposed and displayed on the real endoscopic image or the virtual endoscopic image having the highest degree of similarity. Alternatively, the endoscope guiding method according to claim 16. Endoscope guidance method.