JP3070022B2 - Ultra wide-angle endoscope - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an ultra-wide-angle endoscope provided with an ultra-wide-angle imaging system, and more particularly to an industrial endoscope for inserting a tubular optical system into a narrow gap and using it for optical inspection inside an industrial product. The present invention relates to a mirror or an ultra-wide-angle endoscope suitable for a medical endoscope used for examining or treating the inside of a body cavity while reducing the physical burden on a patient.

[0002]

2. Description of the Related Art Conventional endoscopes are used for both industrial and medical purposes.
As its structure, there are a flexible flexible endoscope and an inflexible rigid endoscope, both of which have an imaging optical system installed at the tip thereof. The optical image formed at the distal end is transferred to the rear end of the endoscope via a glass fiber bundle for a flexible endoscope, and via a relay lens system for a rigid endoscope. The structure is such that the image is transmitted to an observation eyepiece lens installed in the camera or a television camera. Some endoscopes have a light receiving element array arranged at a position where an image is formed by the tip optical system to convert an image into an electric signal.

[0003] Endoscopes generally range from a narrow viewing angle of about 30 degrees to a wide viewing angle of 80 degrees or more.
In an endoscope, a visible range and a direction are important for an operation, and therefore, an endoscope (referred to as a perspective mirror) in which the direction of an optical axis is changed for each application of the operation is manufactured. That is, assuming that the viewing angle from the distal end of the endoscope as viewed from the extension of the tube (fiber bundle tube) of the endoscope is 0 °, in addition to 0 ° of the forward-looking type (direct-viewing type), 30 degrees, 60 for visual type
There is a perspective mirror such as 90 degrees for the side-view type and 120 degrees for the rear oblique type. In any case, in the current situation, an excessive force is often applied to a wall surface or the like by mistake because a perspective mirror deviates from what is seen in the traveling direction.

[0004]

The problem of the conventional endoscope is that of an endoscope for intranasal surgery, which requires careful attention due to the operation of the endoscope. Points will be described.

The operation using the endoscope can be divided into three stages as follows.

[0006] 1) First, an endoscope is inserted into a nasal cavity and observed with a television monitor or an eyepiece.

2) Next, surgical instruments (tweezers, forceps,
Scissors) into the nasal cavity and guide them into the field of view of the operation target site.

3) After that, the operation of the surgical instrument is performed.

Of the above operations, the most difficult is the operation 2). Normally, it is not possible to insert a surgical instrument directly after the operation 1). Actually, since the screen shown on the endoscope is deep in the nasal cavity and has a narrow field of view, it is difficult to see the tip of the instrument. Therefore, after observing the inside of the nasal cavity with an endoscope, the endoscope is once taken out of the nasal cavity, and the endoscope is put back into the nasal cavity together with necessary instruments. At that time, the tip of the instrument is inserted a little before the endoscope so that you do not lose track of the tip of the instrument while observing on the endoscope screen. there were.

In particular, such a perspective mirror has the following drawbacks in operation.

A) In a portion outside the visual field that is not visible on the screen,
In some cases, it may not be able to move forward when hitting a structure of the human body, and may damage the mucous membrane and cause bleeding. Then,
It is difficult to continue surgery.

B) Even when the objective lens is visible on the screen, the objective lens may touch the mucous membrane surface and become cloudy due to the adhesion of mucus, and the endoscope may need to be removed once and the lens surface must be wiped again.

C) When changing the viewing direction, the endoscope must be rotated around the axis of the endoscope. In this case, two-handed operation is required, and it is necessary to release the surgical instrument. More importantly, there is a danger of losing direction while turning the endoscope.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-mentioned points, and an object of the present invention is to prevent a conventional oblique mirror from being seen in the insertion direction at the time of insertion, so that a frequency at which excessive force is applied to a wall surface. It is an object of the present invention to provide an ultra-wide-angle endoscope that solves operational problems such as a high endoscope and that can properly secure a visual field in a direction required during work.

[0015]

In order to achieve the above object, the invention of an ultra wide-angle endoscope according to claim 1 is an invention in which an imaging optical system is provided at the tip of a tube using an optical fiber or a relay lens system. In an endoscope, an ultra-wide-angle lens system provided at the tip of the tube as the imaging optical system, and a solid-state imaging means for receiving a wide-range optical image captured by the ultra-wide-angle lens system and converting the optical image into an image signal Image processing means for performing image processing for converting a wide-angle image obtained from the image signal of the solid-state imaging means into an image having a normal viewing angle; and displaying an image based on an output signal of the image processing means. Image display means, wherein the image processing means includes: image storage means for sequentially storing image data input from the solid-state imaging means for each image frame; and a fixed range centered on a direction designated by a pointing device. A line-of-sight direction calculating means for calculating a line-of-sight direction; calculating a position of a corresponding pixel on the solid-state imaging means with respect to each line-of-sight direction calculated by the line-of-sight direction calculating means; And a corresponding pixel operation means for reading and outputting the image signal from the image storage means.

According to another aspect of the present invention, the above object is achieved.
The invention of an ultra-wide-angle endoscope according to the present invention relates to an endoscope having an imaging optical system at the distal end of a tube, wherein an ultra-wide-angle lens system provided at the distal end of the tube as the imaging optical system; A solid-state imaging unit that is provided near the tip of the body and receives a wide-range optical image captured by the ultra-wide-angle lens system and converts it into an image signal; and a wide-angle image obtained from the image signal of the solid-state imaging unit. Image processing means for performing image processing for converting the image into an image having a normal viewing angle, and image display means for displaying an image based on an output signal of the image processing means, wherein the image processing means comprises the solid-state imaging means Image storage means for sequentially storing image data input from each of the image frames, a gaze direction calculation means for calculating a gaze direction in a certain range centered on a direction designated by a pointing device, and a gaze direction calculation means Calculation Corresponding pixel operation means for calculating the position of the corresponding pixel on the solid-state imaging means for each of the line-of-sight directions obtained, reading out the calculated image signal of the address of the corresponding pixel from the image storage means, and outputting the image signal. It is characterized by having.

Here, at least one of the line-of-sight direction calculating means and the corresponding pixel calculating means performs a calculating process using a look-up table.

The image processing apparatus further includes an external storage device connected between the solid-state imaging means and the image processing means. The external storage device stores image data input from the solid-state imaging means, and stores the stored image data. The image processing device may be supplied to the image processing unit in response to a read instruction signal.

[0019]

[0020]

According to the present invention, by providing the endoscope with an ultra-wide-angle lens (for example, a fish-eye lens) at the distal end of the endoscope, a viewing angle close to 180 degrees around an important viewing direction is secured. Further, a line of sight direction within a certain range is calculated around the direction instructed by the operator, the position of the corresponding pixel on the solid-state imaging means is calculated for each of the calculated line of sight directions, and the calculated corresponding pixel of the corresponding pixel is calculated. Since the image signal of the address is read and output, an image with a normal line-of-sight angle centered on the direction specified by the operator is displayed on the display screen with the distortion of the image caused by the super wide-angle lens being removed.
For this reason, according to the present invention, when inserting the endoscope, the endoscope is inserted while looking forward, or the endoscope is inserted together with a working instrument (for example, a surgical instrument). Even in the working direction, it is possible to gaze at the working direction simply by cutting out the screen without rotating the endoscope, so that work is not only accurate and quick, but also the endoscope and working equipment are placed on the wall of tissue etc. The probability of an accident such as a collision resulting in injury is reduced.

[0022]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 shows an overall schematic configuration of an embodiment of the present invention in which the present invention is applied to a rigid endoscope.
Here, reference numeral 1 denotes an entire ultra-wide-angle endoscope according to the present invention, 2 denotes an ultra-wide-angle imaging optical system disposed at the tip of the endoscope 1, and 3 denotes an optical fiber (light guide cable) for guiding illumination light. It is. Usually, the rear end of the optical fiber 3 is
A suction connector, an air / water supply terminal, a light guide plug (not shown) containing a lamp and a condenser lens, and the like are provided according to the work content.

The illumination light is transmitted from the endoscope 1 by the optical fiber 3.
And illuminates the front of the tip in a range close to 180 degrees. The operator operates the endoscope 1 while holding the main body 4. The main body 4 may be provided with an angle adjustment dial, a focus dial, and the like (both not shown) for operating the tip as required. An optical image of an object (subject) obtained by the ultra wide-angle imaging optical system 2 is a solid-state imaging camera (T) using a solid-state imaging device such as a color CCD (charge coupled device).
V camera) 5 converts it into an electric signal (image signal),
It is sent to the image processing device 6.

The image processing device 6 is a device for converting a wide-range image formed by an ultra-wide-angle lens system and converted into an image signal into an image having a normal viewing angle and displaying the image. In the image processing device 6, as described later, an image memory 61
From the wide-angle image (electronic image) close to 180 degrees stored in the super-wide-angle imaging optical system 2 with a limited range of viewing angles centered on a direction given by a pointing device such as the joystick 7. An image signal in a state where the image distortion caused by the lens has been removed is output.

An image signal output from the image processing device 6 after receiving such image processing is sent to a monitor device 8 such as a CRT-TV receiver or a liquid crystal display, and the direction specified by the operator with the joystick 7 is used. Is enlarged and displayed on the screen of the monitor device 8 substantially in real time in a state where there is no optical distortion similar to when the target is viewed with the naked eye.

FIG. 2 shows a detailed example of the internal structure of the above-mentioned ultra-wide-angle endoscope 1. As shown in FIG. 2, an ultra-wide-angle lens system 1 such as a fish-eye lens constituting the above-described ultra-wide-angle imaging optical system 2 is provided at the tip of a bendable elongated cylinder of the ultra-wide-angle endoscope 1.
1. Optical axis bending prism 12 and objective lens 13
Attach. A plurality of relay lenses 14 are arranged in the cylinder at predetermined regular intervals, and an eyepiece 15 and a solid-state imaging camera 5 are arranged behind the relay lenses 14 in this order. Further, the light guide fiber 17 is designed to be guided to the distal end of the tube of the endoscope 1 in order to illuminate an object (subject).

The object is illuminated by the light emitted from the end of the endoscope 1, and the reflected light from the object is reflected by the ultra-wide-angle lens system 11, the prism 12 for bending the optical axis, and the objective lens 1.
3. Through the relay lens 14 and the eyepiece 15, an ultra-wide-angle optical image is formed, received by the solid-state imaging device of the solid-state imaging camera 5, and converted into an electronic image. There are two known solid-state imaging cameras, a single-panel surface sequential system and a single-panel simultaneous system, depending on the type of solid-state imaging device. The present invention can be applied to either system.

FIG. 3 shows a schematic circuit configuration example of the solid-state imaging camera 5 and the image processing device 6 of FIG. The solid-state imaging camera 5 is C
A general solid-state imaging device 51 such as a CD, and a camera circuit that performs a known image processing such as A / D conversion and γ correction on an analog color image signal supplied from the solid-state imaging device 51. 52.

The image processing device 6 has an image memory 61 for sequentially storing digital output data (image data) from the camera circuit 52 for each image frame, and a fixed value centered on a direction designated by the joystick 7 as a pointing device. Gaze direction calculation circuit 6 for calculating the gaze direction of the range
2 and the position of the corresponding pixel on the solid-state imaging device 51 for each line-of-sight direction calculated by the line-of-sight direction calculation circuit 62, and stores the calculated image signal of the address of each corresponding pixel in the image memory 61. And a corresponding pixel operation circuit 63 that reads out and outputs the same.

Next, with reference to FIGS. 4 and 5, the operation principle of how the image processing device 6 restores a distorted image captured by the solid-state imaging camera 5 will be described.

FIGS. 4 and 5 show the correspondence between the photographing direction by the ultra-wide-angle imaging optical system 2 and points on the image of the solid-state imaging device 51. FIG. The z direction is defined as the optical axis direction of the ultra-wide-angle imaging optical system 2, and the xy plane including the x-axis and the y-axis is defined as the imaging surface of the solid-state imaging device 51. An arbitrary direction inclined by an angle θ in the z-axis direction from the center O of the unit sphere is defined as a w-axis, an intersection between the direction vector and the unit sphere is defined as P, and the z-axis is extended in a negative direction to define an intersection with the unit sphere. Is E, and the intersection of the line segment PE and the xy plane is Q, an image in an arbitrary direction w-axis is mapped to the point Q.

That is, it is assumed that an image in the direction in which the point P on the sphere is viewed from the center O of the sphere is captured at the point Q on the solid-state imaging device (for example, a CCD image sensor) 51 of the solid-state imaging camera 5. Then, the entire image in the w direction from the point O to the point P is in the direction of a square area on the spherical surface centered on the point P as shown in FIG. The image of this square area is projected on an area surrounded by a curve centered on the point Q on the xy plane as shown in FIG. Conversely, the distorted image projected by the fisheye lens is a curved closed area centered on Q, and the image processing device converts the image of the curved closed area into an original image of a square area centered on P. 6.

The line-of-sight direction calculation circuit 62 of the image processing device 6 fetches two values of the angle parameters θ and φ from the joystick 7 in an analog manner to understand the line-of-sight direction w. Then, the corresponding pixel operation circuit 63 outputs the image value of each pixel inside the square region centered on the w direction as follows.

First, in the line-of-sight direction calculation circuit 62, w
Θ _i (i = 1, 2,...) In the direction w _i (i = 1, 2,..., N) when the pixel inside the square region centered on the direction is viewed from the point O.
.., N). The value of n is determined to be a constant value in accordance with the resolution (the number of pixels or the number of scanning lines) of the monitor device 8. Conversely, if θ _i and φ _i are given, the w direction can be easily calculated by the following equation (1). Therefore, a corresponding look-up table may be created in advance, and the back calculation may be obtained from this table. w _i direction the direction cosine (u
_i, v _i, if indicated by the w _i), is calculated by the following equation (1).

[0036]

U _i = sin θ _i cos φ _i v _i = sin θ _i sin φ _i (1) w _i = cos θ _i Next, in the corresponding pixel operation circuit 63, θ _i and φ _i are substituted into the following equation (2). By doing so, the position (address) of the corresponding pixel (x _i , y _i ) on the screen of the solid-state imaging device 51
Is obtained, the image value of the pixel is read out from the image memory 61, and is output from the image processing device 6. Note that a look-up table can be used for the corresponding pixel operation circuit 63 as well.

[0037]

X _i = cos φ _i · sin θ _i / (1 + cos θ _i ) (2) y _i = sin φ _i · sin θ _i / (1 + cos θ _i ) The corresponding pixel operation circuit 63 results in interpolation of image data. Is performed, the image distortion caused by the ultra wide angle lens is removed. Moreover, as described above, the image area displayed on the monitor device 8 is a part of the image captured by the ultra-wide-angle imaging optical system 2, and the center direction of the displayed image can be specified by the joystick 7, This has an excellent advantage that the direction desired by the operator can be selectively displayed.

The image memory 6 is operated by the corresponding pixel operation circuit 63.
The image signal read from 1 is sent to the monitor device 8,
An image with a normal viewing angle in the direction designated by the joystick 7 is displayed.

The image processing device 6 can be constructed using a known microprocessor or an electronic circuit such as a ROM or a RAM, or a commercially available personal computer can be used.

As described above, according to the present invention, in order to realize an omnidirectional endoscope, the ultra wide angle lens 11 is attached to the end of the endoscope 1. As is well known, an image obtained from an ultra-wide-angle lens such as a fisheye lens is largely distorted, particularly in a peripheral region of the image. That is, as the image moves from the center to the periphery of the image, the image rapidly shrinks, and the object appears to be curved.
Therefore, even if this image is output and displayed as it is on a normal image display device, the original shape can hardly be identified. Therefore, in this example, the distortion of the optical image projected two-dimensionally onto the solid-state imaging device 51 in the solid-state imaging camera 5 of FIG. And the image is returned to the original shape.

Further, the viewing angle of the ultra wide-angle lens 11 is close to 180 degrees around the important viewing direction and covers a very wide range at a time.
Displaying all of the images received in step 1 without distortion on an ordinary image display device at the same time is not only technically impossible but also from the viewpoint of display space and the like. Even if it is displayed by using, the image becomes extremely small and cannot be put to practical use. Therefore, in the present invention,
An image is extracted within a certain range at a normal viewing angle centered on the direction specified by the joystick 7 by the operator, and the observation direction at the time of insertion of the endoscope 1 and the observation direction at the time of work are electronically divided for observation. This makes it possible to freely observe many directions with one endoscope without any mechanically movable parts.

Further, according to the operator's selection, the image signal output from the camera circuit 52 is sent to the image memory 61 and simultaneously output to the external storage device 22 for storage.
Thereafter, the image signal read from the external storage device 22 is preferably output to the image memory 61 and displayed on a monitor. In this case, the same image signal can be repeatedly read out, and images in various arbitrary directions can be freely cut out and displayed with the joystick 7 many times. Thus, the patient's burden can be reduced, and the accuracy of the diagnostic test can be expected to be improved.

When displaying the extracted image on the monitor device 8, the entire wide-angle image read out from the image memory 61 is reduced to a reduced image at the corner position of the display screen using a known image synthesizing technique. It is preferable that the cutout image is displayed and the cutout position of the cutout image is clearly indicated by a frame line. In this case, since the entire target object (affected part) can be recognized and the image cutout position can be accurately grasped, the image cutout position can be easily selected, and improvement in work efficiency can be expected.

(Other Embodiments) Although the preferred embodiment of the present invention has been described, the present invention is not limited to this.

For example, FIG. 2 shows an example in which the present invention is applied to a rigid endoscope. However, the present invention is similarly applicable to a flexible endoscope using an optical fiber.

Further, a structure may be employed in which a solid-state imaging device is disposed at the distal end of an insertion tube of an endoscope, and a light image passing through an ultra-wide-angle lens is received by the solid-state imaging device and converted into an image signal.

Although an ultra-wide-angle lens having a viewing angle close to 180 degrees has been exemplified, an endoscope using a wide-angle lens system having a viewing angle smaller than this is also applicable to the present invention.

The illumination light is not limited to visible light, and infrared light is also applicable to the present invention.

[0049]

As described above, according to the present invention, by equipping the end portion of the endoscope with the ultra-wide-angle lens, a viewing angle close to 180 degrees around the important viewing direction can be secured. In addition, the image was cut out at a normal viewing angle around the direction instructed by the operator, and the image distortion caused by the super wide-angle lens was removed by image processing before being displayed on the monitor screen. When inserting an endoscope from a narrow place and working in a specific direction at a specific place in the back, when inserting the endoscope, check the insertion direction in front of the endoscope, or You can insert the endoscope and work equipment, and when you arrive at a specific place, you can work while watching the different work direction different from the insertion direction simply by cutting out the screen without rotating the endoscope. Work is accurate Not only becomes faster, an excellent effect that it is possible to prevent trouble during insertion and task of collision endoscope and working instruments such as the wall surface is obtained.

[Brief description of the drawings]

FIG. 1 is a schematic diagram showing an overall configuration of an ultra-wide-angle endoscope according to an embodiment of the present invention.

FIG. 2 is a partial longitudinal sectional view showing an enlarged example of the internal structure of the ultra-wide-angle endoscope (perspective mirror) according to the embodiment of the present invention.

FIG. 3 is a block diagram illustrating an example of a configuration of an image processing apparatus to which the present invention has been applied.

FIG. 4 is a schematic diagram illustrating a correspondence relationship between a pixel region of a distorted image obtained by an ultra-wide-angle imaging system and a corresponding image region in a shooting direction.

FIG. 5 is a schematic diagram showing the angle relationship of FIG. 4;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Ultra-wide-angle endoscope 2 Ultra-wide-angle imaging optical system 3 Optical fiber 4 Body part 5 Solid-state imaging camera 6 Image processing device 7 Joystick 8 Monitoring device 11 Ultra-wide-angle lens system (ultra-wide-angle lens) 12 Prism for bending optical axis 13 Objective lens 14 Relay lens 15 Eyepiece 17 Light guide fiber 22 External storage device 51 Solid-state imaging device 52 Camera circuit 61 Image memory 62 Viewing direction calculation circuit 63 Corresponding pixel calculation circuit

Continued on the front page (72) Inventor Koji Yamauchi 1-1-3 Higashi, Tsukuba City, Ibaraki Pref. Industrial Technology Research Institute Inside Biotechnology Research Institute (72) Inventor Juri Yamashita 1-3-1 Higashi, Tsukuba City, Ibaraki Pref. In Biotechnology Research Institute (72) Inventor Kazunori Yokoyama 11-7 Manabe Shinmachi, Tsuchiura City, Ibaraki Prefectural General Hospital Tsuchiura Kyodo Hospital Examiner Yuji Mine (56) References JP-A-57-78834 (JP, A) JP-A-1-136625 (JP, A) JP-A-9-56662 (JP, A) JP-A-2-244021 (JP, A) JP-A-8-332169 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) G02B 23/24 A61B 1/00 300 A61B 1/04 372 G02B 23/26

Claims (4)

(57) [Claims] 1. An endoscope having an imaging optical system at the tip of a tube using an optical fiber or a relay lens system, comprising: an ultra-wide-angle lens system provided at the tip of the tube as the imaging optical system. A solid-state imaging unit that receives a wide-range optical image captured by the ultra-wide-angle lens system and converts the optical image into an image signal; and a wide-angle image obtained from the image signal of the solid-state imaging unit is an image having a normal viewing angle. Image processing means for performing image processing for converting the image data into image data; and image display means for displaying an image based on an output signal of the image processing means. The image processing means converts image data input from the solid-state imaging means into an image. Image storage means for sequentially storing each frame, line-of-sight direction calculating means for calculating a line-of-sight direction within a certain range centered on the direction specified by the pointing device, and a line-of-sight direction calculating means. Corresponding pixel calculation means for calculating the position of the corresponding pixel on the solid-state imaging means for each of the line-of-sight directions calculated, reading out the calculated image signal of the address of the corresponding pixel from the image storage means, and outputting the read image signal; An ultra-wide-angle endoscope comprising: 2. An endoscope having an imaging optical system at a distal end of a tube, comprising: an ultra-wide-angle lens system provided at the distal end of the tube as the imaging optical system; A solid-state imaging means for receiving and converting a wide-range optical image captured by the ultra-wide-angle lens system into an image signal; and a wide-angle image obtained from the image signal of the solid-state imaging means at a normal viewing angle. Image processing means for performing image processing for converting the image into an image, and image display means for displaying a video in accordance with an output signal of the image processing means. The image processing means includes image data input from the solid-state imaging means. Image storage means for sequentially storing image data for each image frame; gaze direction calculation means for calculating a gaze direction within a certain range centered on a direction designated by a pointing device; and gaze direction calculation means. Corresponding pixel operation means for calculating the position of the corresponding pixel on the solid-state imaging means with respect to the line of sight, reading out the calculated image signal of the address of the corresponding pixel from the image storage means, and outputting the read out image signal. An ultra-wide-angle endoscope, characterized in that: 3. The ultra-wide-angle endoscope according to claim 1, wherein at least one of the line-of-sight direction calculation unit and the corresponding pixel calculation unit performs a calculation process using a lookup table. 4. An external storage device connected between the solid-state imaging means and the image processing means, wherein the external storage device stores image data input from the solid-state imaging means,
The ultra-wide-angle endoscope according to any one of claims 1 to 3, wherein the stored image data is supplied to the image processing means in response to a read instruction signal.