JP2006271503A - Endoscopic apparatus with three-dimensional measuring function - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce errors in quantization and to improve the distance resolution even if the base line distance is short in the three-dimensional measurement for triangulation by applying spot light from a known position to a subject and receiving the reflection light by a camera. <P>SOLUTION: At least two values of measurement are obtained in one measurement by moving a single lens, or moving the lens and an imaging element together, or moving the imaging face parallel to the original plane while the lens of the camera is fixed, and errors in quantization are reduced by the statistical process of these obtained values. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a three-dimensional measurement technique in an endoscope.

In three-dimensional measurement, various methods have been proposed as active methods. Here, a method of projecting a light pattern will be described.
(1) Method of projecting a stripe pattern (see Patent Document 1)
A parallel light beam is projected as multi-slit light, and the distance to the object is known from the known slit width and the angle of view obtained by imaging the projected stripe.
(2) Method of irradiating spot light (see Patent Document 2)
The spot light is sequentially irradiated while changing the position, and the distance to the object is known from the known optical axis of the spot light and the captured irradiation position information by the principle of triangulation.
JP 2002-131016 A JP-A-5-052531

In the prior art, it is possible to reduce the size by using a CCD or CMOS as an image pickup device. However, since these devices are discretely arranged, a quantization error occurs, and it has been difficult to reduce this error until now. there were.
An object of the present invention is to reduce quantization error and improve distance resolution despite a short baseline distance.
First, the principle of measurement will be described.
A case where dot pattern light emission is used will be described as an example. Fig.1 (a) is a conceptual diagram which shows arrangement | positioning of a light projector (1), a camera (2), the imaging lens (3) of a camera, the image pick-up element (4) of a camera, and a target object (5).
Only one dot is irradiated from the projector (1), and its position is captured by the camera (2). Reflected light from the object (5) that has passed through the opening of the imaging lens (3) forms an image on the imaging element (4), and respective vectors OA and OB are obtained. The three-dimensional position of one point (O) where the object (5) is located is obtained from the distance AB (baseline distance) between the exit hole (A) of the known projector and the lens center (B) of the camera and the intersection of the vectors. It is done.

Also in the case of line pattern projection, the three-dimensional position is obtained from the intersection of the plane on which the light source exists and the vector acquired by the camera. By using in combination with the pattern matching technique, it is possible to obtain a three-dimensional position even for a plurality of line patterns, that is, a stripe pattern or a lattice pattern. The same applies to a plurality of dots.
Since photoelectric conversion is performed by an image sensor using a CCD or CMOS device, the value of the angle ABO cannot be a continuous value due to the discrete arrangement of the elements.
In FIG. 1B, since the original point (O) of the object (5) is imaged at the point P1 of the image sensor, the distance to the object is AO. When the point (O ′) forms an image at the point P2 of the image sensor because the object (5 ′) is approaching, it is estimated that the current position is at the point (O ′) by the principle of triangulation. That is, unless the distance between the points P1 and P2 is subdivided on the image sensor, the distance next to the distance OA is the distance O′A, and the distance between these cannot be measured.

  By moving the camera and / or the pattern projector, the measurement accuracy can be improved even if the baseline distance is small.

In a camera that is required to be downsized, such as an endoscope, an optical-electrical conversion is performed by an image pickup device using a CCD or a CMOS device, so that a quantization error occurs due to the discrete arrangement of the device.
In order to reduce this error, it is only necessary to increase the number of pixels of the image sensor, but the number of pixels cannot be increased infinitely. Therefore, a structure is proposed in which the quantization error can be reduced as if the number of pixels is substantially increased.
The basic concept of problem solving is as follows.
(1) Movement of the camera (or pattern projector) in the endoscope radial direction (2) Movement of the camera (or pattern projector) in the longitudinal direction of the endoscope (3) of the camera and pattern projector Move

FIG. 2 shows a configuration for moving the image sensor that is the imaging surface. In FIG. 2, the distal end of the insertion portion (20) of the endoscope is shown, and a light guide (6) and a light projecting lens (7) are provided as light projectors. A multi-beam spot (dot) light is projected from the light projecting lens (7) and irradiates the object (5). The light reflected from the object (5) forms an image on the image pickup device (4) by CCD or CMOS by the image pickup lens (3). The imaging device (4) is held in a floating state, and can be moved by a minute distance by being coupled to one end of the actuator (8).
The other end of the actuator (8) is fixed at an appropriate location. The image signal from the image sensor is connected to the signal processing circuit (10) via the signal line (9). The signal processing circuit adjusts distance calculation, image contrast, and the like as necessary.
When it is difficult to observe visually with dot-shaped illumination, the entire illumination and dot illumination can be used alternately in a time-division manner, or the distance of light can be varied and infrared light can be measured. It should also be considered to configure the observation with visible light.

To explain the concept, a camera is placed in a first position and an object is imaged. The first position is the position of the lens (L2) in FIG. Next, the camera is moved in the endoscope radial direction, and the object is re-imaged at the second position. The second position is the position of the lens (L3) in FIG. By this movement, the object (11) is imaged first as an image (12) and then as an image (13).
FIG. 3A shows the case where the lens L2 is moved to the left position L3. When the lens is moved by a minute distance in the left-right direction, the imaging position of the object changes, and the imaging moves from the image (12) to the image (13). In this case, the image sensor may be moved in the same direction by the same distance as the lens movement, or may not be moved. The same effect can be obtained even if only the image sensor is moved in the right direction.
As a result, the image forming position of the pattern is different, and two distances can be obtained at the same point. Simply, average these values. Furthermore, using the fact that the head of the endoscope is moving, the average movement amount between the previous image frame and the current image frame is obtained and corrected by this. Although the lens alone or the lens and the image sensor may be moved, in practice, it is desirable that the imaging surface be moved parallel to the original plane while the camera lens is fixed. In any case, the accuracy is improved if the imaging position is different on the image sensor. If the image forming position is shifted by 1/2 pitch in the pixels of the image sensor, the accuracy is approximately doubled.

Furthermore, if the imaging is performed three times while shifting by 1/3 pitch, the accuracy is further improved.
If it is expanded, it is possible to shift the pitch by k / n (k = 0, 1,..., N−1, where n is an integer of 1 or more).
The movement here may be physically performed by an actuator or the like, but may also be performed by dividing incident light by a half mirror and imaging by two or more imaging elements. According to the actuator, images can be obtained at various movement positions such as a 1/2 pixel shift, a 1/3, and a 2/3 pixel shift. According to the half mirror, there are advantages and disadvantages depending on the configuration, such as no moving parts, so it is selected depending on the situation. The structure in the case of using the half mirror (14) is shown in FIG. FIG. 4A shows an example of shifting in the left-right direction.
Up to this point, an example of single-dot light has been shown, but it can also be applied to scanning by sequential lighting, and even if it is a multi-dot that is always lit, if a pattern matching method is adopted, a single dot light can be used. It is clear that the distance meter can do the same as light.

In the previous embodiment, the imaging surface is moved in parallel with the original plane. However, as another embodiment, the imaging surface is moved in a direction perpendicular to the original plane, that is, in the front-rear direction of the endoscope.
FIG. 3B is a conceptual diagram thereof. When the object (11) is imaged with the lens (L2) at the first position, the object (11) is imaged at the position and size of the image (12).
Next, the lens is moved. When the object (11) is imaged with the lens (L3) at the second position, the object (11) is imaged at the position and size of the image (13).
FIG. 3B shows a case where the lens (L2) is moved to the rear position L3.
When the lens is moved a minute distance in the front-rear direction, the imaging position and size of the object change, and the imaging moves from the image (12) to the image (13). In this case, the image sensor is moved in the same direction by the same distance as the lens movement. However, if the focus shift is not a problem, the image sensor does not have to be moved.
In this case, since the imaging position does not change in a part of the pattern, there is a region where the quantization error cannot be partially reduced.
In the back-and-forth movement, instead of physical movement, incident light splitting by the half mirror (14) can be used. In this case, the problem of focusing occurs, but it can be solved by providing a correction lens (L4) in one image sensor.
FIG. 4 shows a structure in the case of using a half mirror (14), and FIGS. 4 (b) and 4 (c) are examples of shifting in the front-rear direction. In FIG. 4B, a correction lens (L4) is inserted for focal length adjustment. In this case, since the focal length differs for each image sensor, strictly speaking, although the camera is not shifted in the front-rear direction, it is possible to reduce the size. The structure shown in FIG. 4C is obtained by shifting the entire camera in the front-rear direction, but is slightly larger in structure. Although the total reflection mirror (15) is used to change the optical path, a structure can be omitted.

Up to this point, it has been described that the camera (including the movement of a single lens or only an image sensor) is moved, but the quantization error can be reduced by moving the pattern projector instead of the camera as well as the movement of the camera. Can understand.
Further, it will be understood that the quantization error can be similarly reduced by moving both the camera and the pattern projector.
Further, since an error can be reduced if an image can be formed and measured on different pixels on the image sensor, other forms such as tilting the camera, tilting the lens or the image sensor, tilting the projector, etc. are conceivable.

The distance can be obtained by moving the lens in the front-rear direction as described above. Although there are correction factors such as the influence of changes in the imaging position, an outline will be described.
As shown in FIG. 3 (b), if the size (or angle of view) of the object decreases by y% when the entire camera is moved backward by xcm,
Object size × focal length = image (12) size × first distance (D12) ----- (1)
Object size × focal length = image (13) size × movement distance (D13) ----- (2)
Distance after movement (D13) -first distance (D12) = x ----- (3)
Size of image (13) / Size of image (12) = (100-y) / 100 ----- (4)
The initial distance from the four equations to the object = x · (100−y) / y
It becomes.
The distance obtained in this way also includes the influence of the quantization error, but it can be used as a supplement to the triangulation according to Embodiments 1 to 3 or can be measured independently. When a half mirror is used, the structure shown in FIG.

  Although the method for reducing the quantization error described here can be widely used, the actuator used to move the camera (imaging device) is small when applied to a field where the baseline cannot be made large, that is, an endoscope. Piezoelectric elements such as piezo elements can be used, and downsizing is possible.

The figure which shows the principle of distance measurement. (A) is an apparatus conceptual diagram, (b) is a figure explaining a quantization error. The figure which shows the insertion part front-end | tip of an endoscope. The figure which shows the camera movement for reducing a quantization error. (A) shows the movement to the left-right direction, (b) has shown the movement to the front-back direction. The figure which shows arrangement | positioning of the image pick-up element by a half mirror. (A) shows the arrangement in the left-right direction, and (b) and (c) show the arrangement in the front-rear direction.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Light projector 2 Camera 3 Lens opening part 4 Image pick-up element 5, 11 Object 12 Image by lens (L2) 13 Image by lens (L3)

Claims (2)

A projector (1) that irradiates an object with a light pattern for measuring a distance, an imaging lens (3) that forms an image of reflected light from the object on an image sensor, and a light of the image formed. An endoscope apparatus with a three-dimensional measurement function, comprising an imaging device (4) for electrical conversion and an actuator (8) for moving the imaging device and / or lens by a minute distance. A projector (1) that irradiates an object with a light pattern for measuring a distance, an imaging lens (3) that forms an image of reflected light from the object on an image sensor, and a light of the image formed. -Two or more image sensors (4) to be electrically converted, and the image on the image sensor are shifted by 2 k / n pitches of pixels (where k is an integer from 1 to n-1 and n is an integer of 2 or more). An endoscope apparatus with a three-dimensional measurement function, comprising a half mirror (14) that forms an image on two or more image sensors.

Images