JP3088852B2 - 3D image input device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional image input apparatus which extends the range of distance detection and secures the resolution of a gray image.

[0002]

2. Description of the Related Art Conventionally, techniques in such a field include:
For example, there are those described in the following documents. Literature 1: Journal of the Institute of Television Engineers of Japan, 45 [4] (1991)
P. 446-452 Reference 2; Journal of the Institute of Television Engineers of Japan, 45 [4] (1991)
P. 453-460 Conventionally, the three-dimensional image input method includes a passive method (passive method) and an active method (active method). The active method is to irradiate the target with energy (light wave, radio wave, sound wave) which is skillfully controlled to acquire three-dimensional information and has some meaning to its shape pattern, shading, spectrum, etc. Refers to the method. On the other hand, the passive method means that even if normal lighting is performed on the object,
This refers to measurement that does not use meaningful energy for measurement. Generally speaking, the active method has higher measurement reliability than the passive method. A typical passive method is a stereo image method, which is shown in FIG.

[0003] FIG.
FIG. 2 is an explanatory diagram of a stereo image method which is one of the two-dimensional image input methods. In this stereo image method, two cameras 1 and 2, which are two-dimensional image input devices, are arranged at a predetermined distance from each other, and the difference between the imaging positions of the subject 3 taken by the left and right cameras 1, 2 is calculated.
That is, the distance to the subject 3 is measured by a triangulation method using the phase difference.

FIG. 3 is an explanatory diagram of two images, that is, a gray image and a distance image of a signal obtained by the stereo image method of FIG. The grayscale image is a color or monochrome image obtained by the cameras 1 and 2 in FIG. The distance image is an image relating to a three-dimensional position, and each pixel has information relating to the depth of the target (subject 3) in the matrix data. From such a grayscale image and a distance image, a stereoscopic image display is performed by a binocular fusion method using a polarizing filter, or a stereoscopic image display is performed by using a lenticular plate. FIG. 4 shows an example of a stereoscopic image display.

[0005] FIG.
FIG. 2 is a principle diagram of a multi-view lenticular system which is one of the two-dimensional image display systems. The multi-view lenticular method is a method in which a lenticular plate 10 composed of a plurality of lens-shaped lens plates is used, and left and right images are arranged in a stripe pattern on the focal plane of each lens plate. In one lens plate, a, b, c,
..., the portion of the f, respectively _{a 1, b 1, c 1} , ..., f 1
Is displayed as a striped multi-view image 11 taken from multiple directions. By the action of the lens plate, the striped multi-view image 11 in each direction enters the left and right eyes 12 and 13 separately, and if the viewpoint is moved, a stereoscopic image in the horizontal direction can be viewed.

[0006]

However, since the distance is inversely proportional to the detected phase difference, the accuracy of the distance detection is reduced in inverse proportion to the square of the distance in the apparatus having the above-described configuration. There is a problem that the distance range obtained is narrow. In addition, when imaging is performed under dark lighting conditions, imaging is performed with a small F-number of the lens. However, in short-distance imaging, a range in which a high resolution can be obtained becomes narrow, and a subject out of the range becomes unclear. There were problems and it was difficult to solve them. SUMMARY OF THE INVENTION The present invention solves the problems of the prior art in that a distance detection range is narrow and a distance range in which a clear image can be obtained is narrow. An apparatus is provided.

[0007]

According to a first aspect of the present invention, there is provided a three-dimensional image input device for inputting an image of a subject and outputting a grayscale image and a distance image.
First multi-eye two-dimensional image that outputs a gray-scale image and a distance image in which a first distance detection setting with a first distance accuracy is performed so as to reduce the first object distance by focusing on the object distance An input device and a second object focusing on a second object distance greater than the first object distance and incorporating a second distance accuracy
And a second multi-eye two-dimensional image input device that outputs a grayscale image and a distance image for which the distance detection is set. Then, the second distance detection setting is performed so that the distance range that is the lower limit value of the first distance accuracy and the distance range that is the upper limit value of the second distance accuracy are overlapped. At least one optical axis of the multi-eye two-dimensional image input device is aligned with at least one optical axis of the second multi-eye two-dimensional image input device so that they have the same line of sight.

[0008] In the second invention, each of the plurality of eyes 2 of the first invention is used.
Determine the role of the imaging distance range of the three-dimensional image input device,
The composition is such that a grayscale image is selected based on each of the distance images and then combined.

[0009]

According to the first aspect of the present invention, since the three-dimensional image input device is configured as described above, the first multi-eye two-dimensional image input device inputs a subject image and outputs a grayscale image and a distance image. I do. Similarly, the second multi-eye two-dimensional image input device inputs the image of the subject and outputs a grayscale image and a distance image. At this time, at least one optical axis of the first multiple-eye two-dimensional image input device and at least one optical axis of the second multiple-eye two-dimensional image input device have the same line of sight.
The distance detection range becomes wider. According to the second invention, based on the distance images output from the first and second multi-eye two-dimensional image input devices, necessary grayscale images are selected and then synthesized,
The signals of the final grayscale image and the distance image are output. Thereby, a clear image can be obtained. Therefore, the above problem can be solved.

[0010]

1 (a) to 1 (c) are block diagrams showing the configuration of a three-dimensional image input apparatus showing an embodiment of the present invention. FIG. 1 (a) is an overall configuration diagram, and FIG. FIG. 3C is a front view of the image input unit, and FIG. The three-dimensional image input device includes an image input unit 20 that inputs an image of a subject and outputs a grayscale image and a distance image. The image input means 20 includes a first multi-eye two-dimensional image input device 21 focused on a first object distance, and a second multi-eye two-dimensional image input device 22 focused on a second object distance. And an optical member including a half mirror 23 and a mirror 24 for forming an optical path. The first multiple-eye two-dimensional image input device 21 includes a first two-dimensional image input device 21a having an optical axis (line of sight) H1 and a second two-dimensional image input device 21b having an optical axis H2. , they are arranged in a narrow optical axis distance L _1. The second multi-eye two-dimensional image input device 22 includes a half mirror 23 and a mirror 2
4, a first two-dimensional image input device 22a adjusted to have the same optical axis H1 as the first two-dimensional image input device 21a, and a second two-dimensional image input device 2 having an optical axis H3
2b, and they are arranged at a wide optical axis interval L ₂ ,
Further, an optical axis interval _Lc is provided between the first two-dimensional image input device 22a and the first two-dimensional image input device 21a. Optical axis H1 of two-dimensional image input devices 21a and 22a
The optical axis H2 of the two-dimensional image input device 21b and the optical axis H3 of the two-dimensional image input device 22b form a horizontal line of sight. Each of these two-dimensional image input devices 21
Reference numerals a, 21b, 22a, and 22b each include a monocular lens and an image sensor for photoelectric conversion.

A density image S21a and a distance image S21b output from the first multi-eye two-dimensional image input device 21,
The density image S22a and the distance image S22b output from the second multiple-eye two-dimensional image input device 22 are respectively stored in a semiconductor storage device or the like, and one of the grayscale image S21a and the distance image S21b is stored in a first predetermined distance range. Selection device 31
And the other grayscale image S22a and the distance image S22b are sent to the second predetermined distance range selection device 32. The first predetermined distance range selecting device 31 outputs the distance image S2
This is an apparatus that removes a portion of a gray image S21a corresponding to a distance image of a certain distance or more using the distance value of 1b. Second
The predetermined distance range selection device 32 is a device that uses the distance value of the distance image S22b to remove a portion of the grayscale image S22a corresponding to a distance image equal to or less than a certain distance. The output side of the first and second predetermined distance range selecting devices 31 and 32 is connected to a perspective image synthesizing device 40. Perspective image synthesis device 4
0 is the first and second predetermined distance range selection devices 31 and 32
Is a device that synthesizes one image from the output of the above, outputs signals of the final grayscale image S41 and the final distance image S42, and stores them in a semiconductor storage device or the like.

FIG. 5 is an explanatory diagram of the angle of view, the setting of the object distance, the line of sight, and the like in the image input means 20 of FIG. FIG.
Optical axis distance L _c between the two-dimensional image input device 21a and 22a of the
Is required to position the first multi-eye two-dimensional image input device 21 rearward with respect to the light incident direction with respect to 22, so that the lenses constituting the two-dimensional image input devices 21a, 21b, 22a, 22b Principal points 0 _21a , 0 _21b , 0 _22a , 0 _22b
Is on a straight line. Each principal point 0 _{21 a} , 0 _21b ,
0 _22a, 0 viewed from _22b angle _{φ 21a, φ 21b, φ 22a} ,
Of phi _22b, phi _21a and phi _22a is are approximately equal.
First and second two-dimensional image input device 21a, 21b is Yes by focusing on an object distance l _1, further first and second two-dimensional image input device 22a, 22b is focusing on the object distance l ₂ Has been done. In general, when a plurality of two-dimensional image input devices (for example, 21a and 21b, 22a and 22b) are separated from each other by certain optical axis intervals L ₁ and L ₂ to image a subject,
The position on the screen where the subject appears is shifted. This shift amount is called a phase difference. The distance is the separation distance L of the two-dimensional image input device.
It is detected in a form proportional to ₁ or L ₂ and inversely proportional to the phase difference. On the other hand, the sharpness of the grayscale image of the subject obtained by the two-dimensional image input device strongly depends on the characteristics of the lens when the number of pixels of the two-dimensional image input device is fixed. In particular,
When the lens is brightened by reducing the F-number or when the image is taken at a short distance, the image is likely to be out of focus and blurred.

FIG. 6 is a diagram showing a resolution characteristic and a phase difference characteristic with respect to the distance of the three-dimensional image input device of FIG. The lenses of each of the two-dimensional image input devices 21a, 21b, 22a, and 22b have a focal length of 16 mm, a brightness of F number 4, and an image sensor has 600 pixels and ／ inch (photosensitive surface of about 6.6 × 8.
8mm) device. The subject pattern is about 1.5
This is a pattern having a thickness corresponding to ２2 pixels / line. Angle of view 0 _21a = 0 of first multi-eye two-dimensional image input device 21
_21b is about 30 °, the optical axis interval L ₁ is about 3.5 cm, and the object distance l ₁ is about 0.6 m. The angle of view φ _22a = φ _{22b of} the second multi-eye two-dimensional image input device 22 is also about 30 °, and the optical axis interval L
₂ is about 8 cm, and the object distance l ₂ is about 1 m.

FIG. 6 shows the distance (m) on the horizontal axis and the resolution MTF (Modulation Transfer Function) on the left vertical axis.
(%), And the phase difference (bit) is plotted on the right vertical axis. Among the curves 51 to 54, the curve 51 is a resolution curve of the first multi-eye two-dimensional image input device 21, and the curve 52 is a distance-phase difference curve. In the curves 51 and 52,
The angle of view is 30 °, the optical axis interval is 3.5 cm, and the object distance is 0.6 m.

A curve 53 is a resolution curve of the second multi-eye two-dimensional image input device 22, and a curve 54 is a distance-phase difference curve thereof. In the curves 53 and 54, the angle of view is 30 °, the optical axis interval is 8 cm, and the object distance is 1 m. A straight line 55 indicates a limit value (upper limit value) of the upper distance accuracy. The upper value is a condition for capturing an image so that the image does not jump out of the screen. In this example, the distance accuracy is about 0.3 cm at the intersection of the curve 52 and the straight line 55, and about 0.45 cm at the intersection of the curve 54 and the straight line 55. A straight line 56 is a lower distance accuracy lower limit value, and a curve 52
At the intersection of the straight line 56 and the intersection of the curve 54 and the straight line 56, the distance accuracy is set to about 2 cm. ΔL in FIG.
Indicates that the intersection of the curve 52 and the straight line 56 and the intersection of the curve 54 and the straight line 55 are covered by a distance of ΔL.

Next, the operation of the apparatus shown in FIG. 1 will be described with reference to FIG. Since the conventional three-dimensional image input device outputs only the curves 51 and 52 in FIG.
The range in which F could be more than a desired value (in this example, 70% or more at the time of focusing) was obtained only in a narrow range of about 45 to 75 cm. Also, the distance range was only as narrow as about 45 to 110 cm. In contrast, in the present embodiment,
The range can be greatly expanded to about 45 to 200 cm in resolution and about 45 to 180 cm in the distance range. In addition, since a fog is provided in the detection area in the distance direction of ΔL, a distance image within a certain distance accuracy can be reliably obtained.

The grayscale image corresponding to the curve 51 in FIG.
The distance image corresponding to the curve 52 is the distance image S21b of FIG. 1, the gray image corresponding to the curve 53 is the gray image S22a of FIG. 1, and the distance image corresponding to the curve 54 of FIG. 1 is the distance image S22b. FIG.
(A)-(c) is explanatory drawing which converts the blurred image of FIG. 1 into a clear image. 61 is a clear image, 61a is a clear image (close), 62 is a blurred image, 62a is a blurred image (far)
It is. FIGS. 7A and 7B show an example in which each of the images is displayed on a display device such as a CRT. FIG. 7 (a),
In (b), clear images 61 and 61a and unclear (blurred) images 62 and 62a are displayed according to the distance to the subject. The blurred image 62a is improved in inverse proportion to the F-number by increasing the F-number of the lens (that is, by making the lens darker). In this case, if the illumination light is not increased by that amount, the same result is obtained. There is a disadvantage that the output voltage of the solid-state imaging device cannot be obtained. On the other hand, there is a method of removing the blurred image 62a using the contrast, but it is very difficult for an image whose contrast changes gradually. Therefore, in the present embodiment, the first and second predetermined distance range selecting devices 31 and 32 and the perspective image synthesizing device 40 of FIG. 1 are used to extend the unclear distance region of the image and secure the distance accuracy. I have to. Hereinafter, the operation of FIG. 1 will be described.

The image input means 20 inputs an image of a subject, outputs gray-scale images S21a and S22a from the two-dimensional image input devices 21a and 21b, and outputs distance images S21b and S22b from the two-dimensional image input devices 21b and 22b.
Is output and stored in a semiconductor memory device or the like. The first predetermined distance range selection device 31 uses the distance value of the distance image S21b to remove the portion of the grayscale image S21a corresponding to the distance image of a certain distance (for example, about 75 cm) or more, and Send to Second predetermined distance range selection device 3
2, a distance image S2 smaller than a certain distance (for example, 73 cm)
The portion of the gray image S22a corresponding to 2b is removed and sent to the perspective image synthesizing device 40. In the perspective image synthesis device 40,
The image from which the shaded image portion has been removed is synthesized into one image. Since the optical axis and the angle of view of the perspective image synthesizing device 40 are the same, it is possible to rewrite a simple semiconductor memory or the like. In some cases, the removal areas do not completely match (overlap or shortage), but in that case, the grayscale image to be inserted later is preferentially inserted. In this way, the perspective image synthesizing device 40 uses the final grayscale image S41.
And the final distance image S42. When the final gradation image S41 and the final distance image S42 are displayed on a display device such as a CRT, the blurred image 62a is displayed as shown in FIG.
Is converted to a clear image and displayed. At this time, since there is the final distance image S42 having the desired distance accuracy, the image can be accurately reproduced by stereoscopically viewing the image with the binocular fusion method or the like.

The present invention is not limited to the above embodiment,
Various modifications are possible. For example, there are the following modifications. (I) In the above embodiment, two object distances l ₁ and l ₂ have been described. However, if the number of optical members of the half mirror 23 and the mirror 24 is increased, two or more object distances can be set. (Ii) The half mirror 23 and the mirror 24 for forming the optical path shown in FIG. 1 may be constituted by other optical members. (iii) It is also possible to arrange a plurality of the three-dimensional image input devices shown in FIG.

[0020]

As described in detail above, according to the first aspect, the first and second plural-eye two-dimensional image input devices are provided, and at least the first plural-eye two-dimensional image input device is provided. Since one optical axis and at least one optical axis of the second multi-eye two-dimensional image input device are aligned and have the same field of view, the distance detection range is widened. According to the second invention, the role of the imaging distance range of the first and second multi-eye two-dimensional image input devices is determined, and the required grayscale image is selected and synthesized based on the distance image. And a clear image can be obtained. Accordingly, over a wide range of distances, the 3
A two-dimensional image input device can be realized.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a three-dimensional image input device according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram of a stereo image method which is one of the conventional three-dimensional image input methods.

FIG. 3 is an explanatory diagram of a grayscale image and a distance image obtained by the stereoscopic image method of FIG. 2;

FIG. 4 is a principle diagram of a multi-lens lenticular system which is one of the conventional three-dimensional image display systems.

FIG. 5 is an explanatory diagram of an angle of view, an object distance setting, a line of sight, and the like in FIG. 1;

FIG. 6 is a diagram showing a resolution characteristic and a phase difference characteristic with respect to the distance in FIG. 1;

FIG. 7 is an explanatory diagram for converting the blurred image of FIG. 1 into a clear image.

[Explanation of symbols]

Reference Signs List 20 image input means 21, 22 first and second multi-eye two-dimensional image input devices 21a, 22a first two-dimensional image input device 21b, 22b second two-dimensional image input device 23 half mirror 24 mirror 31, 32 First and second predetermined distance range selecting device 40 Perspective image synthesizing device S21a, S22a Gray image S21b, S22b Distance image S41 Final gray image S42 Final distance image

Claims (2)

(57) [Claims] A first object distance focusing device that focuses on a first object distance and includes a first distance accuracy to reduce the first object distance;
A first multi-eye two-dimensional image input device that outputs a grayscale image and a distance image for which the distance detection setting has been performed, and a second object distance that is longer than the first object distance. And a second multi-eye two-dimensional image input device that outputs a grayscale image and a distance image in which a second distance detection setting is inserted, and a distance range that is a lower limit value of the first distance accuracy; The second distance detection setting is performed such that a distance range that is an upper limit value of the distance accuracy of 2 overlaps, and at least one optical axis of the first multi-eye two-dimensional image input device and the second A three-dimensional image input device, wherein at least one optical axis of the multi-eye two-dimensional image input device is aligned so as to have the same line of sight. 2. A configuration in which the role of the imaging distance range of each of the plurality of two-dimensional image input devices according to claim 1 is determined, and a grayscale image is selected based on each of the distance images and then combined. A three-dimensional image input device, characterized in that: