JP2002077944A - Stereoscopic imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable a generation of a stereoscopic image without increasing the size of a stereoscopic imaging device. SOLUTION: This stereoscopic imaging device obtains an image having a parallax from a viewpoint for photographing a normal image by more shifting as an amount of moving the viewpoint is larger or an object to be displayed stands closer to pixel, by deciding the direction for shifting the pixel for constituting the normal image by a stereoscopic image generator 141 according to a direction for moving the viewpoint, based on the normal image photographed by a camera 100 and distance information acquired by a distance acquiring means. In the case of a binocular image, an image for the right eye and an image for the left eye are generated, and displayed on a head mounted display 120. In the case of a multiple-lens image, the images in which a parallax is broadened stepwise to the right and to the left from the normal image are generated, and their images are displayed on a stereoscopic video display unit 130.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video device, and more particularly to a video device for generating a stereoscopic video.

[0002]

2. Description of the Related Art In observing an image, observing a subject in three dimensions is very effective in grasping the shape and size of the subject. For this reason, a video device that displays a three-dimensional shape includes a device that displays a binocular image such as a head-mounted display and a device disclosed in Japanese Patent Application Laid-Open No. H11-30864.
Various methods have been proposed, such as one that displays a multi-view image using a lenticular lens as described in No. 2. Similarly, various methods have been proposed for a photographing apparatus for photographing a three-dimensional shape of a subject, such as a photographing apparatus for photographing a subject as a binocular image or a multi-view image.

[0003] The simplest one for displaying a stereoscopic image is a stereo camera. This is a photograph taken by two cameras separated by a distance equivalent to the parallax of the human eyes from the left and right, a photograph taken by the right camera with respect to the subject is taken with the right eye, and a photograph taken with the left camera is taken with the subject. By viewing the photographed photograph with the left eye, a binocular stereoscopic image is shown. Applying the above principle, two housings corresponding to both eyes are built in one housing,
Cameras that can easily capture binocular stereoscopic images have also been devised.

[0004] In addition, a multi-view image can be obtained by photographing one subject from a plurality of viewpoints and simultaneously displaying the images on a stereoscopic image display device using a lenticular lens.

[0005]

However, in order to obtain a binocular stereoscopic image with a stereo camera system, two photographing systems are required, and the size of the apparatus becomes large. Therefore, in order to solve this problem, there is known a technique of taking a stereo image by attaching an attachment to a lens end of a normal camera, but in any case, the size of the device is increased as compared with a normal camera. I will.

Further, since a plurality of photographing systems are required to obtain a multi-view image, the size of the apparatus is further increased.

Further, a method has been proposed in which one camera is moved to a plurality of positions having different viewpoints for photographing in order to obtain a binocular image or a multi-view image, but this method is also used for photographing. It requires a large space and is difficult to use when the shooting position is limited.

Further, as described above, a method that causes an increase in the size of the apparatus and a method that requires photographing from a plurality of photographing positions are described in particular in the size of an apparatus such as an endoscope used for photographing a narrow space. Is not practical in the case of a video apparatus in which the smaller

SUMMARY OF THE INVENTION In view of the above-mentioned problems of the prior art, an object of the present invention is to provide a relatively small stereoscopic video apparatus capable of generating a stereoscopic video from a normal image photographed by a monocular optical system. It is assumed that.

[0010]

According to the present invention, there is provided an image apparatus comprising: a distance obtaining means for obtaining distance information to each part of a subject;
A video obtaining means for obtaining video information of the subject; and a stereoscopic video of the subject by giving different parallax to each part of the subject based on the video information obtained by the video obtaining means and the distance information obtained by the distance obtaining means. And a stereoscopic video display unit for displaying the stereoscopic video generated by the stereoscopic video generation unit.

Here, based on the principle diagram shown in FIG. 2, a method in which the stereoscopic image generating means generates a stereoscopic image which is a binocular image or a multi-eye image by giving a parallax to the image information. explain.

First, a method of generating an image viewed from a position where the viewpoint is shifted from the photographing position will be described. Here, assuming that the viewpoint is shifted to the right as an example, a round object closer to the square object 1b far from the shooting position, such as an image 21b viewed from the camera 20b placed at the assumed position shifted to the right. The object appears to shift leftward as much as 1a. Further, as the displacement of the viewpoint increases, the shift amount also increases in proportion.

For this reason, when generating an image for the right eye, the pixels displaying the object 1a closer to the image acquisition means are shifted to the left more, and the pixels displaying the distant object 1b are shifted to the left less. When generating an image for the left eye, as shown in an image 21d viewed from the camera 20d placed at an assumed position shifted to the left, many pixels displaying the object 1a close to the image acquisition means are shifted to the right, and a distant object 1
It can be seen that a stereoscopic image can be generated by shifting the number of pixels displaying b to the right. in this way,
It is possible to obtain a stereoscopic image by determining the direction in which pixels constituting an image captured by the camera are shifted according to the direction in which the viewpoint is moved, and shifting the viewpoint by a larger amount and shifting the pixel closer to the object to be displayed by a larger amount. it can.

That is, by viewing the right-eye image generated as described above with the right eye and the left-eye image with the left eye,
A binocular image can be obtained.

Similarly, a multi-view image can be obtained by generating images captured from a plurality of viewpoints by changing the shift amount according to the distance from one image according to the above method.

Further, in the video apparatus according to the present invention, the distance obtaining means includes: an irradiating means capable of irradiating the subject with light from a plurality of light emitting positions having different diverging irradiation distances to the subject at substantially the same time; An imaging means capable of independently taking an image of a reflected light from a subject due to light from a position, and a distance from a light emitting position of each part by calculation based on a ratio of the reflection intensity of each part corresponding to each independently taken reflected light image May be used.

Here, the divergent irradiation distance means the distance that light diverges and travels so that the illuminance per unit irradiation area is inversely proportional to the square of the distance.
It does not include the distance traveled in the optical fiber. In addition, substantially at the same time means a time difference at the same time or at such a degree that no motion is recognized in the subject.

Here, a calculation method for calculating a distance from an image photographed by a single eye based on the principle diagram shown in FIG. 3 will be described.

The subject 2 is irradiated with light from two point light sources 30 and 31 having different distances from the subject 2. At this time, the brightness of the point light source 30 close to the subject is set to a known value L ₁ ,
The brightness of the point light source 31 far from the subject is a known value L ₂ , the distance between the two point light sources 30 and 31 is a known value L, the distance from the point light source 30 to the subject is R ₁ , and the distance from the point light source 31 to the subject is Is R ₂ , and the spectral reflectance of the light emitted from each of the point light sources 30 and 31 of the object is Rf. From the positional relationship between the elements, R ₁ + L = R ₂ (1) Assuming that the intensity of light reflected by the light source ² is Lr1, Lr ₁ = Rf · L ₁ / 4πR ₁₂ (2), and the intensity of light reflected from the point light source 31 by the subject 2 is Lr2, and Lr ₂ = Rf · L ₂ / 4πR ₂ ² (3)

[0020] Here, when determining the ratio of these reflected light intensity as _{Wr, Wr = Lr 1 / Lr} 2 = Rf · L 1 · 4πR 2 2 / 4πR 1 2 · Rf · L 2 = L 1 · R 2 ² / L ₂ · R ₁ ² (4). By substituting equation (1) into equation (4) and transforming it into an equation for calculating the distance R ₁ to the subject,

(Equation 1) Is obtained.

From equation (5), L ₁ , L ₂ , and L are known values, and Wr is obtained from the ratio of the luminance of each pixel of the two captured images, so that the distance R 1 to the subject is obtained. Can be.

A computer for inputting and calculating the reflected light images from the point light source 30 and the point light source 31 captured by the camera 20 performs the above-described calculation for each pixel of the captured image, thereby obtaining the subject from the captured image. The distance to each part can be obtained.

Further, the image acquisition means and the imaging means used in the distance acquisition means may be the same means.

In the video apparatus according to the present invention, the stereoscopic video generating means may be capable of changing the magnitude of the parallax, and furthermore, the stereoscopic video generating means may generate the parallax according to the magnification of the image displayed on the display means. The size may be determined.

In the video device according to the present invention, the stereoscopic video generating means may generate a binocular image or a multi-view image.

Further, in the video device according to the present invention,
The image acquisition means can be an endoscope.

[0027]

According to the video apparatus of the present invention configured as described above, a stereoscopic video can be generated from the video obtained by the video obtaining means using the monocular optical system, so that an increase in the size of the apparatus can be prevented. It is possible to provide a relatively small image device capable of stereoscopic viewing.

[0028]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing a schematic configuration of the endoscope apparatus according to the present embodiment.

Here, in the present embodiment, the distance obtaining means includes:
Light is emitted to a subject from multiple light-emitting positions with different divergent irradiation distances at substantially the same time, and reflected light images from the subject caused by the light are independently taken, and the reflection intensity of each of the corresponding portions of the taken reflected light image is taken. A method of calculating the distance from the light emission position of each part by calculation based on the ratio of the distance to obtain the distance to the subject is used.

The video device according to the present embodiment is a camera 1
00 and two light sources 104 and 105 provided on the camera 100 for irradiating light from two positions 50a and 50b having different divergent irradiation distances to the subject 1.
A computer 140 having a function of calculating a normal image and distance distribution information based on two images captured by the camera 100 and outputting an imaged signal;
A stereoscopic image generation device 141 that generates a stereoscopic image based on the normal image and the distance distribution information sent from the camera 40, and a head mounted display 120 that receives an image output signal of the stereoscopic image generation device 141 and displays it as a binocular image. It is composed of

An objective lens 101 is provided at the front end inside the camera 100, and a large number of CCDs are provided on the rear surface thereof.
An image element 102 composed of elements is provided, and the tip of a CCD cable 103 is connected to the image element 102. The proximal end of the CCD cable 103 is connected to a computer 140.

Next, the operation of the video apparatus according to the present embodiment configured as described above will be described.

First, the light source 104 attached to the camera 100 and having a short distance to the subject irradiates the subject 1 with white light for photographing a normal image. White light subject 1
Is reflected by the objective lens 101 and forms an image on the image sensor 102. A video signal from the image sensor 102 passes through the CCD cable 103 to the computer 14.
0 and stored in a memory in the computer 140.

Next, the subject 1 is irradiated with white light for photographing a normal image by the light source 105 attached to the camera 100 and having a long distance to the subject. The reflected light of the white light from the subject 1 is condensed by the objective lens 101 and forms an image on the image sensor 102. The video signal from the image sensor 102 is sent to the computer 140 through the CCD cable 103 and stored in a memory in the computer 140.

As described above, the reflected light image of the light from the long-distance light source 105 and the reflected light image of the light from the short-distance light source 104 are taken independently in synchronization with the switching of the light emission of the light source.

Next, the distance to each part of the subject is calculated by the computer 140 from the two captured images based on the above-described principle, whereby the distance is calculated for each pixel of the image. One of the two captured images is also used as a normal image.

The normal image and the distance information obtained as described above are sent from the computer 140 to the three-dimensional image generator 1.
It is sent to 41. Based on the above-described principle, the stereoscopic image generation device 141 generates a right-eye image and a left-eye image based on the sent normal image and distance information, and the images are displayed on the head-mounted display 120.

The right-eye image and the left-eye image constituting the binocular image displayed on the head-mounted display 120 have an amount by which the pixels are shifted left and right from the original image in accordance with the distance to the subject according to the above-described principle. Is determined, and the coefficient is multiplied by the amount by which each pixel is shifted according to the magnitude of the left and right parallax. By adjusting this coefficient, the three-dimensional effect can be emphasized or reduced.

Further, when the magnification of the displayed image is large, the three-dimensional effect is reduced, and when the magnification is small, the three-dimensional effect is emphasized. The degree of emphasis or reduction of parallax can be automatically determined.

According to the video apparatus of the present invention configured as described above, since the video acquisition means is a monocular optical system,
An increase in the size of the device can be prevented, and a small stereoscopic video device can be provided.

In this embodiment, the subject is irradiated with light substantially simultaneously from a plurality of light-emitting positions having different divergent irradiation distances, and the reflected light image from the subject caused by the light is independently photographed, and the reflected light of the photographed light is reflected. The distance to the subject was measured by a method of calculating the distance from the light emission position of each part by calculation based on the ratio of the reflection intensity of the corresponding part of the image to obtain the distance to the subject, but the distance to the subject was measured. Any distance acquiring means may be used as long as the distance can be acquired.

In this embodiment, a head mounted display is used as a device for displaying a binocular image.
Any device that can display a binocular image may be used.

The case of generating a multiview image in this embodiment will be described with reference to FIGS.
Here, the subject 1a at a short distance and the subject 1b at a long distance are photographed by the monocular camera 20a.

One image 21 photographed by the camera 20a
a, images 21b, 21 having parallaxes assumed to be taken from cameras 20b, 20c, 20d, 20e assumed to be arranged on the left and right of the camera 20a
c, 21d, and 21e are generated by the computer 140 and the stereoscopic video generation device 142 according to the distances calculated by the above-described method to the subjects 1a and 1b, and the assumed positions of the cameras. These five images become multi-view images having different parallaxes, and are displayed on the stereoscopic video device 130.

As the stereoscopic image apparatus 130 used here, any apparatus can be used as long as it can display a multi-view image, such as an apparatus using a lenticular lens as disclosed in JP-A-11-308642. Is also good. When a lenticular lens is used, five images 21a to 21e serving as the basis of the multiview image are processed and displayed for the lenticular lens, and are stereoscopically viewed together with the lens.

Next, a second embodiment of the present invention will be described. The second embodiment shows an example in which the video apparatus according to the present invention is applied to an endoscope, and FIG. 5 is a diagram showing a schematic configuration of the present embodiment.

The endoscope apparatus according to the present embodiment includes an endoscope 200 inserted into a body cavity of a subject,
0, an illumination unit 260 provided with two light sources 262 and 265 for irradiating light from two positions 51a and 51b having different divergent irradiation distances to the subject 10 inside the endoscope 200. The image sensor 202 has a function of calculating a normal image signal and distance distribution information based on two images taken through the objective lens 201, outputting a signal obtained by imaging, and a function of controlling the entire endoscope apparatus. A computer 240, a stereoscopic video generation device 241 for generating a stereoscopic video based on the normal image and the distance distribution information sent from the computer 240, and a video output signal from the stereoscopic video generation device 241 to be displayed as a binocular image. And a head mount display 120.

The endoscope 200 has a C
A CD cable 204, a short distance light guide 206, and a long distance light guide 208 are provided. An image pickup device 202 is connected to a distal end portion of the CCD cable 204, and a reflection prism 203 is attached to the image pickup device 202. The distal end of the reflecting prism 203, the short-distance light guide 206, and the long-distance light guide 208, that is, the distal end of the endoscope 200 are provided with the objective lens 201, the short-distance illumination lens 205, and the long-distance illumination lens 207. Is provided. The proximal end of the CCD cable 204 is connected to a computer 240 and is connected to the short-distance light guide 20.
6 and the base end of the long distance light guide 208 are connected to the lighting unit 260.

The illumination unit 260 includes a short-distance white light source 262 that emits white light for a normal image as a light source for irradiating the endoscope 200 with the short-distance illumination lens 205 through the short-distance light guide 206, A power supply 263 for a short-distance white light source electrically connected to the short-distance white light source 262, and a short-distance white light condenser lens 26 for condensing white light emitted from the short-distance white light source 262.
1. A long-distance white light source 265 that emits white light for normal images as a light source for irradiating the endoscope 200 with the long-distance irradiation lens 207 through the long-distance light guide 208, and the long-distance white light source 265 And a power source 266 for a long-distance white light source electrically connected to the camera, and a long-distance white light condenser lens 264 for condensing the white light emitted from the long-distance white light source 265.

A CCD cable 204 extending from the endoscope 200 is connected to the computer 240.

Next, the operation of the endoscope apparatus according to the present embodiment configured as described above will be described.

First, the endoscope 200 is inserted into the body cavity of the subject by the hand of the operator. Thereafter, the short-range white light source power supply 263 is driven, and the short-range white light source 262 is driven.
Emits white light. The white light is incident on the light guide 206 via the short-distance white light condensing lens 261 and is guided to the distal end of the endoscope 200.
The subject 1 by the illumination lens 205 located at a distance close to 0
It is irradiated to 0. The reflected light of the white light from the subject 10 is condensed by the objective lens 201 and is reflected by the reflecting prism 20.
3 and is imaged on the normal image pickup device 202. A video signal from the image sensor 202 is sent to the computer 240 through the CCD cable 204 and stored in a memory in the computer 240.

Next, similarly to the short-distance light source, the long-distance white light source power supply 266 is driven, and the long-distance white light source 265 is driven.
Emits white light. The white light is incident on the light guide 208 through the long-distance white light condensing lens 264 and is guided to the distal end of the endoscope 200, and then the subject 1
Subject 1 by the illumination lens 207 at a distance far from 0
It is irradiated to 0. The reflected light of the white light from the subject 10 is condensed by the objective lens 201 and is reflected by the reflecting prism 20.
3 and is imaged on the normal image pickup device 202. The video signal from the image sensor 202 is sent to the computer 240 through the CCD cable 204 and stored in a memory in the computer 240.

As described above, the reflected light image of the light from the long-distance light source and the reflected light image of the light from the short-distance light source are independently captured in synchronization with the switching of the light emission of the light source.

Next, the distance to each part of the subject is calculated from the two captured images by the computer 240 based on the above-described principle, and the distance is calculated for each pixel of the image. Is calculated. One of the two captured images is also used as a normal image.

The normal image and the distance information processed as described above are sent from the computer 240 to the three-dimensional image generator 24.
Sent to 1. Based on the above-described principle, the stereoscopic image generation device 241 generates a right-eye image and a left-eye image based on the transmitted normal image and distance information, and the images are displayed on the head-mounted display 120.
Thereby, the image of the endoscope can be viewed as a stereoscopic video.

In this embodiment, as in the first embodiment, the stereoscopic effect can be enhanced or reduced, and the degree of parallax enhancement or reduction is automatically determined based on the magnification of the displayed image. It can also be done.

The same effect as that of the first embodiment can be obtained by the video device according to the present embodiment configured as described above.

Also in this embodiment, the subject is irradiated with light almost simultaneously from a plurality of light emitting positions having different diverging irradiation distances, and the reflected light image from the subject is independently photographed by the light, and the reflected light of the photographed light is reflected. The distance to each part of the subject was measured by a method of calculating the distance from the light emission position of each part by calculation based on the ratio of the reflection intensity of each part corresponding to the light image to obtain the distance to the subject. Any means can be used as the distance obtaining means as long as the method can achieve the following.

In this embodiment, a head-mounted display is used as a device for displaying a binocular image.
Any device that can display a binocular image may be used.

Also in the present embodiment, a multiview image can be generated and displayed as in the first embodiment.

[Brief description of the drawings]

FIG. 1 is a schematic configuration diagram of a video device according to a first embodiment of the present invention;

FIG. 2 is a principle diagram of a stereoscopic image generation method used in the present invention.

FIG. 3 is a principle diagram of a distance calculation method used in the present invention.

FIG. 4 is a schematic configuration diagram of a video device according to the first embodiment of the present invention.

FIG. 5 is a schematic configuration diagram of a video device according to a second embodiment of the present invention.

FIG. 6 is a schematic configuration diagram of a video device according to a second embodiment of the present invention.

[Explanation of symbols]

 Reference Signs List 1 subject 100 camera 120 head mounted display 121 video cable 130 stereoscopic video display device 140, 240 computer 141, 241 stereoscopic video generation device 142, 242 stereoscopic video generation device 200 endoscope 260 lighting unit

Claims (8)

[Claims] 1. A distance obtaining means for obtaining distance information to each part of a subject, a video obtaining means for obtaining video information of the subject by a monocular optical system, and a video information obtained by the video obtaining means and the distance obtaining means. Based on the obtained distance information, a stereoscopic video generating unit that generates a stereoscopic video of the subject by giving different parallax to each part of the subject, and displays the stereoscopic video generated by the stereoscopic video generating unit. An image device comprising: a stereoscopic image display means. 2. The illuminating device according to claim 2, wherein the distance obtaining unit irradiates the subject with light from the plurality of light emitting positions having different divergent irradiation distances to the subject at substantially the same time, and the light from each of the light emitting positions of the irradiating unit. An imaging unit capable of independently capturing a reflected light image from the subject; and calculating a distance from a light emitting position of each unit by an operation based on a ratio of a reflection intensity of each unit corresponding to each of the independently captured reflected light images. 2. The video apparatus according to claim 1, wherein the video apparatus comprises: 3. The video apparatus according to claim 2, wherein said video acquisition means and the imaging means used in said distance acquisition means are the same. 4. The video apparatus according to claim 1, wherein said stereoscopic video generating means is capable of changing the magnitude of the parallax. 5. The video apparatus according to claim 2, wherein said stereoscopic video image generation means determines the magnitude of the parallax according to an enlargement ratio of an image displayed on the display means. . 6. The image apparatus according to claim 1, wherein said stereoscopic image generation means generates a binocular image. 7. An image apparatus according to claim 1, wherein said stereoscopic image generation means generates a multi-view image. 8. The video apparatus according to claim 1, wherein said video acquisition means is an endoscope.