JPH10221775A - Medium recorded with stereoscopic vision image pickup display program, and compound eye image input/output device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a medium on which a stereoscopic vision image pickup display program for controlling an image output device is recorded and a compound eye image input/output device excellently displaying a non-overlapped area at the time of performing stereoscopic vision image pickup display of a pair of images picked up by a compound eye image pickup device and having parallax each other, and having a function to select two-dimensional display and three-dimensional display. SOLUTION: This compound eye image input/output device having two image pickup optical systems, picking up and outputting a pair of left and right images having the parallax each other is provided with a display means performing the stereoscopic vision image pickup display in a state where non-overlapped areas 303b and 303a are overlapped by newly adding image data to a pair of left and right images 301b and 301a forming a stereoscopic vision image 306 including the non-overlapped areas 303b and 303a when a pair of left and right images 301b and 301a are overlapped to perform the stereoscopic vision image pickup display.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a medium storing a stereoscopic imaging display program and a compound eye image input / output device having a stereoscopic imaging display function.

[0002]

2. Description of the Related Art Conventionally, a stereoscopic imaging display method using a compound-eye imaging device has been known. In this method, two imaging optical systems are arranged on the left and right at an interval given by the base line length, and images are taken from two viewpoints.

[0003] When a subject of interest is imaged from two viewpoints on the left and right in this way, the position of the object in the image captured by each imaging system is different from each other in the horizontal direction of the image. That is, this is the parallax, and by viewing this parallax in stereo, the observer sees an image with a three-dimensional effect.

FIG. 11 is a block diagram of a conventional compound eye image input / output device for stereoscopic imaging display. The compound eye imaging apparatus 101 includes two left and right imaging optical systems 102a and 102b.
Each of which has lenses 103a and 103b and CCDs 104a and 104b as image pickup devices. This imaging optical system includes a synchronization signal generator 105,
The two imaging optical systems 102a and 102b perform synchronized imaging. The compound-eye imaging apparatus 101 includes A / D converters 106a and 106b connected to the left and right imaging optical systems, respectively, and further includes a memory 107. The compound-eye imaging apparatus 101 is connected to a personal computer (hereinafter, referred to as a PC) 109 via an interface cable 108. In the PC10, a parallel interface 111, a CPU 112, a memory 113, an image processing unit 114, a stereoscopic image display control unit 11 are connected to a CPU bus 110.
5, storage device 116, display controller 117
Is connected, and input of an image signal from the compound-eye imaging device 101 is performed by the parallel interface 111,
The output of the image to the display 119 is performed via the display controller 117.

First, in the compound eye imaging apparatus 101, the lenses 103a and 103a in the two imaging optical systems 102a and 102b are provided.
The two left and right images formed in 03b are respectively CCD1
Obtained in O4a, 104b. Image is sync signal generator 1
05 based on the signal from the imaging optical system 102a, 1
02b is acquired synchronously. The obtained electric signals of the image are converted into digital signals by A / D converters 106a and 106b, respectively, and stored in the memory 107. These two types of image signals are input to a parallel interface 110 in the PC10 via an interface cable.

[0006] The input image data is first transferred to the memory 113 in the PCLO 9 via the CPU bus 110. Here, image processing in the PC10 is performed in this memory area. Next, image processing such as left-right difference correction is performed by the image processing unit 114, and a control signal to be displayed on the display 119 is generated by the stereoscopic image display control unit 115. Based on this signal, the display controller 117 transfers the left and right image data to the VRAM 118 and
19 is displayed. Similarly, the left and right images captured once by the compound-eye imaging device 101 are stored in the storage device 11 under the control of the CPU 112.
6 is stored in the storage device 116 and is reproduced from the storage device 116 and displayed on the display 119 again.

There are various methods for stereoscopically viewing an image obtained by the compound-eye imaging device 101. One is that the left and right images are alternately output on the display 119 on the left and right sides, and the viewer views the liquid crystal shutter glasses 120 that switch the left and right shutters in synchronization with the switching of the display of the left and right images. This is to obtain a stereoscopic image. In this display method, the stereoscopic image display control unit 11
In 5, the display controller 117 is controlled so that the left and right images obtained by the compound-eye imaging apparatus 101 are switched and displayed on the display 119 in accordance with the vertical synchronization frequency of about 120 Hz. A signal is transmitted to the liquid crystal shutter glasses 120 to control the shuttering of the left and right fields of view. The observer can see a stereoscopic image by wearing the liquid crystal shutter glasses 120 and viewing the image on the display 119.

Another display method is to alternately arrange two left and right images every other line in the vertical scanning direction, and
A striped stereoscopic image is created. The display screen has polarizers whose polarization directions are orthogonal to each other every other line in the vertical scanning direction, similarly to the created stereoscopic image, and the images are displayed in stripes with different polarization directions. When the striped stereoscopic image created is displayed on this display, the image captured by the right imaging optical system is displayed by transmitting only polarized light in a certain direction, and the image captured by the left imaging optical system. Is displayed by transmitting only polarized light orthogonal to the right image. On the other hand, the observer wears polarized glasses with the function of transmitting only the same polarized light as the image displayed on the display on the left and right, so that only the right image is displayed for the right eye and only the left image is displayed for the left eye It is supposed to be. Using these glasses, the observer can see the right image only with the right eye and the left image only with the left eye, and the observer can see an image with a three-dimensional effect.

In this case, the stereoscopic image display control section 115 creates a stereoscopic image by arranging the left and right images line by line in the vertical scanning direction, and generates a control signal to be displayed on the display 119. At this time, the display 119 must be a display having a polarizing plate as described above.

As described above, in the stereoscopic image pickup display, the parallax of images picked up from different viewpoints is used.

In a compound-eye imaging apparatus, the parallax of an image captured is determined by the base line length of the imaging optical system, its convergence angle, and the distance of the target object from the imaging optical system. Accordingly, when a pair of captured left and right images are displayed to fill the horizontal size of the display image, the image seen by the observer has a certain stereoscopic effect.

FIG. 12 shows how a stereoscopic image having a different stereoscopic effect is displayed by changing the amount of overlap. The image in the figure is a case where it is displayed on a display window having a specific size, and shows the size at that time. In FIG. 12A, first, a landscape 202 is imaged by the compound-eye imaging device 201. It is assumed that imaging optical systems are arranged in the compound-eye imaging device 201 in a horizontal direction.
The images captured by the left and right imaging optical systems are a pair of left and right images 203a and 203a having parallax as shown in FIG.
03b is displayed.

The two images 203a and 203b are superimposed on the display image to fill the horizontal size as shown at 205, and are displayed by stereoscopic imaging. Here, the size of the display image is w. In the case of using stereoscopic images, two images 203a and 203b may be displayed in chronological order when liquid crystal shutter glasses are used, and two images may be displayed when a polarizing plate type display is used. May be displayed by alternately arranging the lines one by one in the vertical scanning direction. The image obtained here has a parallax determined by the base line length and the convergence angle of the two imaging optical systems and the distance of the target object 204 from the imaging optical system, and gives a certain stereoscopic effect to the observer.

However, when creating a stereoscopic image, the amount of overlap between the two left and right images (hereinafter referred to as the amount of overlap)
, It is possible to control the amount of displacement of the object in the image caused by the parallax and to view images having various stereoscopic effects. Now, it is assumed that the target object 204 in the image has a shift amount by d in the horizontal direction of the two left and right images 203a and 203b. Then, when the stereoscopic image 205 is created by superimposing the full horizontal size of the image, that is, the overlap amount of the left and right images as w, the target object 2
04 exist in the image two horizontally apart by d.
Here, as shown in FIG. 12C, the right image 203a is shifted horizontally by Δd with respect to the left image 203b, that is, the overlap amount of the two images is set to w−Δd
When the stereoscopic image 206 is created in this manner, the shift amount of the target object in the image is d− △ d, which is seen as an image having a different stereoscopic effect by the observer.

As described above, by setting the overlap amount arbitrarily to the compound eye image actually taken by the observer,
It is possible to change the three-dimensional appearance.

[0016]

The above-mentioned prior art has the following problems.

The above two left and right images 203a and 203b
Each have a non-overlapping area (hereinafter referred to as a non-overlapping area) having a width of Δd. Conventionally, when displaying a stereoscopic image using a set of images having parallax with each other, attention is paid only to an overlap region that allows stereoscopic vision, and how to display a non-overlap region. Such a clear method has not been established. For example, when viewing a stereoscopic image by the stereoscopic imaging display method using liquid crystal shutter glasses, the observer views the non-overlapping region of each of the left and right images with one eye, that is, the non-overlapping region in the right image. Is only the right eye, and the non-overlapping area in the left image can be seen only by the left eye, and the area cannot be seen by the other eye, so the image appears to flicker. Therefore, it hinders the stereoscopic imaging display of the overlap area and gives an uncomfortable feeling. Similarly, in the case where the left and right images are alternately arranged one line at a time in a stripe form, even if the data of one image is input in a stripe form in the stereoscopic image, the data of the other image is not provided. Therefore, there is a problem that the non-overlap area can be seen only by one eye.

Further, a plane image output apparatus having a function of selecting and displaying a two-dimensional display and a three-dimensional display according to a window in a screen as an apparatus for outputting a stereoscopic image is provided. Is not clearly established yet.

In view of the above-mentioned problems of the prior art, it is an object of the present invention to display a non-overlapping region well when stereoscopically displaying a set of images having parallax captured by a compound-eye imaging device. It is an object of the present invention to provide a medium recording a stereoscopic imaging display program for controlling an image output device having a function of selecting two-dimensional display and three-dimensional display, and a compound-eye image input / output device.

[0020]

A compound eye image input / output apparatus according to the present invention is a compound eye image input / output apparatus having two image pickup optical systems for picking up and outputting a pair of left and right images having parallax therebetween. When a set of left and right images are overlapped and stereoscopically imaged and displayed, by adding new image data to each of a pair of left and right images forming a stereoscopic image including a non-overlapping region, A display unit is provided for displaying a stereoscopic image by overlapping the non-overlapping areas.

Further, the display means for stereoscopically imaging and displaying newly adds an image area to each of a pair of left and right images forming a stereoscopic image including a non-overlapping area, and adds a new image area to the image area and the non-overlapping area. A non-overlapping area may be overlapped by substituting an arbitrary value to perform stereoscopic imaging display. In addition, the display means for stereoscopic imaging display adds a new image area to each of a pair of left and right images forming a stereoscopic plane image including a non-overlapping area, and adds one image area to one of the image areas. The image data of the other non-overlapping area may be substituted, and the non-overlapping area may also be overlapped and stereoscopically displayed.

The medium storing the stereoscopic imaging display program according to the present invention is provided in a compound-eye image input / output device that has two imaging optical systems and captures and outputs a pair of left and right images having parallax. A medium on which a stereoscopic image display program is recorded, wherein the program forms a stereoscopic image including a non-overlapping area when displaying a stereoscopic image by overlapping a set of left and right images. By adding new image data to each of the pair of left and right images, the non-overlapping area is also overlapped to function as display means for stereoscopic imaging display.

The display means for stereoscopically imaging and displaying newly adds an image area to each of a pair of left and right images forming a stereoscopic image including a non-overlapping area, and adds a new image area to the image area and the non-overlapping area. A non-overlapping area may be overlapped by substituting an arbitrary value to perform stereoscopic imaging display.

[0024] The display means for stereoscopically imaging and displaying newly adds an image area to each of a pair of left and right images forming a stereoscopic image including a non-overlapping area, and outputs one of the image areas. The image data of the other non-overlapping area may be substituted for the area, and the non-overlapping area may also be overlapped and stereoscopically imaged and displayed.

Accordingly, in a stereoscopic image pickup display using a compound eye image input / output device, a non-overlapping region of a set of left and right images generated when adjusting the stereoscopic effect of a stereoscopic image is substituted by an arbitrary value. By performing the dimensional display, the observer can recognize the non-overlapping region with both the left and right eyes, and can provide an easily viewable stereoscopic image.
Alternatively, by displaying the non-overlapping area two-dimensionally as it is, an image in which two-dimensional display and three-dimensional display are mixed can be observed.

Further, a display image mode selection means for selecting the display means for performing the above-mentioned stereoscopic imaging display may be provided.

Further, the program may cause the computer to function as a display image mode selecting means for selecting a display means for performing the above-described stereoscopic imaging display.

Therefore, in the stereoscopic image pickup display using the compound-eye image input / output device, a plurality of stereoscopic image generating means can be provided and selected by the display image mode selecting means. A visual image can be observed.

Also, an image having display means for switching between a two-dimensional image and a three-dimensional image in a stereoscopic image in which a two-dimensional image and a three-dimensional image are mixed to a display method corresponding to each of the two-dimensional image and the three-dimensional image. An output device may be provided.

Further, the program stores the computer
A function as a means for controlling an image output apparatus having a display means for stereoscopically imaging and displaying a two-dimensional image and a three-dimensional image in a stereoscopic image in which a two-dimensional image and a three-dimensional image are mixed, by switching to a display method corresponding to each of them. May be.

Further, a display means for switching to a display method corresponding to each of them and performing stereoscopic image pickup and display is provided with two-dimensional image in the stereoscopic image.
The three-dimensional image part is three-dimensional image part and the three-dimensional image part is 3
The display may be switched to the two-dimensional image display.

Accordingly, an image output device having a function of selecting and displaying a two-dimensional display and a three-dimensional display according to a window in a screen as an output device of a stereoscopic image is provided. Is established, and a stereoscopic image in a different display mode can be displayed by selecting a display mode.

[0033]

Embodiments of the present invention will be described below with reference to the drawings.

(First Embodiment of the Present Invention) In the first embodiment, in the stereoscopic image pickup display using the compound eye image input / output device, the left and right images generated when adjusting the stereoscopic effect of the stereoscopic image are adjusted. An embodiment in which a non-overlapping area is displayed as a two-dimensional image by substituting an arbitrary value will be described.

FIG. 1 is a block diagram of a compound-eye image input / output device according to the first embodiment of the present invention. The compound-eye imaging device 1 has two left and right imaging optical systems 2a and 2b, each of which has lenses 3a and 3b and a CCD 4 as an imaging device.
a, 4b. Here, in the present embodiment, a CCD is used as an image sensor, but another image sensor may be used. The imaging optical system includes a synchronization signal generator 5, and performs synchronized imaging with the two imaging optical systems 2a and 2b. Further, the compound-eye imaging apparatus 1 includes A / D converters 6a and 6b connected to left and right imaging optical systems, respectively.
Further, a memory 7 is provided. The compound eye imaging apparatus 1 is connected to a personal computer (hereinafter, referred to as a PC) 9 via an interface cable 8. C in PC9
Parallel interface 11 and CPU on PU bus 10
l2, memory 13, image processing unit 14, stereoscopic image display control unit 15, storage device 16, display controller 1
7, and a stereoscopic image generation unit 21 is connected unlike the conventional compound eye image input / output device.
The input of image signals from the compound-eye imaging device 1 is performed by the parallel interface 11, and the output of images to the display 19 is performed via the display controller 17.

First, two left and right images formed by the lenses 3a and 3b in the two imaging optical systems 2a and 2b in the compound eye imaging apparatus 1 are acquired by the respective CCDs 4a and 4b.
An image is acquired by synchronizing the left and right imaging optical systems 2a and 2b based on a signal from the synchronization signal generator 5. The obtained image signals are converted into digital images by A / D converters 6a and 6b, respectively, and stored in the memory 7. These two image signals are input to the parallel interface 10 in the PC 9 via the interface cable 8.

The input image data is transferred to the memory 13 in the PC via the CPU bus 10. Here, image processing in the PC 9 is performed in this memory area. First, the image processing section 14 performs image processing such as left-right difference correction. Then, the stereoscopic image generation unit generates a stereoscopic image including the non-overlapping area displayed by the method proposed in the present embodiment. This display method and generation method will be described later. Thereafter, image processing for displaying on the display 19 is performed by the stereoscopic image display control unit 15. In the present embodiment, a case will be described in which the compound-eye image input / output device performs stereoscopic imaging display using liquid crystal shutter glasses. Therefore, in this display method, the stereoscopic image display control unit 15 controls the display controller 17 so that the left and right images obtained by the compound-eye imaging device 1 are switched and displayed on the display 19 in accordance with the vertical synchronization frequency of about 120 Hz. On the other hand, a signal synchronized with the switching of the left and right images is transmitted to the liquid crystal shutter glasses 20 to control the shuttering of the left and right fields of view.

After the stereoscopic image display controller 15 processes the stereoscopic image as described above, the image data is transferred to the VRAM 18 under the control of the display controller 17.
And displays it on the window of the display 19. The observer can see a stereoscopic image by wearing the liquid crystal shutter glasses 20 on the left and right images switched on the display 19 according to the vertical synchronization frequency of about 120 Hz.

Next, a method of displaying a non-overlapping area of a stereoscopic image according to the present embodiment will be described. FIG.
Is an explanatory diagram of the present embodiment. The image size in the figure is a value on the display screen of a specific size as in the case of FIG. By changing the amount of overlap between the left and right image areas to be displayed when performing stereoscopic imaging display as described in the section of “Prior Art”, the observer can change the stereoscopic effect of the stereoscopic image in a white manner. can do. At that time, a non-overlap area corresponding to the overlap amount is generated in each of the left and right images. The observer can see this non-overlapping area with one eye, that is, the non-overlapping area in the right image can be seen only with the right eye, the non-overlapping area in the left image can only be seen with the left eye, and the other eye can see it. Cannot view that area, an unnatural image is displayed.

Therefore, in order to remove this unnaturalness,
In the stereoscopic image generation unit 21 in FIG. 1, an arbitrary value is substituted for a non-overlapping area in each of the left and right images and displayed on the display 19. In FIG. 2, right image 3
01a and the left image 301b are images captured by the compound-eye imaging device and displayed on the display window of the display. The horizontal size of the display image at this time is w. Here, in order to change the stereoscopic effect by paying attention to the target object 302 in the left and right images, a stereoscopic image is generated with the overlap amount being w- △ d. At this time, the non-overlapping amount Δd is set in the stereoscopic image generation unit 21 using a user interface such as a keyboard and a mouse when the observer performs stereoscopic imaging display. However, it is also possible to set at any time through the interface during stereoscopic imaging display.

When a stereoscopic image is generated, non-overlapping areas 303a and 303b having a width of △ d are included in the left and right images.
Can be. The horizontal size of the generated stereoscopic images is w +
Δd is obtained. Therefore, as shown in FIG. 2, image regions 304a and 304b having a width of △ d are generated in a direction in which the non-overlapping regions of the captured left and right images 301a and 301b do not occur. As a result, the horizontal size of the left and right images is w + △ d
Images 305a and 305b. Next, the left and right images 305
a, 305b, the non-overlapping areas 303a,
303b and newly generated image areas 304a and 304b
The luminance value 0 is substituted as an arbitrary value for all the pixels of. The values to be substituted here are image areas 303a and 304b, and 30
Any value may be used as long as the pixel values of 3b and 304a are the same.

The pair of left and right images 305a and 305b to be stereoscopically imaged and displayed in this manner have arbitrary values in the newly added image area and non-overlap area. When this is generated by the stereoscopic image generation unit 21 and displayed on one display window, the observer can view the stereoscopic image 306.
As shown in (1), the newly added image area and the non-overlap area have the same values assigned to the left and right images to be stereoscopically viewed, so that the observer can see the two-dimensional image. That is, in the entire image, the left and right images can be divided into a portion that can be stereoscopically viewed and a plain portion that is two-dimensionally displayed. Therefore, unlike the related art, it is possible to provide a good stereoscopic image without causing the image data that cannot be stereoscopically viewed in the non-overlapping region to flicker.

The program including the method of displaying a non-overlapping area of a stereoscopic image according to the present embodiment is recorded on a medium on which the program is recorded.

In the present embodiment, the display controller controls the display controller so that the left and right images are switched and displayed on the display in accordance with the vertical synchronization frequency of about 120 Hz. Although the example in which the signal is transmitted to the liquid crystal shutter glasses 20 and the left and right fields of view are shuttered has been described, the left and right images are arranged for each line in the vertical scanning direction, and each image is polarized or directed to the display display. The same applies to the case where stereoscopic imaging display is performed by providing the property. FIG. 3 shows the system configuration diagram. Here, reference numeral 402 denotes a 3D display. The system configuration is almost the same as that at the time of display by the liquid crystal shutter glasses of FIG. 1, and the same members are given the same numbers. The difference is that the display 402 is a display such as a 3D display that can have polarization or directivity according to display pixels. In this case, the non-overlapping areas 303a and 303b and the newly generated image area 30 in the stereoscopic image generation unit 21 of FIG.
The luminance value 0 is substituted as an arbitrary value into all the pixels 4a and 304b, and the stereoscopic image display control unit 401 stripes the left and right images on one display window for each line in the vertical scanning direction of the display 402. Display side by side. Then, display is performed with polarization or directivity of the display according to pixels corresponding to the left and right images.
The same effect can be obtained if the observer wears glasses that transmit only the polarized light of each image display on a polarization display, and observes the screen at the center of the surface of the directional display.

In this embodiment, a system configuration via a PC has been described as a compound-eye imaging input / output device, but the same applies to a system other than a PC, for example, a workstation. The same applies to a display-integrated compound eye image input / output device including a control function of a PC and a stereoscopic display.

(Second Embodiment of the Present Invention)
In the embodiment, in a stereoscopic imaging display using a compound-eye image input / output device, each of the left and right images that do not have the non-overlapping region of the left and right images generated when adjusting the stereoscopic effect of the stereoscopic image is not included. An embodiment in which image data of the same area is added to the image and displayed as a two-dimensional image will be described.

This embodiment can be described using the compound eye image input / output device of FIG. 1 as in the case of the first embodiment. Image capture by a compound-eye imaging device and input to a PC,
The method of outputting to the display by the liquid crystal shutter glasses is the same as that of the first embodiment.

However, the difference from the first embodiment is that the non-overlapping region in each of the left and right images is the same as the left or right image having no such region in the stereoscopic image generation unit 21 in FIG. Add image data of area to 2
A two-dimensional image is being generated.

In FIG. 4, the right image 501a and the left image 50
1b is an image captured by the compound eye imaging device and displayed on the display window of the display. The horizontal size of the display image at this time is w. Here, in order to change the stereoscopic effect of the target object 502 in the left and right images, a stereoscopic image is generated with the overlap amount being w- △ d. Also in the present embodiment, the amount of overlap △ d is set in the stereoscopic image generating unit 21 using a user interface such as a keyboard and a mouse when the observer performs stereoscopic imaging display. When a stereoscopic image is generated, a non-overlap area 50 having a width of △ d is included in each of the left and right images.
3a and 503b are formed. The horizontal size of the generated stereoscopic image is w + 画像 d. Therefore, as shown in FIG. 4, an image area 504 having a width of △ d in a direction in which a non-overlapping area of the left and right side images 501a and 501b is not generated.
a, 504b. As a result, the left and right images become images 505a and 505b having a horizontal size of w + △ d. Next, in the left and right images 505a and 505b, the input imaging data is substituted for the non-overlapping areas 503a and 503b, and the newly generated image area 504 is obtained.
The image data of the left non-overlapping area 503b is assigned to the right image area 504a, and the image data of the right non-overlapping area 503a is assigned to the left image area 504b.

The left and right images 505a and 505b to be displayed stereoscopically are displayed in the image area 5 newly added.
04a and 504b have image data of images 503b and 503a in the non-overlapping area of the other image, respectively. When this is generated by the stereoscopic image generation unit 21 to generate a stereoscopic image and displayed on one display window, as shown in the stereoscopic image 506, the observer can view the non-overlapping area of each image stereoscopically viewed. Since the same image data is substituted, the observer can view the image as a two-dimensionally displayed image. That is, the entire image can be viewed separately as a stereoscopically visible portion where the left and right images overlap and image data displayed in two dimensions. Therefore, it is possible to provide a good stereoscopic image without causing the image data that cannot be stereoscopically viewed in the non-overlapping area to flicker as in the related art.

A program including a method of displaying a non-overlapping area of a stereoscopic image according to the present embodiment is recorded on a medium on which the program is recorded.

In this embodiment, the display controller controls the display controller so that the left and right images are switched and displayed on the display in accordance with the vertical synchronization frequency of about 120 Hz. In the example described above, the left and right images are transmitted to the liquid crystal shutter glasses 20 to perform shuttering of the left and right fields of view, but the left and right images are arranged for each line in the vertical scanning direction, and each image is polarized or displayed on the display. The same applies to the case of performing stereoscopic imaging display by providing directivity. FIG. 3 shows the system configuration diagram. Here, reference numeral 402 denotes a 3D display. The system configuration is almost the same as that at the time of the display by the liquid crystal shutter glasses of FIG. 1, and the same members are given the same numbers. The difference is that the display 402 is a display such as a 3D display that can have polarization or directivity according to display pixels. In this case, an image area is generated in a direction in which the non-overlapping area of each of the left and right images captured by the stereoscopic image generation unit 21 in FIG. 3 does not occur, and the non-overlapping area of the other image is generated in that image area. Assign a value. Then, the stereoscopic image display side control unit 401 arranges the left and right images on one display window in a stripe shape for each line in the vertical scanning direction of the display, and displays the left and right images in accordance with the pixels corresponding to the left and right images. Display with polarization or directivity. The same effect can be obtained if the observer wears glasses that transmit only the polarized light of each image display on a polarization display, and observes the screen at the center of the surface of the directional display.

(Third Embodiment of the Present Invention) In a third embodiment of the present invention, a plurality of stereoscopic image generating means relating to a non-overlapping area at the time of stereoscopic image pickup and display, and generation of the stereoscopic image. An example of a compound eye image input / output device having a display image mode selection means for selecting a means will be described. FIG. 5 is an explanatory diagram of the present embodiment. The compound eye imaging input / output device in the present embodiment is almost the same as that in FIG. 1 in the first embodiment and the second embodiment, and the same members are assigned the same numbers.

However, the difference from FIG. 1 is that the stereoscopic image generating unit 601 has a display image mode selecting unit 602 for selecting the display mode of the non-overlapping area of the stereoscopic image, and the stereoscopic image generating unit A 603 and the stereoscopic image generating unit A603. It is provided with a visual image generating means B604 and a stereoscopic image generating means C605.

The operation of the compound eye imaging input / output device according to this embodiment will be described. Using the compound eye imaging device 1, two left and right images are captured and input to the PC 9. After performing left-right difference correction and the like by the image processing unit 14 in the PC, the stereoscopic image generation unit 601 generates a stereoscopic image.
Unlike the embodiments and the second embodiment, first, the observer uses the display image mode selection means 602 to perform the stereoscopic image generation means A603 and the stereoscopic image generation means B60.
4. Select one of the display modes of the stereoscopic image generating means C605. The display image mode selection unit 602 can change the stereoscopic image to be generated and displayed by selecting the display image at any time during the stereoscopic imaging display by a user interface such as a keyboard and a mouse. In the present embodiment, the display image mode is selected by inputting the number of the stereoscopic image generating means using the keyboard.

Here, three stereoscopic image generating means that can be selected by the display image mode selecting means 602 will be described. First, the stereoscopic image generation unit A603 deletes each non-overlapping region of the two captured left and right images, and generates a stereoscopic image only from the overlapped region.

Next, in the stereoscopic image generating means B604, FIG.
As shown in the figure, an image area is added to each of the two left and right images in a direction opposite to the non-overlap area by an amount of non-overlap Δd, and arbitrary values are added to this area and the non-overlap area, for example, in the present embodiment. Then, the luminance value 0 is substituted.
In this way, a left and right image having a width of w + △ d is generated. Finally, the stereoscopic image generating means C605 adds an image area to each of the two left and right images in the direction opposite to the non-overlap area by the non-overlap amount Δd as shown in FIG. Then, the pixel value of the non-overlapping area of the other image is substituted for the image area. In this way, a left and right image having a width of w + △ d is generated.

After the viewer selects a desired display image mode in this way, a stereoscopic image can be generated by the stereoscopic image generating means corresponding to the display image mode.

The display controller 17 is controlled by the stereoscopic image display control unit 15 to switch and display these generated left and right images on one display window of the display 19 according to a vertical synchronization frequency of about 120 Hz. On the other hand, a signal synchronized with the switching of the left and right images is transmitted to the liquid crystal shutter glasses 20 to control the shuttering of the left and right fields of view. Thus, the observer can observe the stereoscopic image. FIG. 6 shows a state in which the stereoscopic images generated by the respective stereoscopic image generating means are displayed on the display. Note that a display referred to in this embodiment mode is a display that can be observed using liquid crystal shutter glasses. Here, the stereoscopic image generated by the stereoscopic image generation unit A603 is represented by a stereoscopic image generation unit B6 as shown by 1002 in FIG.
The stereoscopic image generated in 04 is displayed as 1003 in FIG. 6B, and the stereoscopic image generated in the stereoscopic image generating unit C605 is displayed as 1004 in FIG. 6C.

As described above, the observer can select a desired display image mode by the display image mode selection means 602 and observe a stereoscopic image.

In the above-described embodiment, a program including a stereoscopic imaging display program for causing a computer to function as a display image mode selecting means for selecting a desired display image mode is recorded on a medium recording the program.

In this embodiment, the display controller controls the display controller so that the left and right images are switched and displayed on the display in accordance with the vertical synchronization frequency of about 120 Hz. Although an example in which a signal is transmitted to liquid crystal shutter glasses and shuttering of the left and right fields of view is performed has been described,
The same applies to the case where the left and right images are arranged for each line in the vertical scanning direction, and each image is displayed stereoscopically by giving polarization or directivity to the display. FIG. 7 shows the system configuration diagram. Where 4
02 is a 3D display. The system configuration is almost the same as that at the time of the display by the liquid crystal shutter glasses of FIG. 5, and the same members are given the same numbers. The difference is that the display 402 is a display such as a 3D display that can have polarization or directivity according to display pixels. Accordingly, the stereoscopic image display control unit 401 arranges the left and right images on a single display window in a stripe pattern for each line in the vertical scanning direction of the display, and polarizes the display in accordance with the pixels corresponding to the left and right images. Or display with directivity. The observer can observe a stereoscopic image by wearing spectacles that transmit only the polarized light of each image display on the polarization display, and observe the screen at the center of the surface on the directional display. . Thus, similar effects can be obtained for other systems using a 3D display.

(Fourth Embodiment of the Present Invention) In a fourth embodiment of the present invention, a compound eye image input / output output by an image output device having a function of selecting and displaying two-dimensional display and three-dimensional display An example of stereoscopic imaging display by the device will be described. FIG. 8 is an explanatory diagram of the present embodiment. The compound eye imaging input / output device in the present embodiment is almost the same as that in FIGS. 5 and 7 in the third embodiment, and the same members are denoted by the same reference numerals.

However, the difference from FIGS. 5 and 7 is that, for example, a display 801 of a rear barrier wrench type which can switch between two-dimensional display and three-dimensional display in window units is used as an image output device. . The present applicant has already disclosed a rear barrier wrench type display in Japanese Patent Application Laid-Open Nos. 8-148611 and 8-14.
No. 8612 discloses this. FIG. 9A shows a display principle diagram of a rear barrier wrench type display.

The rear barrier wrench type display
It comprises a backlight 901, a checkered mask 902, a lenticular lens array A903, a lenticular lens array B904, a PDLC (polymer dispersed liquid crystal) 905, and a display LCD 906. First, light is emitted from a backlight 901 and passes through a checkered mask 902. This is to separate the left and right images from the display into irradiation light to be displayed with directivity. The separated irradiation light enters the lenticular lens array A903 and the lenticular lens array B904. In the lenticular lens array A903, lenticular lenses are vertically arranged, and when light separated by the checkered mask 902 is incident on the lenticular lens array A903, the light separated for displaying the right image is left on the display, that is, the right eye of the observer. Towards 907, the light separated for displaying the left image refracts and travels to the right with respect to the display, ie, toward the left eye 908 of the observer. Similarly, the lenticular lens array B904 has lenticular lenses arranged side by side. While the lenticular lens array A 903 separates the image in the left-right direction, the lenticular lens array B 904 plays a role of expanding the vertical viewing area. In this manner, the light is separated into light having directivity. On the other hand, the left pixel 910 and the right pixel 909 are arranged on the display LCD 906 by the stereoscopic image display control unit every other line in the vertical scanning direction. Here, the lenticular lens array A90
3. P transmitted through the lenticular lens array B904
The right pixel 909 is displayed on the right side of the viewing zone for the observer, and the left pixel 910 is displayed on the left side of the viewing zone for the observer, of the pixels arranged in a stripe shape when irradiated through the DLC 905. The observer can observe a stereoscopic image.

Next, the operation principle of the PDLC 905 will be described. As shown in FIG. 9B, the PDLC 905 has a special polymer 912 containing liquid crystal molecules 911 between electrodes 913, and the outside thereof is sandwiched between base materials 914. Here, when a voltage is applied between the electrodes, the liquid crystal molecules 911 in the special polymer 912 are in a transmission state. Therefore,
Since the left and right directional lights passing through the lenticular lens array A 903 and the lenticular lens array B 904 pass through as they are, the image on the display LCD 906 is separated to the left and right, so that the observer can observe a stereoscopic image. Can be. On the other hand, when no voltage is applied, the liquid crystal molecules 911 in the special polymer 912 are in a scattering state. Therefore, the left and right directional lights passing through the lenticular lens array A 903 and the lenticular lens array B 904 lose their directivity, and the image on the display LCD 906 is displayed as a two-dimensional image. Display switching on the PDLC 905 is controlled by a control signal from a stereoscopic image display control unit 802 in the PC 9. This switching is performed in units of windows, and a two-dimensional image and a three-dimensional image, that is, a stereoscopic image can be mixedly displayed on the display screen.

The operation of the compound eye imaging input / output device according to the present embodiment will be described. Using the compound eye imaging device 1, two left and right images are captured and input to the PC 9. After performing left-right difference correction and the like by the image processing unit 14 in the PC 9, the stereoscopic image generation unit 601 generates a stereoscopic image, which is different from the first and second embodiments. First, the observer uses the display image mode selection means 602 to select a stereoscopic image generating means A603 and a stereoscopic image generating means B60.
4. Select one of the display modes of the stereoscopic image generating means C605. The display image mode selection unit 602 can change the stereoscopic image to be generated and displayed by selecting the display image at any time during the stereoscopic imaging display by a user interface such as a keyboard and a mouse. In the present embodiment, the display image mode is selected by inputting the number of the stereoscopic image generating means using the keyboard.

Here, three stereoscopic image generating means which can be selected by the display image mode selecting means 602 will be described. The image generated and displayed for the stereoscopic image is the same as that in the third embodiment. However, the difference between the present embodiment and the third embodiment is that the processing method of the stereoscopic image generating means B604 and the stereoscopic image generating means C605 of the stereoscopic image generating unit and the stereoscopic image generated by these generating means This is a display method when is displayed by the stereoscopic image display control unit 802. In the third embodiment, the overlapping region and the non-overlapping region are mixedly displayed in one display window, and the same pixel value is used in both the left and right images in the non-overlapping region in any case. By substituting, the image is displayed so as to be seen as a two-dimensional image even when the image is displayed stereoscopically. However, in the present embodiment, another window is newly generated in the overlap area for three-dimensional display and the non-overlap area for two-dimensional display, and the stereoscopic image display control unit 802 performs two-dimensional display for each window. A control signal indicating whether there is a three-dimensional display or a three-dimensional display is transmitted to a rear barrier wrench type display 801 to perform display according to the control signal.

First, the stereoscopic image generating means A603 deletes each non-overlapping area of the two picked-up left and right images, and generates a stereoscopic image only from the overlapped area. First, the stereoscopic image generation unit B604 separates the overlap region and the non-overlap region, substitutes the data of the right image and the left image into the overlap region, and substitutes the luminance value 0 of the pixel into the non-overlap region. . Next, the stereoscopic image display control unit 802 first sets the three-dimensional display overlapping area to the stereoscopic image display control unit 80.
2 transmits a control signal 803 for transmitting liquid crystal molecules in the PDLC 905 of the rear barrier wrench type display 801 in the display, and the left and right images are displayed in a stripe shape every other line in the vertical scanning direction. Then, in the display window of the overlap area of the display 801, the left and right images arranged in a stripe pattern have respective directivities as described above, and the left and right images respectively appear on the left and right eyes of the observer. ing. 2
For the non-overlapping area for displaying the dimension, the PDLC 905 of the rear barrier wrench type display 801 is used.
The control signal 803 for setting the liquid crystal molecules in the inside to the scattering state is transmitted, and the image data to be substituted is displayed. Then, the display window of the non-overlap area is displayed as it is in two dimensions.

Similarly, the stereoscopic image generating means C605 first separates the overlap area and the non-overlap area. The data of the right image and the left image are assigned to the overlap area, and the image data is assigned to the non-overlap area. Next, a control signal 803 for three-dimensionally displaying the overlap region from the stereoscopic image display control unit 802 is
A control signal 803 for two-dimensionally displaying the non-overlap area is transmitted. Then, the PDLC 905 in the display of the rear barrier wrench type is controlled, the window displaying the overlap area is displayed three-dimensionally, and the window displaying the non-overlap area is displayed two-dimensionally as it is. Thus, the displayed image is as shown in FIG. FIG. 6 shows a state in which the stereoscopic images generated by the respective stereoscopic image generating means are displayed on the display. However, the display referred to in the present embodiment is a 2D-3D display 8 using a rear barrier wrench method.
01. Here, the stereoscopic image generated by the stereoscopic image generating unit A603 is as shown by 1002 in FIG. 6A, and the stereoscopic image generated by the stereoscopic image generating unit B604 is as shown by 1003 in FIG. 6B. The stereoscopic image generating means C6
The stereoscopic images generated in 05 are displayed as indicated by 1004 in FIG.

As described above, in the present embodiment, the compound-eye imaging input / output device freely selects the display mode of the non-overlapping region of the stereoscopic image by the display image mode selection means 602 and sets the display image mode. A stereoscopic image can be generated by the corresponding stereoscopic image generating means. In addition, a rear barrier wrench type display 801
By controlling the PDLC 905 inside, a window necessary for the display mode selected by the observer can be generated, and an appropriate display can be performed.

In the above-described embodiment, the computer switches the two-dimensional image and the three-dimensional image in the stereoscopic image in which the two-dimensional image and the three-dimensional image are mixed into display methods according to the respective methods to perform the stereoscopic imaging. A program including a stereoscopic imaging display program that functions as a unit that controls an image output device having a display unit for displaying is recorded on a medium that stores the program.

(Fifth Embodiment of the Present Invention) In a fifth embodiment of the present invention, an integrated type multi-eye image input / output device including all of a compound-eye imaging device, a PC, and an image output device will be described. FIG. 10 is an explanatory diagram of the compound eye image input / output device according to the present embodiment. The compound eye imaging input / output device according to the present embodiment is substantially the same as that of FIG. 5 in the third embodiment and the fourth embodiment, and the same members are denoted by the same reference numerals.

However, the difference from the fourth embodiment is that
The processing functions from imaging of an image with a compound eye to generation of a stereoscopic image, and output to an image output device are all in a single device, and the image output device is, for example, like a camera finder in the present embodiment. That is, the device is used. However, this finder 1101 uses a display using a rear barrier wrench method.

In this embodiment, as in the fourth embodiment, the compound-eye imaging input / output device allows the display image mode selection means 602 to freely select the display mode of the non-overlapping area of the stereoscopic image. For the selected generating means, the observer controls the PDLC 905 in the rear barrier wrench type display to generate a window necessary for the desired display mode selected by the observer, and perform an appropriate display. be able to.

However, one of the plurality of stereoscopic image generating means is selected by the display image mode selecting means, and a display range of not only a rear barrier wrench type display but also a liquid crystal shutter It is possible to use a display for glasses or other 3D displays.

[0077]

As described above, the present invention provides a non-overlapping area of a set of left and right images generated when adjusting the stereoscopic effect of a stereoscopic image in stereoscopic image display using a compound eye image input / output device. Is substituted for an arbitrary value to perform two-dimensional display, so that the observer can recognize the non-overlapping region with both the left and right eyes, and can provide an easily viewable stereoscopic image. Or the overlap area is 3
By displaying two-dimensionally and non-overlapping areas as they are two-dimensionally displayed, it is possible to provide an image in which two-dimensional display and three-dimensional display are mixed.

In a stereoscopic image pickup display using a compound-eye image input / output device, a plurality of stereoscopic image generating means can be provided and selected by a display image mode selecting means. A visual image can be observed.

Further, as an image output device having a function of selecting and displaying two-dimensional display and three-dimensional display in accordance with a window on the screen as an output device for a stereoscopic image, a stereo image including a non-overlapping area is displayed. By establishing a display method and selecting a display mode, a stereoscopic image in a different display mode can be displayed.

[Brief description of the drawings]

FIG. 1 is a block diagram of a compound-eye binocular image input / output device according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating a stereoscopic image generation method according to the first embodiment of the present invention.

FIG. 3 is a block diagram of a compound-eye image input / output device applicable to the first embodiment of the present invention.

FIG. 4 is a diagram illustrating a stereoscopic image generation method according to a second embodiment of the present invention.

FIG. 5 is a block diagram of a compound-eye image input / output device according to a third embodiment of the present invention.

FIG. 6 is a diagram illustrating a state in which a stereoscopic image generated by each stereoscopic image generating unit according to a third embodiment of the present invention is displayed on a display.

FIG. 7 is a block diagram of a compound-eye image input / output device applicable to a third embodiment of the present invention.

FIG. 8 is a block diagram of a compound-eye image input / output device according to a fourth embodiment of the present invention.

FIG. 9 is a view showing a display principle of a display using a rear barrier wrench method.

FIG. 10 is a block diagram of a compound eye image input / output device according to a fifth embodiment of the present invention.

FIG. 11 is a block diagram of a conventional compound eye image input / output device.

FIG. 12 is a diagram illustrating a state in which a stereoscopic image having a different stereoscopic effect is displayed by changing the amount of overlap. (A) is a figure which images a landscape with a compound-eye imaging device. FIG. 3B is a diagram in which a stereoscopic image is created by superimposing a pair of left and right images to the full width of a display image. (C) is a diagram in which a right image is horizontally shifted by Δd with respect to a left image to create a stereoscopic image.

[Explanation of symbols]

1, 101, 201 Compound eye imaging device 2a, 2b, 102a, 102b Imaging optical system 3a, 3b, 103a, 103b Lens 4a, 4b, 104a, 104b CCD 5, 105 Synchronization signal generator 6a, 6b, 106a, 106b A / D converter 7, 107 Memory 8, 108 Interface cable 9, 109 Personal computer (PC) 10, 110 CPU bus 11, 111 Parallel interface 12, 112 CPU 13, 113 Memory 14, 114 Image processing unit 15, 115, 401, 802 Stereoscopic image display control unit 16, 116 Storage device 17, 117 Display controller 18, 118 VRAM 19, 119 Display 20, 120 Liquid crystal shutter glasses 21, 601 Stereoscopic image generating unit 202 Landscape 2 3a, 301a, 305a, 501a, 505a
Right image 203b, 301b, 305b, 501b, 505b
Left side image 204, 302, 502 Target object 205, 206, 306, 506, 1002, 100
3, 1004 Stereoscopic image 207 Overlap area 208, 303a, 303b, 503a, 503b
Non-overlap area 304a, 304b, 504a, 504b Image area 402 3D display 602 Display image mode selection means 603 Stereoscopic image generation means A 604 Stereoscopic image generation means B 605 Stereoscopic image generation means C 801 Rear barrier wrench method 2D- 3D display 803 Control signal 901 Backlight 902 Checkered mask 903 Lenticular lens array A 904 Lenticular lens array B 905 PDLC (polymer dispersed liquid crystal) 906 Display LCD 907 Right eye 908 Left eye 909 Right pixel 910 Left pixel 911 Liquid crystal molecules 912 Special polymer 913 Electrode 914 Base material 1101 Rear barrier wrench type 2D-3D finder

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Hideki Morishima 3-30-2 Shimomaruko, Ota-ku, Tokyo Inside Canon Inc.

Claims (12)

[Claims] 1. A compound eye image input / output device having two image pickup optical systems for picking up and outputting a pair of left and right images having parallax from each other, comprising: At the time of visual imaging display, by adding new image data to each of the pair of left and right images that form a stereoscopic image including a non-overlapping region, the non-overlapping region is also overlapped so that the stereoscopic image is displayed. A compound eye image input / output device comprising a display means for imaging and displaying. 2. A medium storing a stereoscopic image display program provided in a compound-eye image input / output device which has two image pickup optical systems and picks up and outputs a pair of left and right images having parallax with each other. When the stereoscopic imaging display is performed by overlapping the set of left and right images, the program causes the computer to respectively execute the set of left and right images that form a stereoscopic image including a non-overlapping area. A medium in which a stereoscopic image display program is recorded, wherein the non-overlapping area is overlapped to function as display means for stereoscopic image display by newly adding image data. 3. The compound-eye image input / output device according to claim 1, wherein the display means for performing stereoscopic imaging display displays new ones of the pair of left and right images forming a stereoscopic image including a non-overlapping area. To the image area,
A compound eye image input / output apparatus, wherein an arbitrary value is substituted for the image area and the non-overlap area, and the non-overlap area is also overlapped and stereoscopically imaged and displayed. 4. The medium storing the stereoscopic imaging display program according to claim 2, wherein the display means for stereoscopic imaging display displays the set of right and left images forming a stereoscopic image including a non-overlapping area. Add a new image area to each image,
A medium in which a stereoscopic imaging display program is recorded, wherein an arbitrary value is substituted into the image area and the non-overlap area, and the non-overlap area is also overlapped to perform stereoscopic imaging display. 5. The compound-eye image input / output device according to claim 1, wherein the display means for stereoscopically image-capturing and displaying each of the pair of left and right images forming a stereoscopic image including a non-overlapping area is newly provided. To the image area,
A compound-eye image input / output apparatus characterized in that image data of the other non-overlapping area is substituted for one of the image areas, and the non-overlapping area is also overlapped for stereoscopic imaging display. 6. A medium on which the stereoscopic imaging display program according to claim 2 is recorded, wherein the display means for stereoscopic imaging display displays the pair of left and right images forming a stereoscopic image including a non-overlapping area. New image areas are added to both images, respectively.
A stereoscopic imaging display program which substitutes image data of another non-overlapping area into one of the two image areas and overlaps the non-overlapping area to perform stereoscopic imaging display. Medium on which is recorded. 7. The compound eye image input / output apparatus according to claim 1, further comprising: a display image mode selecting means for selecting a display means for performing stereoscopic imaging display according to claim 3 or 5. Image input / output device. 8. A medium in which the stereoscopic imaging display program according to claim 2 is recorded, said program comprising: a display image for selecting a display means for displaying a computer for stereoscopic imaging according to claim 4 or 6. A medium in which a stereoscopic imaging display program is recorded, which functions as mode selection means. 9. The method of claim 1 or claim 3 or claim 5.
8. The stereoscopic image input / output apparatus according to claim 7, wherein the two-dimensional image and the three-dimensional image in the stereoscopic image in which the two-dimensional image and the three-dimensional image are mixed are switched to a display method according to each of the three-dimensional image display. A compound eye image input / output device, comprising: an image output device having a display unit that performs the operation. 10. A medium in which a stereoscopic imaging display program according to claim 2 or claim 4 or claim 6 or claim 8 is recorded, wherein the program causes a computer to mix a two-dimensional image and a three-dimensional image. A stereoscopic imaging system, which functions as a means for controlling an image output apparatus having a display unit for stereoscopically imaging and displaying a two-dimensional image and a three-dimensional image in a stereoscopic image by switching to a display method corresponding to each of them. A medium on which a display program is recorded. 11. The compound-eye image input / output device according to claim 9, wherein the display means for switching to a display method corresponding to each of the two-dimensional image pickup and display includes a two-dimensional image portion in the stereoscopic image. 3 on the display
A compound eye bi-image input / output device characterized in that a three-dimensional image portion is displayed by switching to a three-dimensional image display. 12. A medium in which the stereoscopic imaging display program according to claim 10 is recorded, wherein the display means for performing a stereoscopic imaging display by switching to a display method corresponding to each of the two-dimensional image parts in the stereoscopic image. Is a two-dimensional image display and 3
A medium storing a stereoscopic imaging display program, wherein a three-dimensional image portion is displayed by switching to three-dimensional image display.