JP2002232913A - Double eye camera and stereoscopic vision image viewing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To facilitate adjustment of a parallax amount of each person in the case of photographing and viewing a stereoscopic vision image. SOLUTION: The stereoscopic vision image viewing system includes a parallax amount storage means that stores parallax amounts respectively proper to a photographer and a viewer of a stereoscopic vision image and each person calls the parallax amount to photograph an object and view an image with the parallax amount suitable for each person.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a stereoscopic camera for capturing a stereoscopic image and a system for observing the stereoscopic image.

[0002]

2. Description of the Related Art Conventionally, a stereoscopic imaging display method using a compound-eye imaging device has been known. This involves two imaging optics,
Images are taken from two viewpoints by arranging them on the left and right at intervals given by the base line length.

[0003] It is assumed that the left and right eyes of a human have a distance of about 65 mm on average, and the base length of two imaging optical systems is set to 65 mm in stereoscopic imaging display. As described above, when an image of a subject of interest is captured from two viewpoints on the left and right, the positions of the subjects in images captured by the respective imaging systems are different from each other. That is, this is the parallax, and by viewing this parallax in stereo, the user can see an image with a three-dimensional effect.

There are various methods for stereoscopically viewing an image obtained from two viewpoints on the left and right. One is to output the left and right images alternately to the left and right on the display, and to view the stereoscopic image by using the liquid crystal shutter glasses that switch the left and right shutters in synchronization with the switching of the display of the left and right images on the user side. What you get.

In another display method, a left and right two images are alternately arranged every other line in the horizontal direction in an area of one previously created stereoscopic image, and a striped image composed of two left and right images is formed. Create

The display screen has a polarizing plate whose polarization direction changes alternately every other line in the horizontal direction like the stereoscopic image created, and the image is displayed in a stripe shape with the polarization directions different. You.

[0007] When the stripe-shaped stereoscopic image created is displayed on this display, the image picked up by the right image pickup optical system is displayed by transmitting only polarized light in only one direction, and the image picked up by the left image pickup optical system. The displayed image is displayed by transmitting only polarized light different from the right image. on the other hand,
The user wears polarized glasses with the function of transmitting only the same polarized light as the image displayed on the display on the left and right, the right eye transmits only the polarized light where the right image is displayed, and the left eye is the left image Is designed to transmit only the polarized light indicated. Using these glasses, the user can view the right image only with the right eye and the left image only with the left eye, and the user can see an image with a three-dimensional effect.

As described above, in the stereoscopic image pickup display, the parallax of images picked up from different viewpoints is used. That is, the user superimposes two images having parallax on a focused subject (hereinafter, referred to as a main subject),
In other words, a three-dimensional image is created by fusing.

In general, when a user performs stereoscopic viewing by fusing images of two viewpoints on the left and right with respect to a main subject, fusion of the main subject can be easily performed if the parallax of the main subject between the two left and right images is small. . Therefore, when capturing an image, it is necessary to arrange the imaging optical system so that the parallax of the main subject is reduced.

Conventionally, this problem has been solved by arranging the imaging optical system with a convergence angle and by arranging the imaging optical system in parallel. FIG. 2 shows an arrangement diagram of the imaging optical system in a case where the imaging optical system is arranged without having a convergence angle, that is, stereoscopic imaging by parallel vision.

The two imaging optical systems 101a and 101b are arranged in parallel with each other at an interval given by a base line length 1 with the origin 0 as the center, and the lenses 102a and 102b, respectively.
b and an image sensor, here, CCDs 103a and 103b. Lens 102a, CCD 103a, lens 1
The distance between 02b and the CCD 103b is v. In addition, the main subject 1 is located at a position A that is z away from the origin 0 in the imaging direction.
04.

In FIG. 2, right and left CCDs 103 are shown.
The main subject 104 has a parallax d (= l · v /
z) to form an image. The image pickup optical systems 101a and 101b are provided with convergence angles so that the user can easily fuse the images, thereby reducing the parallax of the main subject.

In this figure, the angle OAB (θ) consisting of the center of the lens, here the center B of the left lens, the position A where the object 104 exists, and the origin O is given by the following equation: θ = arctan {l / (2z)} (1) is given. Here, z represents the distance between the two imaging optical system groups and the object, and l represents the base line length of the two imaging optical systems. Therefore, by rotating the two imaging optical systems by an angle θ from the straight line BC around the centers B and C of the lenses 102a and 103a as rotation centers (that is, by setting the convergence angle S of the imaging system to S = θ), The position of the main subject formed on both CCDs 103a and 103b is set to the center of the image, and parallax is set to 0.
Can be

FIG. 3 shows imaging optical systems 101a and 101 so that the parallax between two images of the main subject 104 to be imaged is set to 0.
It is the figure which arrange | positioned giving b to a convergence angle.

As described above, the imaging optical systems 101a, 101
By providing b with a convergence angle, the parallax of the main subject 104 can be set to 0 as long as there is no physical restriction such as collision of the imaging optical system.

On the other hand, as a method of disposing the imaging optical system by moving it in parallel, there is a method of reducing a base line length or the like.

FIG. 4 shows an arrangement diagram in which the base line length of the imaging optical systems 101a and 101b is reduced from 1 to 1 '. When the base line length of the imaging optical system is reduced, the parallax between the captured left and right images can be reduced.

As described above, there has been a compound-eye imaging system that adjusts the three-dimensional effect by changing the angle of convergence or moving the imaging optical system in parallel.

[0019]

However, the amount of parallax that gives a comfortable stereoscopic effect in stereoscopic vision varies greatly between individuals. For example, assume that Mr. A does not like images with a strong three-dimensional effect, whereas Mr. B likes images with a strong three-dimensional effect. In such a case, if Mr. A observes the stereoscopic image with the parallax adjusted for Mr. B, if the stereoscopic effect is too strong, or if Mr. B observes the parallax adjusted for Mr. A, conversely, In some cases, the three-dimensional effect is lacking and feels unsatisfactory.

Therefore, when a plurality of persons photograph and observe a stereoscopic image, it is necessary to adjust the amount of parallax each time so as to suit the person. An object of the present application is to facilitate adjustment of the amount of parallax of each individual when capturing and observing a stereoscopic image.

[0021]

In order to solve the above-mentioned problems, a compound-eye camera according to the present invention comprises a holding means for holding a value dependent on each photographer, and a value which depends on the photographer from the holding means. And selecting means for selecting

The compound-eye camera of the present application includes a control means for controlling an imaging system of the compound-eye camera, a holding means for holding a value dependent on each photographer, and a photographer for storing a value dependent on itself from the holding means. It has a selecting means for selecting, and enables the control means to control the imaging system of the compound-eye camera based on the value selected by the selecting means.

Further, the stereoscopic observation system of the present application has a holding means for holding a parallax amount depending on each observer of a stereoscopic image, and a selecting means for allowing the observer to select a parallax amount depending on itself from the holding means. And a position determining means for determining a position to be displayed on the display based on the amount of parallax selected by the selecting means.

Further, the compound-eye camera or the stereoscopic image observation system of the present application comprises: holding means for holding a parallax amount depending on each photographer or observer; setting means for setting a current photographer or observer; Selecting means for selecting a parallax amount depending on the current photographer or observer set by the setting means from the holding means.

Further, the compound eye camera of the present application enables a photographer to record an image at the same time as recording an image.

[0026]

Embodiments of the present invention will be described below with reference to the drawings.

(First Embodiment) FIG. 1 is a diagram showing the configuration of the present embodiment. The compound eye imaging apparatus has two imaging optical systems 1
a and 1b, each of which includes lenses 2a and 2b and CCDs 3a and 3b as image pickup devices.

The images picked up by the two image pickup optical systems 1a and 1b are sent to a signal processing section 4, where image processing such as synthesis of a stereoscopic image, image correction, and image output is performed. The signal processing unit 4 is connected to the subject position detection unit 5, the finder 6, and the external device interface 7. The finder 6 is a device that outputs an image after image correction and synthesis,
A stereoscopic image can be viewed using this. In addition, editing on an external personal computer, etc.,
Interface 7 when displaying on the display
To other external devices via.

The set parallax selecting section 9 is connected to the personal parallax information storage section 10. In the individual parallax information storage unit 10,
A parallax value that gives an appropriate stereoscopic effect is stored in a database for each individual. This database is, for example, as shown in FIG. 9, and when a photographer's name is selected, a parallax amount suitable for the photographer can be found. The set amount of parallax in FIG. 9 is a value obtained by dividing d shown in the conventional example by the pitch of the CCD, and indicates how many pixels d is in the image. The set parallax amount of Mr. A is 0. At this time, the convergence angle S may be set to θ as shown in FIG. Further, the set parallax amount of Mr. B is 10 pixels. For example, if the parallax amount when the imaging systems are parallel as shown in FIG.
The base line length 1 may be changed to 1 'as shown in FIG. Further, when the parallax amount is 30 pixels as in Mr. E, this can be dealt with by further extending the base line length.

The set parallax selecting section 9 selects a parallax amount that gives the current photographer an appropriate stereoscopic effect based on the photographer's name from the personal parallax information storage section 10 described above, and a position detecting means. Send to 5.

The subject position detection unit 5 detects the parallax of the main subject from the user interface used when the user selects the main subject in the stereoscopically displayed image and the depth of the selected main subject from the imaging optical system. And a calculation unit that calculates a convergence angle required to make the parallax selected by the set parallax selecting unit 9.

Here, when a point included in the subject of interest is designated by a pointing device such as a mouse from an image captured by the left imaging optical system displayed on the finder, a template of a certain size centered on the point is designated. A corresponding point in the right image is detected by matching using. From this set of corresponding points, the parallax at that position can be determined, and the position of the main subject, that is, the depth from the imaging optical system can be calculated from the parallax. Furthermore, the amount of convergence angle required when only the convergence angle is controlled to make the parallax of the main subject the parallax selected by the set parallax selecting means 9 is calculated. However, the method of selecting the main subject in the image is not limited to using only the user interface, and a method of automatically extracting the main subject in the image is also conceivable. Further, it is also possible to determine a point at the center of the image in advance, assuming that the main subject is at the center of the image, and adjust the parallax of that portion.

Now, it is assumed that the subject 104 is imaged as an object to be imaged by the two image pickup optical systems in FIG. Here, when the subject 104 is selected by the subject position detection unit 5 in the compound-eye imaging device of FIG. 1, similarly, the depth z from the center O of the two imaging optical systems of the selected subject 104 is detected by the subject position detection unit 5. You.

The subject position detecting section 5 is connected to the image pickup optical system driving devices 8a and 8b, and uses the position information of the main subject obtained here and the set parallax value from the set parallax selecting section 9, The convergence angles of the two imaging optical systems 1a and 1b are calculated, and the imaging optical system driving device 8 connected to each of them.
a, 8b.

The image pickup optical system driving devices 8a and 8b control the two image pickup optical systems 1a and 1b based on the amount of convergence angle transferred from the subject position detection unit 5.

Next, the operation of this embodiment will be described with reference to the flowchart of FIG. First, the user (Mr. A) selects his / her name from the personal disparity information storage unit 10 displayed on the finder 6 as shown in FIG. Selection section 9) (S51). Next, a main subject in the image is selected (S52). This method uses a user interface as shown in the configuration or is performed automatically. Subsequently, the subject position detection unit 5 calculates the position of the main subject in the left and right images, the current convergence angle of the imaging system, and the base line length.
The distance to the main subject is obtained, and a convergence angle is calculated so that Mr. A's appropriate parallax amount Da is obtained (S53). Then, the result is transferred to the imaging optical system driving devices 8a and 8b (S5).
4) The imaging optical system driving devices 8a and 8b
The convergence angles of a and 1b are controlled (S55), and the user (A
(3), a stereoscopic image that gives a comfortable stereoscopic effect to the user is photographed (S56).

Here, a method of calculating the convergence angle so as to obtain the appropriate amount of parallax Da in S53 will be described. First, when the imaging system at that time is as shown in FIG. 2, the base line length is L, the distance between the CCD and the lens is v, and the parallax of the main subject at that time is D pixels, CC
Assuming that the pitch of D is p, the distance Z to the main subject can be obtained by Expression (2).

Z = L · v / (D · p) Equation (2) Then, assuming that the distance Z to the main subject, the base line length L of the imaging system, and the appropriate amount of parallax are Da pixels, the appropriate amount of parallax Da Is 0 pixels, the convergence angle Sa of the imaging system is obtained by the formula (3), and the image is driven as shown in FIG. 3 to obtain a stereoscopic image with no parallax.

Sa = arctan {L / (2 · Z)} Equation (3) When the appropriate parallax amount Da is not 0 pixel, the base line length L
“a” is obtained by Expression (4), and driving is performed as shown in FIG. 4 to obtain a desired stereoscopic image.

La = Z · Da · p / v Equation (4) Here, when the user changes from A to B, the convergence angle Sb and the base line length Lb of the appropriate parallax amount Db are obtained again, and You can do it.

As described above, in the present embodiment, the user does not need to change the amount of parallax while viewing a stereoscopic image, and the photographer can maintain the appropriate amount of parallax for each individual in advance. Also, when the photographer changes, by selecting only the photographer's name, there is an effect that stereoscopic vision can be easily performed with a stereoscopic effect suitable for the photographer.

(Second Embodiment) FIG. 6 is a diagram showing a system according to a second embodiment. The second embodiment is a system that transfers a stereoscopic image captured by a compound-eye camera to a PC and performs stereoscopic viewing using the PC. 6, reference numeral 61 denotes a compound eye camera, 62 denotes a PC main body, 63 denotes a display, 64 denotes liquid crystal shutter glasses, and 65 denotes a screen for designating an observer. The compound-eye camera 61 captures a stereoscopic image as in the first embodiment, and transfers the image to the PC main body 62 via the external interface 7 in FIG. The PC body 62 converts the stereoscopic image from the compound-eye camera 61 into liquid crystal shutter glasses 64.
Is output to the display 63 so as to allow stereoscopic viewing.

Here, a method of stereoscopically displaying first- and left-right stereoscopic images having parallax with each other will be described. This control is performed by using software on the PC main body 62.
This software operates under the control of the CPU in the PC main body 62. First, the display controller
A setting is made so as to switch the left and right first-class stereoscopic images according to a video signal having a vertical scanning frequency of 20 Hz. Next, when the stereoscopic image from the compound-eye camera 61 is alternately output to the display 63 on the left and right images, the stereoscopic image is displayed on the display 63 at a frequency of 120 Hz. On the other hand, at the same time
The liquid crystal shutter glasses 64 connected to the C body 62 output a signal that transmits the right eye when displaying the right image on the display 63 and transmits the left eye when displaying the left image. The liquid crystal shutter glasses 64 receive a signal from the PC main body 62, and allow the liquid crystal shutter glasses 64 to pass through either the left or right, thereby enabling stereoscopic viewing.

When stereoscopic viewing is performed by such a method, a left image and a right image are alternately displayed on the display 63 as shown in FIG. The amount of parallax at one point P on the main subject 801 is represented by a shift amount between the L point existing in the left image and the R point in the right image. Although the three-dimensional effect is determined from the parallax amounts of all points on the subject, an approximate three-dimensional effect can be obtained only from the parallax amounts of the representative points.

The amount of parallax of the stereoscopic image from the compound-eye camera 61 is an amount suitable for the photographer, and the stereoscopic effect is suitable for the photographer. However, it is possible to change to the stereoscopic effect suitable for each individual by changing the above-described shift amount in the horizontal direction. That is, by controlling the position where the left and right images are output alternately in the horizontal direction, the stereoscopic effect can be changed. For example, if the left and right images are displayed apart from each other in the horizontal direction, the three-dimensional effect is enhanced.

Then, if the photographer of the stereoscopic image and the observer of the stereoscopic image are the same, the stereoscopic display is performed on the display 63 and the liquid crystal shutter glasses 64 are displayed.
However, if the photographer and the observer are different, it is possible to respond to each individual by changing the parallax amount suitable for the observer and changing the display position. However, when the photographer and the observer of the stereoscopic image are different, the setting has to be performed every time. Therefore, in the present embodiment, as shown in a screen 65 specifying the observer in FIG. 6, a parallax amount suitable for the observer who observes the stereoscopic image is held in advance, and the stereoscopic image 61 At the same time, photographer data is also input, the photographer and the observer are compared, and if they differ, the display position is calculated, and the image is displayed so that stereoscopic viewing is possible with a parallax amount suitable for the observer.

Next, the operation of this embodiment will be described with reference to the flowchart of FIG. In step 701, a stereoscopic image and photographer data are input from the compound-eye camera 61.
In step 702, a parallax value suitable for the photographer is read from the photographed human data. This has a database composed of names and proper parallax as shown in FIG. 9 shown in the first embodiment, and by using the database, a proper parallax value of a photographer of a stereoscopic image can be obtained. In step 703, an observer is selected as shown in FIG.
02, a parallax value suitable for the observer is read from the database. In step 704, the parallax value suitable for the photographer and the parallax value suitable for the observer are compared.
At 6, the display is displayed on the display 63 based on the parallax value. If they are different, calculation is performed in step 705 to change the display position of the image so as to obtain a parallax value suitable for the observer. In step 706, the image is displayed at the changed position on the display 63.

In step 705, the display position of the image is changed by calculating dt-dl when the amount of parallax suitable for the photographer is dt and the amount of parallax suitable for the observer is dl, and shifted inward by that amount. (If it is negative, move it outward).

As described above, in the present embodiment, even when the photographer of the stereoscopic image is different from the observer of the image, the adjustment of the stereoscopic effect can be easily performed, which is suitable for the observer. There is an effect that a stereoscopic image can be observed with a stereoscopic effect.

In the present embodiment, the parallax amount at one point on the main subject has been described, but this is because the main subject exists in the foreground in the normal image. In our study, it is effective to adjust the parallax amount of the subject located in the foreground in the image to the preference of the observer. When the foreground is not the main subject, the parallax amount of the main subject is reduced. In other words, it is more effective to adjust the parallax amount of the object existing at the forefront.

In this embodiment, the case where the stereoscopic image photographed by the compound-eye camera is transferred to the PC has been described. However, the photographed stereoscopic image is held in the memory of the compound-eye camera, and the stereoscopic image is converted to the compound-eye image. It is also effective when observing with a stereoscopic display on a camera. Furthermore, when observing a stereoscopic image on a display while photographing the stereoscopic image,
As described in the first embodiment, the photographer is selected, and the convergence angle and the base line length are controlled. At this time, it is also effective to change the image output position as in the second embodiment instead of the base line length control. In such a case, there is an effect that it is possible to shoot and observe a good stereoscopic image simply by selecting a photographer (observer).

[0052]

As described above, the first embodiment according to the present application is described.
According to the invention, there is an effect that the photographer can easily set a value depending on himself.

According to the second aspect of the present invention, there is an effect that the photographer can easily set a value depending on himself and control the imaging system based on the value.

According to the third aspect of the present invention, there is an effect that the photographer can easily set the amount of parallax depending on himself, and can control the imaging system based on the amount of parallax.

According to the fourth aspect of the present invention, there is an effect that the observer can easily set the amount of parallax depending on the observer and obtain a stereoscopic effect suitable for the observer.

According to the fifth invention of the present application, there is an effect that the parallax amount depending on itself can be easily set only by setting the photographer or the observer.

According to the sixth aspect of the present invention, when an image is recorded, the photographer also records the image at the same time, so that it is easy to use a database depending on the photographer.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a first embodiment.

FIG. 2 is a view for explaining stereoscopic imaging display by parallel vision;

FIG. 3 is a view for explaining stereoscopic imaging display by convergence angle control;

FIG. 4 is a diagram showing a stereoscopic imaging display by base line length control;

FIG. 5 is a flowchart for explaining the operation of the first embodiment;

FIG. 6 is a diagram illustrating a system according to a second embodiment.

FIG. 7 is a flowchart for explaining the operation of the second embodiment;

FIG. 8 is a diagram illustrating stereoscopic display using liquid crystal shutter glasses.

FIG. 9 is a diagram showing a database of a personal information storage unit.

FIG. 10 is a diagram showing a method for selecting an appropriate amount of parallax.

[Explanation of symbols]

 1a, 1b, 101a, 101b Imaging optical system 2a, 2b, 102a, 102b Lens 3a, 3b, 103a, 103b CCD 4 Signal processing unit 5 Subject position detection unit 6 Viewfinder 7 Interface 8a, 8b Imaging optical system driving device 9 Setting parallax Selection unit 10 Individual parallax information storage unit 61 Compound eye camera body 62 PC body 63 Display 64 Liquid crystal shutter glasses 65 Designation screen 104, 801 Main subject

 ────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Takeo Sakimura 3-30-2 Shimomaruko, Ota-ku, Tokyo F-term in Canon Inc. (reference) 2H059 AA10 AA13 AA18 5C061 AA02 AB04 AB08 AB17 AB20

Claims (6)

[Claims] 1. A compound-eye camera having a plurality of imaging systems, comprising: holding means for holding a value dependent on each photographer; and selecting means for the photographer to select a value dependent on the photographer from the holding means. Compound eye camera characterized by the following. 2. A compound-eye camera having a plurality of image-capturing systems, a control means for controlling the image-capturing system of the compound-eye camera, a holding means for holding a value dependent on each photographer, and A compound-eye camera having a selection unit for selecting a dependent value, wherein the control unit controls an imaging system of the compound-eye camera based on the value selected by the selection unit. 3. The compound eye camera according to claim 1, wherein the value dependent on each photographer held in the holding means is a parallax amount of a stereoscopic image. 4. A stereoscopic observation system for displaying a stereoscopic image on a display and performing stereoscopic vision, comprising: holding means for holding a parallax amount dependent on each observer of the stereoscopic image; A stereoscopic image characterized by comprising: selecting means for selecting a parallax amount depending on itself from the means; and position determining means for determining a position to be displayed on a display based on the parallax amount selected by the selecting means. Observation system. 5. A compound-eye camera for capturing a stereoscopic image, a stereoscopic image observation system for observing a stereoscopic image, or a compound-eye camera having a stereoscopic display, which holds a plurality of parallax amounts depending on each photographer or observer. Holding means, setting means for setting a current photographer or observer, and selecting means for selecting a parallax amount depending on the current photographer or observer set by the setting means from the holding means. A compound eye camera or a stereoscopic image observation system characterized by the following. 6. A compound-eye camera having a plurality of image pickup systems, wherein a photographer simultaneously records an image when recording an image.