JPH1066106A - Double-eye camera system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a double-eye camera system that can display whichever image that is picked up with a compound camera is, a stereoscopic picture or a panoramic image without an operator being aware of it or also changing methods for display means. SOLUTION: This system consists of two image-pickup optical parts 1, 1a, 2 and 2a which converge light of an object to be picked up and form an optical image on image-forming planes, camera head parts which have image-pickup devices 7 and 8 that are arranged at respective image-forming planes of the parts 1, 1a, 2 and 2a, overlapped amount operating means 17 to 19 which operate and store the overlapped amount of an image-pickup area of plural image signals that are obtained by the camera head parts, and display means 21 and 22 which show the image signals that are obtained by the camera head parts, and the means 21 and 22 alternately and time sequentially show plural image signals that are obtained by the camera head parts based on the overlapped amount, so that the overlapped amount may overlap two image signals.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a compound eye camera system having two or more imaging optical systems.

[0002]

2. Description of the Related Art Conventionally, there has been a compound eye camera having a plurality of image pickup systems including an image forming optical system and an image pickup device and capable of taking a stereoscopic image. In recent years, for the purpose of generating a wide panoramic image or a high-definition image as disclosed in JP-A-6-6680 or the like, an image signal obtained by each image sensor of a compound-eye camera has been used. A compound eye imaging device that generates two composite images has been proposed.

[0003]

However, when an image captured by such a compound eye camera is reproduced, there is no means for determining whether the image is a stereoscopic image or a panoramic image. Therefore, for example, there is a possibility that a problem that a panoramic image is reproduced by a reproducing unit that reproduces a stereoscopic video may occur.

[0004] Further, if a conventional compound-eye camera attempts to perform panoramic photographing on a relatively short-distance subject, an image obtained by each photographing element may cause a panoramic photographing due to a difference in the position of each photographing system. There is a problem that the region where the subject is not photographed is different. FIGS. 13 to 16 are diagrams for explaining this problem. FIG. 13 is a first diagram illustrating a positional relationship between a subject and a viewpoint. FIG. 14 is a diagram showing how the subject looks at each viewpoint position in FIG. FIG. 15 is a second diagram illustrating the positional relationship between the subject and the viewpoint. FIG. 16 is a diagram showing how the subject looks at each viewpoint position in FIG.

In FIG. 13, reference numerals 131 to 133 are:
Each shows a spherical subject. Reference numerals 134 and 135
Indicates viewpoint positions at which the subjects 131 to 134 are respectively viewed. FIGS. 14A and 14B are views showing the states of the subjects 131 to 133 viewed from the viewpoint positions 134 and 135, respectively. First, when the subjects 131 to 133 are viewed from the viewpoint position 134, as shown in FIG.
The subject 132 is partially hidden by the subject 131. On the other hand, from the viewpoint position 135, the subjects 131-1
When the object 33 is viewed, as shown in FIG.
Is not hidden by the subject 132,
It can be seen that 3 is in the shadow of the subject 131 and is no longer visible. From this, even if an attempt is made to synthesize a panoramic image by superimposing two overlapping images of two images taken by partially overlapping the imaging regions from different viewpoint positions, the overlapping regions of the two images do not match. It can be seen that it is difficult to combine images so that the joint between the overlapping regions of the two images is not noticeable.

When a three-dimensional subject is photographed at a short distance, there is a problem that the shape of the subject itself looks different depending on the viewpoint position. FIG. 15 and FIG. 16 are diagrams for explaining this problem.
Reference numeral 6 denotes a subject having a cubic shape. Also, reference numeral 1
57 and 158 indicate viewpoint positions. And FIG.
6 (a) and 6 (b) show the viewpoint positions 157 and 158, respectively.
FIG. 6 is a diagram showing a state of a subject 156 viewed from the side. First,
When viewing the subject 156 from the viewpoint position 157, FIG.
The subject 156 looks substantially square as shown in FIG. On the other hand, when the subject 156 is viewed from the viewpoint position 158, FIG.
It looks like two trapezoids sharing a part as in (b). From this, even if an attempt is made to synthesize a panoramic image by superimposing two overlapping images of two images captured by overlapping a part of the imaging region from different viewpoint positions, the subject in the overlapping region of the two images is Since the shapes look different, it can be seen that these two images cannot be superimposed.

As described above, even when panoramic image capturing is performed by a conventional compound-eye image capturing apparatus, the above-described problem often occurs due to a difference in the image capturing position between the left and right image capturing systems, and the panoramic image synthesis is often not successful. Note that such a difference between the photographing positions of the left and right photographing systems will be referred to as “parallax” hereinafter. Therefore, with the conventional compound-eye camera, in order to avoid the problem of parallax, panoramic photography can be performed only for a distant subject such as a landscape.

By the way, in order to solve the problem of the parallax, in order to solve the above problem, within the common visual field of the first and second optical systems, the image forming position of each object point by distance within a predetermined photographing distance range. The two optical systems may be arranged such that the position of the center of gravity of the intensity of the light beam captured by the optical system from the object point is unchanged on the imaging surface.

[0009] If the aberration of the optical system is so great as to affect the difference between the photographed images, there is a problem in image quality. In a realistic optical system, the aberration can be considered to have no effect on parallax, so that the center of the entrance pupil, which is the image of the stop that restricts the spread of the light beam incident on the optical system, substantially matches in the first and second optical systems. With such an arrangement, the two optical systems can be arranged so that the respective image forming positions of the object point according to the distance in the common visual field are not changed.

When the entrance pupil of the optical system and the object-side principal point are located close to each other, the object-side principal points of the first and second optical systems are substantially matched. By arranging, the two optical systems can be arranged so that the respective imaging positions of the object points according to the distance in the common visual field are not changed.

From the above, the influence of the parallax can be reduced by reducing the center of the entrance pupil of each of the first and second imaging optical units, that is, the interval between the intersection points of the entrance pupil and the optical axis. is there.

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and a first object of the present invention is to provide an image processing apparatus for an operator, regardless of whether an image captured by a compound-eye camera is a stereoscopic video or a panoramic video. It is an object of the present invention to provide a compound-eye camera system capable of performing display without being conscious of it and without changing the method of the display means.

A second object of the present invention is to allow an operator to be conscious of an image taken by a compound-eye camera having two or more imaging optical systems, whether it is a stereoscopic image or a panoramic image. Instead, to provide a compound-eye camera system capable of displaying.

A third object of the present invention is to allow an operator to be conscious of whether a picture taken by a compound-eye camera capable of taking both a stereoscopic picture and a panoramic picture is either a stereoscopic picture or a panoramic picture. It is an object of the present invention to provide a compound-eye camera system capable of performing display without any problem.

A fourth object of the present invention is to make it easy to determine whether an image captured by a compound-eye camera capable of capturing both a stereoscopic image and a panoramic image is a stereoscopic image or a panoramic image. is there.

A fifth object of the present invention is to provide a compound-eye camera system capable of more accurately synthesizing a panoramic image captured by a compound-eye camera.

A sixth object of the present invention is to quickly display a stereoscopic image or a panoramic image in a compound-eye camera system from the first object to the fifth object.

A seventh object of the present invention is to alleviate the problem of parallax at the time of panoramic photographing in the compound eye camera systems of the second to fourth objects.

[0019]

In order to achieve the above object, according to the present invention, there are provided two imaging optical units for converging subject light to form an optical image on an image forming surface; A camera head unit having an image sensor arranged on each image plane of the optical unit, and an overlap amount calculation for calculating or storing an overlap amount of a photographing region of a plurality of video signals obtained by the camera head unit And a display unit for displaying a video signal obtained by the camera head unit, wherein the display unit converts the plurality of video signals obtained by the camera head unit into two images based on the overlap amount. The signals are alternately displayed in time series so that the overlap amounts overlap.

According to the second aspect of the present invention, at least two image pickup optical units for converging subject light to form an optical image on an image forming surface, and arranged on each image forming surface of the at least two image pickup optical units. A camera head unit having an image sensor, an overlap amount calculating unit for calculating or storing an overlap amount of a photographing region of a plurality of video signals obtained by the camera head unit, and an overlap amount calculation unit obtained by the camera head unit. Display means for displaying a plurality of video signals, wherein the display means reproduces the plurality of video signals obtained by the camera head unit, and reproduces the plurality of video signals obtained by the camera head unit. When the overlap amount is larger than a predetermined amount, the plurality of video signals are displayed as a stereoscopic image, and when the overlap amount is smaller than a predetermined amount, a previous image signal is displayed based on the overlap amount. It is obtained so as to display a plurality of video signals as one continuous image.

According to a third aspect of the present invention, at least two imaging optical units for converging subject light to form an optical image on an imaging surface, and arranged on each imaging surface of the at least two imaging optical units. The at least two imaging optical units and the imaging device, a first position where the imaging optical axes of the at least two imaging optical units are substantially parallel, and the at least two imaging optical units and A camera head unit movably provided at a second position in which an amount of overlap of a shooting region shot by an image sensor is smaller than the first position; and a plurality of video signals obtained by the camera head unit. And a display unit for displaying a plurality of video signals obtained by the camera head unit. The display unit includes a camera. When playing back a plurality of video signals obtained by the head unit, when shooting is performed at the first position, the plurality of video signals are played back as a stereoscopic video, and shooting is performed at the second position. When the plurality of video signals are displayed, the plurality of video signals are reproduced as one continuous video based on the overlap amount.

According to a fourth aspect of the present invention, when the display means reproduces a plurality of video signals obtained by the camera head unit, the display unit displays the overlap amount of the plurality of video signals obtained by the camera head unit. When the amount is larger than a predetermined amount, the plurality of image signals are displayed as a stereoscopic image, and when the amount of overlap is smaller than a predetermined amount, the plurality of image signals are displayed as one continuous image based on the amount of overlap. It is intended to be displayed.

According to a fifth aspect of the present invention, the overlap amount calculating means includes a focal length and convergence angle information of at least two photographing optical systems of the camera head unit, and a first overlap amount obtained from subject distance information. A second overlap amount calculated from a plurality of video signals obtained by the camera head unit is calculated, and at least the second overlap amount is obtained when the plurality of video signals obtained by the camera head unit is a panoramic video. The display on the display means is performed based on the lap amount.

According to a sixth aspect of the present invention, in the compound eye camera system according to any one of the first to fifth aspects, the storage means for storing the plurality of video signals obtained by the camera head and the amount of overlap is provided. It had.

In the invention according to claim 7, when the at least two imaging optical units are at the second position, the distance between respective object-side principal points of the at least two imaging optical units, more specifically, the at least The distance between the intersection of the exit pupil of each of the two imaging optical units and the optical axis is smaller than at least when the imaging optical unit is at the first position.

[0026]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to first to fifth embodiments.

(First Embodiment) FIG. 1 is a block diagram showing a configuration of a compound-eye camera system according to a first embodiment of the present invention. Shooting lens 1, focus lens 1
a is housed in a lens barrel 3, and the photographing lens 2 and the focus lens 2 a are housed in a lens barrel 4.
Is fixed to a camera body (not shown) so as to be rotatable about the shafts 5 and 6. The focus lenses 1a and 2a are movable in the optical axis direction by a focus adjustment mechanism (not shown). Image sensors 7 and 8 for converting optical images formed by the photographing lenses 1 and 2 into corresponding electric video signals are disposed behind the focus lenses 1a and 2a in the lens barrels 3 and 4, respectively. These imaging elements 7 and 8 are photoelectric conversion elements. Note that the configurations up to this point are collectively referred to as a “compound eye camera unit” hereinafter. Also, in the figure, positions of intersections of the entrance pupils of the photographing lenses 1 and 2 and the optical axis are A and B, respectively,
2 are indicated by C and D. Lens barrel driving means 1
Reference numerals 1 and 12 rotate the lens barrels 3 and 4 about the axes 5 and 6, respectively, and also serve as a means for detecting a convergence angle which is an intersection angle of the photographing optical axes of the photographing lenses 1 and 2. Release button 13
Generates a release signal when operated. When operated, the overlap amount changing means 14 operates the lens barrel driving means 11 and 12 to change the overlap amount (area) of the photographing range of each of the photographing lenses 1 and 2. Note that the photographing lenses 1 and 2 of the compound-eye camera unit in FIG. 1 have optical axes substantially parallel to each other.
Intersection points A and B between the entrance pupils of these photographing lenses 1 and 2 and the optical axis
Are also very different. Therefore, by reproducing the image shot in this state, an image obtained by shooting the subject from the left side can be viewed only by the viewer's left eye, and an image obtained by shooting the subject from the right side can be viewed only by the viewer's right eye. Thus, a so-called three-dimensional image in which the subject looks three-dimensional can be reproduced.

FIG. 2 is a diagram showing an operation example of the compound eye camera unit of FIG. That is, FIG. 2 is a diagram showing a state in which the lens barrels 3 and 4 are rotated about the axes 5 and 6 in a direction in which the amount of overlap of the imaging ranges of the imaging lenses 1 and 2 decreases. As can be seen from FIG. 2, the axes 5 and 6 correspond to the photographing lens 1,
2 is closer to the object than the intersections A and B between the entrance pupil and the optical axis. When the lens barrel is rotated three or four times in a direction to reduce the amount of overlap, the entrance pupils of the photographing lenses 1 and 2 are The distance between the intersection points A and B with the axis becomes smaller,
It can be seen that the parallax of the image captured in Step 2 is also reduced.

Returning to FIG. 1, the photometric means 15 measures the brightness of the subject and outputs a corresponding electric signal, and the focus detecting means 16 detects the in-focus state of the photographing lenses 1 and 2 and responds. Outputs electrical signals. The signal processing unit 17 converts an image signal obtained by the imaging elements 7 and 8 into a predetermined video signal and performs correction. Overlap amount calculating means 18
Calculates the amount of overlap between the images captured by the imaging lenses 1 and 2 from the focal length, the convergence angle, and the subject distance of the imaging lenses 1 and 2. The memory 19 stores image data and the like obtained by the signal processing means 17. The system controller 20 controls the entire compound-eye camera. The display calculation unit 21 reads out the video signal stored in the memory 19 and generates a video signal to be displayed on a display unit such as a monitor described later. The monitor 22 displays the video signal generated by the display calculation means 21.

Next, the photographing operation will be described. FIG.
FIG. 4 is a flowchart showing a shooting operation flow of the compound eye camera system of FIG. Unless otherwise specified, all operations are performed by the system controller 20. In FIG. 3, first, when a power switch of a compound-eye camera system (not shown) is turned on (step S100), a shooting standby state is set (steps S101 and S102). When changing the overlap amount in this shooting standby state,
By operating the overlap amount changing means 14, the process moves to step S111. Then, step S11
In step 1, the lens barrels 3 and 4 are rotated about the axes 5 and 6 in response to the operation of the overlap amount changing means 14 to change the overlap amount of the photographing areas of the photographing lenses 1 and 2. If the overlap amount is not changed in step S101, or if the overlap amount is changed to a desired overlap amount (area) in step S111, the release button 13 is operated to generate a release signal (step S102). ), The brightness of the subject is measured by the photometric unit 15, and the aperture and shutter speed are determined according to the photometric value (step S103). When the release button 13 is not operated, the process returns to step S101.

Next, the focus state of the subject is detected by the focus detecting means 16. If the subject is out of focus, the focus lenses 1a and 2a are moved in the optical axis direction by a focus adjustment mechanism (not shown). Then, focus adjustment is performed (step S104). next,
Exposure to the imaging elements 7 and 8 is performed with the aperture value and shutter speed determined in step S103,
The two optical images are stored as electrical image signals in the imaging devices 7 and 8 (step S105). Next, the overlap amount calculating means 18 calculates the overlap amount between the two images shot at this time, the focal length of the shooting lenses 1 and 2,
It is calculated from the distance to the photographed subject and the angle between the photographing optical axes of the photographing lenses 1 and 2 (step S10).
6). Next, the image signals accumulated in the imaging elements 7 and 8 are
Predetermined signal processing is performed by the signal processing means 17 (step S107). Further, the signal processing unit 17 also corrects the trapezoidal distortion of each of the image signals obtained by the imaging devices 7 and 8 (Step S108).

Here, the trapezoidal distortion will be described. FIG.
FIG. 2 is a diagram illustrating the principle of trapezoidal distortion. FIG.
Are the imaging optical system L (barrel 4 side) and R (barrel 3 side) of FIG.
2 shows an imaging optical system having an equivalent photographing range. In FIG. 12, the imaging optical system includes a photographing lens 30 and imaging elements 31 and 32. As can be seen from FIG. 12, the image pickup device 31 and the image pickup device 30 have the same angle of view as the image pickup optical systems L and R.
It can be seen that the reference numerals 32 are mutually inclined according to the intersection angle of the optical axes of the imaging optical systems L and R in FIG. This is because, as described above, in the first embodiment, the imaging system L (the lens barrel 4)
This is because the optical axes of the imaging system R (on the side of the lens barrel 3) intersect at a predetermined intersection angle, and are arranged obliquely with respect to the center subject. Therefore, the image formed by the photographing lens 30 does not match the image formed on one plane indicated by the dotted line S, and becomes a distorted image. This is called trapezoidal distortion. Therefore, in the present embodiment, the trapezoidal distortion is corrected, and the images captured by the imaging optical systems L and R are reduced by one imaging lens 30 having the same imaging range of the imaging optical systems L and R shown in FIG. To be equivalent to an image formed on two planes.

Referring back to FIG. 3, the signal processing means 17 converts the image signal subjected to the trapezoidal distortion correction into a predetermined video signal (step S108), and thereafter, the overlap amount and the identification indicating that they are paired with each other. Photographing information such as a signal and photographing conditions is added (step S109). Then, the video signal generated by the signal processing unit 17 is recorded (stored) in the memory 19 (step S110). Thus, the photographing operation ends.

Next, the operation of reproducing a photographed image will be described. FIG. 4 is a flowchart showing a reproducing operation of the compound-eye camera system, and FIG. 5 is a diagram showing a state of recording and displaying an image. It is assumed that a set of images 25 and 26 as shown in FIG. In FIG. 5A, an image 25 indicates a left image, and an image 26 indicates a right image. The hatched portions 25 'and 26' indicate the amount of overlap (region) between the left image 25 and the right image 26, respectively. FIG.
Returning to, first, when the play button of the compound-eye camera system (not shown) is operated (step S121), the display calculation means 21 makes the set of images 25 and 26 recorded in the memory 19.
And the left image 25 is first displayed on the monitor 2
2 (step S122), and then the right image 2
6 is displayed on the monitor 22 while being offset with respect to the left image 25 according to the overlap amount (step S123). Thereafter, the operations of step S122 and step S123 are repeated, and the left image 25 and the right image 26 are alternately displayed. FIG. 5B shows a display state of the monitor 22 at this time. As can be seen from this figure, the right image 26 is displayed at a position where the overlap region of the left image 25 just overlaps. You can see that. However, as described above, since the left image 25 and the right image 26 are displayed on the monitor 22 in a time-division manner,
They are not actually displayed at the same time.

Then, the user shields this image from the right eye when the left image 25 is displayed, and shields the left eye from the right image 26 when the image 26 is displayed. By looking through the glasses with the liquid crystal shutter, the overlapping areas 25 'and 26'
Can be viewed as a stereoscopic image. If this image is to be used as a panoramic image, the left image 25 and the right image 26 may be photographed so that the overlap areas 25 'and 26' become smaller, and similar display may be performed. At this time, the user may or may not use the glasses with the liquid crystal shutter.

As described above, in the invention of the first embodiment,
The following effects are obtained.

(1) The amount of overlap of images captured by the plurality of photographing lenses 1 and 2 is calculated from the focal length, the convergence angle, and the subject distance of the photographing lenses 1 and 2, thereby calculating Is simpler, and the calculation can be started from the time when the shooting conditions are determined, which is advantageous for continuous shooting of moving images and still images.

(2) Since the images 25 and 26 photographed by the photographing lenses 1 and 2 are recorded in the memory 19 in a state where processing such as synthesis is not performed, the images recorded in the memory 19 are The present invention is not limited to the reproducing method according to the present embodiment, and it is possible to easily cope with reproducing by various reproducing methods.

(3) Since the photographing lenses 1 and 2 are provided so as to be rotatable by the shafts 5 and 6 and to be able to stop at arbitrary positions, a three-dimensional object having an arbitrary aspect ratio according to the photographer's intention is provided. It is possible to shoot video and panoramic video.

Although the first embodiment has described shooting and display of a still image, the present invention is not limited to this, and can be similarly applied to a moving image. Needless to say, there is.

Further, the overlap calculating means 18 may selectively read out overlap data stored in advance based on the focal length, the convergence angle and the subject distance of the photographing lenses 1 and 2. In this case, since it is not necessary to calculate the overlap amount, there is an advantage that the overlap amount can be obtained more quickly.

Further, in the first embodiment, the trapezoidal distortion correction is performed at the time of shooting, but the trapezoidal distortion correction may be performed at the time of reproduction. In this case, there is an advantage that the processing time of the signal processing unit 17 can be shortened, and higher-speed continuous shooting can be performed.

(Second Embodiment) The second embodiment relates to a reproducing operation of an image photographed by the compound eye camera system shown in the first embodiment.

FIG. 6 is a flowchart showing a reproducing operation flow of the compound-eye camera system according to the second embodiment. The configuration of the compound-eye camera system is the same as that of the first embodiment. Also, in the memory 19,
As in the first embodiment, it is assumed that a pair of images 25 and 26 as shown in FIG.

First, when a play button (not shown) of the compound-eye camera system is operated (step S201), the display calculation means 21 sets a set of images 25 stored in the memory 19,
26 is read out, and the amount of overlap calculated by the overlap calculating means 18 in the photographing information associated therewith is compared with a predetermined value (step S202). And
If the overlap amount is larger than a predetermined value, the first
As shown in FIG. 5B, as in the case of the first embodiment,
First, the left image 25 is displayed on the monitor 22 (step S203), and then the right image 26 is offset on the left image 25 according to the overlap amount and displayed on the monitor 22 (step S204). ). Thereafter, the operations of step S203 and step S204 are repeated,
The left image 25 and the right image 26 are displayed alternately.
Then, as in the case of the first embodiment, the user shields the right eye side when the left image 25 is displayed and shields the left eye side when the right image 26 is displayed. By looking through the glasses with the liquid crystal shutter, the overlap area 25 ', 2
6 'can be viewed as a stereoscopic image. If the overlap amount is smaller than the predetermined value in step S202, as shown in FIG. 5 (c), the display calculation means 21 performs the overlap operation on the right image 26 of the set of images 25 and 26. The wrap portion 26 'is deleted (step S
205), as shown in FIG. 5D, the overlapping is performed so that the overlapping portion 25 'of the left image 25 complements the deleted overlapping portion 26' of the right image 26.
After generating two horizontally long video signals, that is, a panoramic video (step S206), these are displayed on the monitor 22 (step S207).

As described above, in the second embodiment, when displaying a panoramic image, a plurality of images 25 and 26 are combined into one image, so that it is easy to print the displayed image. .

In the second embodiment, step S2
In 02, the predetermined value to be compared with the overlap amount is fixed, but this predetermined value may be changed as appropriate according to the user's preference. In this case, the user
There is an advantage that the user can switch the reproduction between the stereoscopic video and the panoramic video by himself.

(Third Embodiment) The third embodiment relates to a reproduction operation of an image photographed by the compound eye camera system shown in the first embodiment.

FIG. 7 is a flowchart showing a reproducing operation flow of the compound-eye camera system according to the third embodiment. The configuration of the compound-eye camera system is the same as that of the first embodiment. Also, in the memory 19,
As in the first embodiment, it is assumed that a pair of images 25 and 26 as shown in FIG.

First, when a play button (not shown) of the compound-eye camera system is operated (step S301), the display calculation means 21 sets a set of images 25 stored in the memory 19,
26, and compares the overlap amount (hereinafter, this overlap amount is referred to as "first overlap amount") calculated by the overlap calculation means 18 in the accompanying photographing information with a predetermined value (step S30).
2). When the first overlap amount is larger than the predetermined value, it is determined that the set of images 25 and 26 is a stereoscopic image, and as in the case of the first embodiment, FIG.
As shown in (b), first, the left image 25 is displayed on the monitor 22 (Step S303), and then the right image 26 is offset with respect to the left image 25 according to the first overlap amount. 22 (step S3
04). Thereafter, the operations of step S303 and step S304 are repeated, and the left image 25 and the right image 26 are alternately displayed.

The user is provided with a liquid crystal shutter having a configuration in which the right eye is shielded from light when the left image 25 is displayed, and the left eye is shielded from light when the right image 26 is displayed. By looking through the glasses, it is possible to view the overlap areas 25 'and 26' as a stereoscopic image. If the first overlap amount is smaller than the predetermined value in step S302, it is determined that the set of images 25 and 26 is a panoramic video, and it is further determined whether it is a still image or a moving image. (Step S305).

When the set of images 25 and 26 is a still image, the overlap areas 25 'and 26' based on the first overlap amount of each of the set of images 25 and 26 are provided. , Corresponding points are extracted (step S306). Then, from the corresponding points of the respective images obtained by the corresponding point extraction, a new set of images 25,
A more accurate overlap amount is calculated (step S307). (Hereinafter, this overlap amount is referred to as “second overlap amount”). Then, as shown in FIG. 5C, the overlap portion 26 ′ based on the second overlap amount is excluded from the right image 26 (step S3).
08), and as shown in FIG. 5D, the overlap section 25 ′ based on the second overlap amount of the left image 25.
Are connected so as to supplement the deleted overlap portion 26 'of the right image 26, and generate one horizontally long video signal, that is, a panoramic video (step S309).
This is displayed on the monitor 22 (step S310).

If it is determined in step S305 that the set of images 25 and 26 is a moving image, the image 25 and the set of images 25 and 26 are overlaid on the right side of the set of images 25 and 26 based on the first overlap amount. The wrap portion 26 'is deleted (step S311), and the left image 25 is removed as shown in FIG.
Of the right side image 26 so as to compensate for the deleted overlap section 26 'to generate one horizontally long video signal, that is, a panoramic video (step S312). 22 (step S313). Hereinafter, the operation from step S311 to step S313 is repeated every time the next image is input.

As described above, in the invention of the third embodiment,
The following effects are obtained.

(1) The captured video is a panoramic video,
In addition, since the display is performed using the second overlap amount only in the case of a still image, it is not necessary to increase the speed of calculating the second overlap amount so much that the second overlap amount is calculated. The operation circuit that performs the calculation can be configured at low cost.

(2) Since the second overlap amount is calculated by limiting the range in which the corresponding point is extracted based on the first overlap amount, the calculation speed is high.

(Fourth Embodiment) FIG. 8 is a block diagram showing the configuration of a compound-eye camera system according to the fourth embodiment of the present invention.

In FIG. 8, a photographing lens 51 and a focusing lens 51a are housed in a lens barrel 53,
The focus lens 52a is housed in a lens barrel 54, and the lens barrels 53 and 54 are fixed to a camera body (not shown) so as to be rotatable about a shaft 55 and a shaft 56 (shown by dotted lines). The focus lenses 51a and 52a are movable in the optical axis direction by a focus adjustment mechanism (not shown). Focus lenses 51a, 52a are provided in the lens barrels 53, 54.
Image sensors 57, 58 for converting the optical images formed by the photographing lenses 51, 52 behind to the corresponding electric video signals.
Is arranged. These imaging elements 7 and 58 are, for example, photoelectric conversion elements. Reflecting mirrors 59, 60
Are arranged in front of the photographing lenses 51 and 52, respectively, and guide the subject light flux to the photographing lenses 51 and 52. Note that the configurations up to this point are collectively referred to as a “compound eye camera unit” hereinafter. In the present embodiment, it can be seen that the thickness in the shooting direction can be reduced because the compound eye camera section has the above-described configuration.

In the figure, the positions of the intersections between the entrance pupils of the photographing lenses 51 and 52 and the optical axis are indicated by A and B, respectively.
The photographing light beam 52 is indicated by C and D. Points E and F indicate the symmetrical positions of the intersections A and B between the entrance pupils of the photographing lenses 51 and 52 and the optical axis when the reflecting surfaces of the reflecting mirrors 59 and 60 are symmetrical, that is, the virtual image positions. That is, in the state of FIG.
52 is equivalent to a case where the intersections A and B between the entrance pupils of the respective photographing lenses 51 and 52 and the optical axis coincide with the positions of the virtual image positions E and F, and the respective photographing ranges C and D are photographed. It can be seen that the distance between the intersections A and B between the entrance pupils of the photographing lenses 51 and 52 and the optical axis is very small. Therefore,
The parallax described above hardly occurs in the images photographed by the photographing lenses 51 and 52 even for a subject at a short distance.

The lens barrel driving means 61 and 62 include the lens barrels 53 and 5
4 is rotated about the shafts 55 and 56.
The release button 63 generates a release signal when operated. The mode switching means 64 includes the photographing lens 5
A stereoscopic image is formed from a panoramic shooting mode in which images having no subject change due to the difference in shooting positions 1 and 52 are combined to obtain one horizontally long image, and a video having a subject change due to the difference in shooting positions of the shooting lenses 51 and 52. Switching between two photographing modes, that is, a stereoscopic photographing mode for enabling reproduction, is performed.
Note that the state of the compound-eye camera unit in FIG. 8 is a state of the panorama shooting mode.

FIG. 9 is a diagram showing an operation example of the compound-eye camera unit of FIG. The compound-eye camera unit in FIG. 9 shows a state of the stereoscopic imaging mode. As can be seen from FIG. 9, in the stereoscopic imaging mode, the optical axis of each imaging optical system is
They are substantially parallel, and the positions of intersections A and B between the entrance pupils of the imaging lenses 51 and 52 and the optical axis are also different. Therefore, by reproducing an image taken of the subject from the left side only to the viewer's left eye and an image taken of the subject from the right side only to the viewer's right eye, the so-called three-dimensional Images can be played.

Returning to FIG. 8, the photometric means 65 measures the brightness of the subject and outputs a corresponding electric signal, and the focus detection means 66 detects the in-focus state of the photographing lenses 51 and 52 and responds. Outputs electrical signals. The signal processing means 67
The image signals obtained by the imaging elements 57 and 58 are converted into predetermined video signals according to the shooting mode. The overlap amount calculating means 68 calculates and outputs a different overlap amount according to the shooting mode and the subject distance. In the fourth embodiment, it is assumed that, as the amount of overlap, a value obtained by actually measuring the amount of overlap at each shooting mode and subject distance is stored in advance. The memory 69 stores the image data and the like obtained by the signal processing means 67. The system controller 70 controls the entire compound-eye camera. The display calculation unit 71 generates a video signal to be displayed on a display unit such as a monitor described later from the video signal stored in the memory 69 and the video signal generated by the signal processing unit 67. The monitor 72 displays the video signal generated by the display calculation means 71, and also serves as a finder for determining the composition of the subject.

Next, the operation will be described. FIG. 10 is a flowchart showing an operation flow of the compound eye camera system of FIG. Unless otherwise specified, all the operations are performed by the system controller 70. 10, first, when a power switch of a compound eye camera system (not shown) is turned on (step S400), the brightness of the subject is measured by the photometric unit 65, and the aperture and the shutter speed are determined according to the photometric value (FIG. 10). Step S401). Next, the focus state of the subject is detected by the focus detecting means 66, and when the subject is not focused, the focus lenses 51a and 52a are moved in the optical axis direction by a focus adjustment mechanism (not shown). Adjustment is performed (step S402). Next, step S40
Image sensor 5 with aperture value and shutter speed determined in 1
Exposure is performed on the imaging lenses 7 and 58, and the optical images of the imaging lenses 51 and 52 are stored as electrical image signals in the imaging devices 57 and 58 (step S403). Next, the signal processing means 67
Converts the video signals obtained by the imaging devices 57 and 58 into predetermined video signals (step S404).

Next, the current photographing mode is detected by the lens barrel driving means 61 and 62 (step S405). Then, when the shooting mode is the panoramic shooting mode, the overlap amount calculating means 68, based on the current focus position,
The overlap amount A is output (step S406).
Then, the display calculation means 71 displays the video signal generated by the signal processing means 67 on the monitor 72 based on the overlap amount A in a manner similar to the panoramic video display method shown in the second embodiment. (Step S407). If the current shooting mode is the stereoscopic shooting mode in step S405, the overlap amount calculator 68 outputs the overlap amount B based on the current focus position (step S408). Next, the display calculation means 71 displays only the video signal obtained by the image sensor 57 on the monitor 72 (step S409). By the operation up to this point, the user can confirm the current shooting mode by seeing the image displayed on the monitor 72.

Here, when the user wants to switch the photographing mode, the user can switch the mode by operating the mode switching means 64 (step S41).
0). When the mode switching unit 64 is operated, the process proceeds to step S416, where the lens barrel driving units 61 and 62
When the current state of the lens barrels 53 and 54 is the state shown in FIG. 8, the lens barrels 53 and 54 are driven to the state shown in FIG. 9, and when the state shown in FIG. , Lens barrel 53,
The shaft 55 and the shaft 56 are respectively set so that 54 is in the state shown in FIG.
, The process returns to step S401, and the above operation is repeated.

If the mode switching means 64 is not operated in step S410, the flow shifts to step S411 to detect the release button 63. If the operation of the release button 63 is not detected in step S411, the operation from step S401 to step S411 is repeated. If the operation of the release button 63 is detected in step S411, the process proceeds to step S401.
After that, similarly to the operation of step S403, the photometry, the focus adjustment, and the exposure to the imaging elements 57 and 58 are performed (step S403).
412). Next, the signal processing means 67 includes the imaging device 57,
After converting the video signal obtained in step 58 into a predetermined video signal (step S413), each video signal is further converted into a shooting mode, an identification signal indicating that they are paired with each other, a shooting condition, and an overlap amount A. , B (step S414). Then, the video signal generated by the signal processing means 67 is recorded in the memory 69 (Step S415). Thus, one photographing operation is completed. In the fourth embodiment, in the case of the panoramic photographing mode, the signal processing means 67 in the steps S406 and S412 inverts the video signal left and right to cancel the mirror images by the reflection mirrors 59 and 60.

Since the reproducing operation may be performed by any of the first to third embodiments, the description is omitted.

As described above, in the invention of the fourth embodiment,
The following effects are obtained.

(1) At the time of panoramic photographing, the intersections A and B between the entrance pupils of the plurality of photographing lenses 51 and 52 and the optical axis are made substantially coincident, so that each photographing lens 51, The image captured in 52 has almost no parallax, and it is possible to perform panoramic synthesis of each image without noticeable joints.

(2) Since the direction of the luminous flux incident on the photographic lenses 51 and 52 is changed by the reflection mirrors 59 and 60, the entrance pupils and the optical axes of the plurality of photographic lenses 51 and 52 have a simple configuration. It is possible to make the intersection points A and B substantially coincide with each other.

(3) Since the monitor 22 also serves as a finder, the system configuration can be simplified and the cost can be reduced.

(4) Since the finder image at the time of stereoscopic photography is not stereoscopically displayed, there is no need for the user to wear liquid crystal shutter glasses for stereoscopic vision, and there is no means for stereoscopic vision in the camera section. Since it is not necessary, the camera section can be made compact.

(5) The amount of overlap between the left and right images of the subject is high in accuracy since the measured values in the photographing mode are respectively stored.

(Fifth Embodiment) FIG. 11 is a diagram showing the configuration of a compound-eye camera system according to a fifth embodiment. In the form of the compound-eye camera system, the video recording unit and the video reproduction unit may be separated as shown in FIGS.

In FIG. 11, the compound-eye camera recording unit 80
Is assumed to include components other than the memory 19, the display operation means 21, and the monitor 22 in the configuration of the compound eye camera system shown in FIG. The card type memory 81
Both the compound-eye camera recording unit 80 and a video reproducing unit 82 described later are detachable via a connector (not shown) for making an electrical connection. The video reproducing unit 82 has a connector for making an electrical connection to the card-type memory 81 and the display calculating means 21 of the configuration of the compound-eye camera system shown in FIG. The monitor 83 displays the video generated by the video playback unit 82. In this embodiment, when recording an image, as shown in FIG.
Is attached to the compound-eye camera recording unit 80. The image recorded in the card-type memory 81 by the compound-eye camera recording unit 80 is stored in the card-type memory 81.
It can be displayed by attaching it to the video reproduction unit 82 as shown in FIG. still,
The description of the photographing operation and the display operation is omitted.

As described above, in the fifth embodiment, the compound-eye camera recording unit 80 and the video reproducing unit 82 are separated, and the card-type memory 81 for recording images is provided in the compound-eye camera recording unit 80. The following effects are achieved by being detachable.

(1) The recording section of a compound-eye camera for recording an image can be made compact, and the mobility of photographing can be improved.

(2) As the recording capacity of the card-type memory increases, more images can be recorded at once.

[0079]

As described above, according to the first aspect of the present invention, two imaging optical units for converging subject light to form an optical image on an imaging surface, and each of the two imaging optical units A camera head unit having an image sensor arranged on an image forming plane, an overlap amount calculation unit for calculating or storing an overlap amount of a shooting region of a plurality of video signals obtained by the camera head unit, Display means for displaying a video signal obtained by the camera head unit, wherein the display unit displays the plurality of video signals obtained by the camera head unit on the basis of the amount of overlap and overlaps the two video signals with each other. The display is performed alternately in time series so that the lap areas overlap.

Thus, whether the photographed image is a stereoscopic image or a panoramic image, the operator does not need to be aware of the operation and perform different operations. Also, there is no need to change the playback method depending on whether the captured video is a stereoscopic video or panoramic video,
The reproduction method is simple, and the cost can be reduced.

According to the second aspect of the present invention, at least two imaging optical units for converging subject light to form an optical image on an imaging surface, and at least two imaging optical units on the at least two imaging optical units. A camera head having an arranged image sensor; overlap amount calculating means for calculating or storing an overlap amount of a plurality of video signal imaging regions obtained by the camera head; Display means for displaying a plurality of video signals obtained by the camera head unit, when reproducing the plurality of video signals obtained by the camera head unit, When the overlap amount is larger than a predetermined amount, the plurality of video signals are displayed as a stereoscopic image. When the overlap amount is smaller than a predetermined amount, the plurality of video signals are displayed based on the overlap amount. It is obtained so as to display the plurality of video signals as one continuous image.

Therefore, whether the captured image is a stereoscopic image or a panoramic image, the operator does not need to be aware of the operation and perform different operations. Also, there is no need for a special signal or the like for discriminating between the stereoscopic video and the panoramic video.

According to the third aspect of the present invention, at least two image pickup optical units for converging subject light to form an optical image on an image forming surface, and at least two image forming surfaces of the at least two image pickup optical units. An image pickup device disposed, the at least two image pickup optical units and the image pickup device have a first position in which a photographing optical axis of the at least two image pickup optical units is substantially parallel, and the at least two image pickup optical units. And a camera head unit movably provided to a second position where the amount of overlap of a shooting region shot by the image sensor is smaller than the first position, and a plurality of camera head units obtained by the camera head unit. An overlap amount calculation unit that calculates or stores an overlap amount of a shooting region of a video signal, and a display unit that displays a plurality of video signals obtained by the camera head unit. When playing back a plurality of video signals obtained by the camera head unit, when shooting is performed at the first position, the plurality of video signals are played back as a stereoscopic video, and shooting is performed at the second position. When the plurality of video signals are displayed, the plurality of video signals are reproduced as one continuous video based on the overlap amount.

Therefore, even with a compound-eye camera capable of capturing both a stereoscopic image and a panoramic image, the image displayed by the user is
There is no need to consider whether the image is a stereoscopic image or a panoramic image. Further, even with a compound-eye camera capable of capturing both a stereoscopic video and a panoramic video, it is easy to determine whether the captured video is a stereoscopic video or a panoramic video.

According to a fourth aspect of the present invention, in the compound-eye camera system according to the third aspect, when displaying a plurality of video signals obtained by the camera head unit, the display means obtains the plurality of video signals obtained by the camera head unit. When the overlap amount of the plurality of video signals is larger than a predetermined amount, the plurality of video signals are displayed as a stereoscopic image, and when the overlap amount is smaller than a predetermined amount, the plurality of video signals are displayed based on the overlap amount. A plurality of video signals are displayed as one continuous video.

Thus, in a compound-eye camera capable of capturing both a stereoscopic image and a panoramic image, there is no need for a special signal or the like for discriminating the stereoscopic image and the panoramic image. Further, in a compound-eye camera capable of capturing both a stereoscopic video and a panoramic video, the position of the optical system at the time of capturing can be set to an arbitrary position between the first position and the second position.

According to a fifth aspect of the present invention, in the compound eye camera system according to any one of the second to fourth aspects, the overlap amount calculating means includes a focal length and a convergence angle of at least two photographing optical systems of the camera head unit. Information, a first overlap amount obtained from subject distance information, and a second overlap amount obtained from a plurality of video signals obtained by the camera head unit, and at least a plurality of image data obtained by the camera head unit. When the video signal is a panoramic video, a display on the display means is performed based on the second overlap amount.

As a result, in the case of a panoramic video, more accurate image synthesis can be performed. Therefore, even if a high quality image such as a still image is required, the panoramic image can be synthesized with a high quality that can withstand it. it can. Further, since the first overlap amount does not require a very high precision, there is no need to precisely detect the convergence angle, and the mechanism for driving the imaging optical unit can be simplified.

According to a sixth aspect of the present invention, in the compound eye camera system according to any one of the first to fifth aspects, the storage means for storing the plurality of video signals obtained by the camera head unit and the overlap amount is provided. It had.

Therefore, it is not necessary to calculate the amount of overlap between the video signals to be displayed, and the display can be performed quickly.

According to the seventh aspect of the present invention, in the compound eye camera system according to any one of the third to fifth aspects, when the at least two imaging optical units are at the second position, the at least two imaging optical units The distance between each object-side principal point, more specifically, the distance between the intersection of each exit pupil of each of the at least two imaging optical units and the optical axis, the at least two imaging optical units are located at the first position. It is made to be smaller than a certain time.

Thus, the parallax between each image captured by at least two image pickup optical units during panoramic image capturing is reduced, and a panoramic image with no noticeable joint is obtained for a subject having a larger subject distance difference. Can be.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a compound eye camera system according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating an operation example of the compound eye camera unit of FIG. 1;

FIG. 3 is a flowchart illustrating a shooting operation flow of the compound eye camera system of FIG. 1;

FIG. 4 is a flowchart showing a reproducing operation of the compound eye camera system.

FIG. 5 is a diagram showing a recording and display state of an image.

FIG. 6 is a flowchart illustrating a reproduction operation flow of the compound-eye camera system according to the second embodiment.

FIG. 7 is a flowchart illustrating a reproduction operation flow of the compound-eye camera system according to the third embodiment.

FIG. 8 is a block diagram showing a configuration of a compound-eye camera system according to a fourth embodiment of the invention.

FIG. 9 is a diagram illustrating an operation example of the compound eye camera unit of FIG. 8;

FIG. 10 is a flowchart showing an operation flow of the compound eye camera system of FIG. 8;

FIG. 11 is a diagram of a compound-eye camera system according to a fifth embodiment.

FIG. 12 is a diagram illustrating the principle of trapezoidal distortion.

FIG. 13 is a first diagram illustrating a positional relationship between a subject and a viewpoint.

FIG. 14 is a diagram showing how a subject looks at each viewpoint position in FIG. 13;

FIG. 15 is a second diagram illustrating a positional relationship between a subject and a viewpoint.

FIG. 16 is a diagram showing how a subject looks at each viewpoint position in FIG. 14;

[Explanation of symbols]

 1, 2, 51, 52 Photographing lens 1a, 2a, 51a, 52a Focus lens 3, 4, 53, 54 Lens tube 7, 8, 57, 58 Image pickup device (photoelectric conversion device) A, B Entrance pupil and optical axis 11, 12, 61, 62 Lens barrel driving means 13, 63 Release button 14 Overlap amount changing means 15, 65 Photometry means 16, 66 Focus detection means 17, 67 Signal processing means 18, 68 Overlap amount calculation Means 19, 69 Memory 20, 70 System controller 21, 71 Display calculation means 22, 72 Monitor 59, 60 Reflection mirror 64 Mode switching means

Claims (7)

[Claims] 1. A camera comprising: two imaging optical units for converging subject light to form an optical image on an imaging surface; and imaging devices arranged on the respective imaging surfaces of the two imaging optical units. A head unit, an overlap amount calculation unit that calculates or stores an overlap amount of a plurality of video signal shooting regions obtained by the camera head unit, and a display unit that displays a video signal obtained by the camera head unit Wherein the display means displays the plurality of video signals obtained by the camera head unit in a time series alternately so that the two video signals are overlapped with each other based on the overlap amount. And a compound eye camera system. 2. An image forming apparatus comprising: at least two image pickup optical units for converging subject light to form an optical image on an image forming surface; and image pickup devices arranged on respective image forming surfaces of the at least two image pickup optical units. A camera head unit, overlap amount calculation means for calculating or storing an overlap amount of a shooting region of a plurality of video signals obtained by the camera head unit, and a plurality of video signals obtained by the camera head unit. And displaying the plurality of video signals obtained by the camera head unit when the plurality of video signals obtained by the camera head unit are reproduced by a predetermined amount. If the overlap is smaller than a predetermined amount, the plurality of video signals are displayed as a stereoscopic image, and if the overlap amount is smaller than a predetermined amount, the plurality of video signals are displayed based on the overlap amount. A compound eye camera system characterized by displaying as two continuous images. 3. An image pickup apparatus comprising: at least two image pickup optical units for converging subject light to form an optical image on an image forming surface; and image pickup devices arranged on respective image forming surfaces of the at least two image pickup optical units. The at least two imaging optical units and the imaging device are imaged by the first position where the imaging optical axes of the at least two imaging optical units are substantially parallel, and are captured by the at least two imaging optical units and the imaging device. A camera head unit movably provided at a second position where the amount of overlap of the shooting regions is smaller than the first position, and an overlap of the shooting regions of a plurality of video signals obtained by the camera head unit. An overlap amount calculating means for calculating or storing a lap amount; and a display means for displaying a plurality of video signals obtained by the camera head unit, wherein the display means is obtained by the camera head unit. When playing back a plurality of video signals, the plurality of video signals are played back as a stereoscopic video when shooting is being performed at the first position, and when the shooting is being performed at the second position. A compound eye camera system, wherein the plurality of video signals are reproduced as one continuous video based on an amount of overlap. 4. The display means, when reproducing a plurality of video signals obtained by the camera head, wherein the overlap amount of the plurality of video signals obtained by the camera head is larger than a predetermined amount. When the plurality of video signals are displayed as a stereoscopic video, and when the overlap amount is smaller than a predetermined amount, the plurality of video signals are displayed as one continuous video based on the overlap amount. The compound eye camera system according to claim 3, wherein 5. The apparatus according to claim 1, wherein the overlap amount calculating means includes a first overlap amount obtained from focal length and convergence angle information of at least two photographing optical systems of the camera head unit, subject distance information, and the camera head unit. Calculate a second overlap amount obtained from the obtained plurality of video signals, at least based on the second overlap amount when the plurality of video signals obtained by the camera head unit is a panoramic video. The compound eye camera system according to claim 2, wherein a display is performed on the display unit. 6. The compound eye camera system according to claim 1, further comprising a storage unit for storing a plurality of video signals obtained by said camera head unit and said overlap amount. Compound eye camera system. 7. A distance between respective object-side principal points of the at least two imaging optical units when the at least two imaging optical units are at the second position, more specifically, the at least two imaging optical units. The distance between the intersections of the respective exit pupils and the optical axis is smaller than when the at least two imaging optical units are at the first position. The compound eye camera system as described.