JP3107957B2 - Compound eye imaging device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a compound eye imaging apparatus,
In particular, the present invention relates to a compound eye imaging device having a microphone.

[0002]

2. Description of the Related Art Some imaging devices such as video cameras have a microphone for recording sound along with images. In the case of a VTR-integrated video camera using a zoom lens, for example, by combining a unidirectional microphone and a bidirectional microphone and combining the outputs of both microphones according to the focal length of the zoom lens, stereo Gets audio. That is, the direction of maximum sensitivity of the unidirectional microphone is directed toward the front of the imaging device, and the direction axis of the bidirectional microphone is arranged in a horizontal plane and perpendicular to the direction of maximum sensitivity of the unidirectional microphone. Thus, the sum signal (L + R) of the left and right voices is output from the unidirectional microphone.
Is output from the bidirectional microphone, and a difference signal (LR) between the left and right sounds is output, and a stereo sound can be reproduced from the sum and difference signals. Furthermore, when the zoom lens of the image pickup device is on the telephoto side, the output of the unidirectional microphone is emphasized, and when the zoom lens is on the wide-angle side, the output of the bidirectional microphone is emphasized. The obtained audio signal is obtained.

[0003] A compound-eye imaging apparatus that shoots a common subject using two imaging systems (for example, two video cameras) and combines two images obtained by each imaging system to generate one image is known. Is being developed. In a compound-eye imaging apparatus, generally, a driving system for rotating each imaging system is provided to change a convergence angle according to a subject distance, but by controlling each imaging system independently by this driving system, In addition, it is possible to direct the optical axis to a subject that is not in the front direction of the compound-eye imaging device. Further, by making the optical axes of the respective imaging devices parallel to each other, or by making the optical axes of the respective imaging devices spread mutually in front of the imaging devices, an extremely wide angle which cannot be obtained with a single camera can be obtained. (Panoramic image) can be captured.

[0004]

In the case where a compound eye image pickup apparatus is formed by using two conventional image pickup apparatuses with microphones, a panoramic image is obtained simply by adding the microphone outputs of the respective image pickup apparatuses. However, there is a problem that an audio signal corresponding to the angle of view cannot be obtained.

An object of the present invention is to provide a compound-eye imaging apparatus capable of acquiring a sound corresponding to a change in the angle of view.

[0006]

According to the present invention, there is provided a compound-eye imaging apparatus according to the present invention, wherein first and second video camera units are rotated, and the first and second video camera units are rotated so that light from each of the video camera units is changed. In a compound-eye imaging apparatus having a driving unit for changing the direction of an axis, a microphone and a sound processing unit for changing the directivity of the microphone according to the direction of the optical axis of each video camera unit are provided.

[0007]

The sound processing means for changing the directivity of the microphone according to the direction of the optical axis of each video camera unit is provided, so that sound corresponding to the change in the angle of view can be obtained.

In the present invention, a microphone can be provided commonly to the first and second video camera units. Further, as the microphone, a combination of a unidirectional microphone and a bidirectional microphone having a maximum sensitivity direction orthogonal to the maximum sensitivity direction of the unidirectional microphone can be used,
In this case, the outputs of the unidirectional microphone and the bidirectional microphone may be combined according to the intersection angle of the optical axis. Further, a microphone rotating means for rotating the microphone so as to orient the directional axis of the microphone in the direction of the intersection of the optical axis of the first video camera unit and the optical axis of the second video camera unit may be provided.

[0009]

Next, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is an external perspective view showing a compound-eye imaging device according to an embodiment of the present invention, FIG. 2 is a block diagram showing a configuration of the compound-eye imaging device, and FIG. 3 is a block diagram showing a configuration of an audio processing unit.

This compound eye imaging apparatus (camera) has two
In this configuration, two video camera units 10 _R and 10 _L are stored together in the camera body 13, and a microphone 38 is also attached to the camera body 13. The video camera units 10 _R and 10 _L are arranged in substantially parallel to each other in substantially the same direction, and each include an optical unit, a main unit, and an electric circuit unit inside the main unit. The optical section is specifically composed of lens groups 1 _R and 1 _L constituting a zoom lens. Semiconductor elements using a CCD are used as the imaging elements 2 _R and 2 _L.

Further, the camera body 13 has
Video camera unit 10_R, 10_LImage output corresponding to VTR
Terminal 50 for outputting to a storage device such as_R, 50_LAnd
A terminal 52 for outputting an audio signal or a power supply from outside
Supply and even remote control signals
Terminal (not shown) for
An inductor 24 is also provided. Viewfinder 24
Is the video camera unit 10 _R, 10_LImaging output of one of
It reflects power. The microphone 38 is connected to each video
Camera 10_R, 10_LLens group 1_R, 1_LDuring this
Video camera unit 10_R, 10_LSo that it is almost in the same direction as
It is fixed to. In addition, the camera body 13
Button or switch to select the main camera selection switch.
Switch 25, camera standby button 27, zoom operation button
28, status storage button 31, 3D-panorama off
A switch 35 and an initial state selection switch 32 are provided.
Have been. The initial state selection switch 32 is
The camera status at the time of standby
Is the camera's status when the status storage button 31 is pressed.
The status is memorized and the status is automatically set during standby.
It is for returning to the state.

Next, the internal configuration of the compound eye imaging apparatus will be described.

A right focus motor 8 _R and a left focus motor 8 _L are provided for focusing the lens groups 1 _R and 1 _L of the video camera units 10 _R and 10 _L , respectively. _A right zoom motor 7 _R and a left zoom motor 7 _L are provided to adjust the focal lengths of _R and 1 _L. Each lens group 1 _R , 1 _L has an image sensor 2 _R , 2 _L
Apertures 3 _R and 3 _L are provided to adjust the amount of light received by the light source. Furthermore, the zoom motor 7 _R, 7 _L, and the position detecting circuit 11 for detecting the position of each focus motor 8 _R, lens driven by _{8 L R, 11 L, 12} R, 12 L are provided, these Is input to the control unit 15. The control unit 15 stores the variations in the sensitivity of the imaging devices 2 _R and 2 _L , the individual differences between the lens groups 1 _R and 1 _L , and the initial values of the angles and intervals of the video camera units 10 _R and 10 _L during manufacturing. An EEPROM 16 for storage is connected.

A right convergence angle motor 17 _R and a left convergence angle motor 17 _L for rotating the video camera units 10 _R and 10 _L in a horizontal plane with the light receiving surfaces of the image pickup devices 2 _R and 2 _L substantially as the center of rotation. The video camera units 10 _R and 10 _L are provided with rotation angle detectors 19 _R and 19 _L , respectively, and the outputs are input to the control unit 15. Then, on the right side of the video camera unit 10 _R, an adjustment unit 21 for adjusting the distance between the right and left of the video camera unit 10 _R, 10 _L, the control unit 1 detects this gap
5 and an adjustment motor 22 for driving the adjustment unit 21 are attached.

A synchronizing signal generating circuit 36 for generating a synchronizing signal
Is provided, and each of the imaging elements 2 _R and 2 _L is driven in synchronization with a synchronization signal. Analog video signal output from the image pickup elements 2 _R, 2 _L are input to an unillustrated via amplifier converts each digital image signal A / D converter 40 _R, 40 _L. These analog video signals are also input to focus detection circuits 33 _R and 33 _L to detect the focus state. Specifically, the focus detection circuits 33 _R and 33 _L take out a specific high-frequency component (for example, 1 MHz) of the video signal in a predetermined area in the imaging screen using a band-pass filter, and the amplitude value of the high-frequency component Is input to the control unit 15. Focusing is performed by moving the positions of the focus lenses in the lens groups 1 _R and 1 _L by the focus motors 8 _R and 8 _L so that the amplitude of the high-frequency component is maximized.

The digital video signals output from the A / D converters 40 _R and 40 _L are supplied to the image memories 41 _R and 41 _L whose writing and reading are controlled by the synchronizing signal from the synchronizing signal generating circuit 36 described above. Each is stored.
The digital video signals stored in each of the image memories 41 _R and 41 _L are input to an image correlation processing unit 46, where the image correlation calculation is performed. According to the calculation result obtained by the image correlation processing section 46, each image memory 41 _R ,
Image synthesis processing unit 47 is provided for synthesizing the image in the 41 _L. As will be described later, this camera (compound eye imaging device)
Is provided with a stereoscopic photographing mode and a panoramic photographing mode, and the image combining processing unit 47 performs panoramic combining of images when performing panoramic photographing.

D / A converters 42 _R and 42 _L for converting digital image signals output from the respective image memories 41 _R and 41 _L or the image synthesizing processing unit 47 into analog signals are provided. D / A corresponding to the left video camera unit 10 _L
The transducer 42 _L in order to switch between the output and the output of the image memory 41 _L of a digital image signal the image synthesis processing section 47 to input, solid - panorama selector switch 3
5 is connected. The output of the D / A converter 42 _R, 42 _L are respectively connected to the terminals 50 _R, 50 _L,
It is input to the main camera selection switch 25. The output side of the main camera selection switch 25 is connected with a viewfinder 24, a viewfinder 24 by one of the output video signal to the D / A converter 41 _R, 41 _L
The image is displayed on the. At the time of panorama shooting, the main camera selection switch 25 is set to the left video camera unit 1
By switching to the 0 _L side, the image synthesis processing unit 47 is obtained from the signals captured by the left and right video camera units 10 _R and 10 _L.
Viewfinder 2 with panoramic image synthesized by
It can be observed at 4.

An audio processing section 37 for processing an audio signal from the microphone 38 is provided, and an output of the audio processing section 37 is connected to a terminal 52. Microphone 38
, As shown in FIG. 3 is constituted of a microphone unit 38 _S unidirectional microphone unit 38 _M and bidirectional. Here, the microphone units 38 _M and 38 _S independently operate as microphones.
In order to clarify the distinction from the microphone 38 configured by combining these, the microphone unit is referred to as a microphone unit. Hereinafter, the audio processing according to the present invention will be described.

The unidirectional microphone unit has a maximum sensitivity to the sound source in front of the microphone, and the sensitivity decreases as the sound source shifts from the front.
It has a directional characteristic with a sensitivity of 0. That is, assuming that the front sensitivity of the unidirectional microphone unit is S _M and the angle of the sound source with respect to the front of the microphone unit is α, the sensitivity S (α)
Is

[0020]

(Equation 1) It is represented as On the other hand, the bidirectional microphone unit has the maximum sensitivity to the sound source at the front and back of the microphone, and has the characteristic that the sensitivity is theoretically zero in the 90 ° and 270 ° directions. If the front sensitivity of the bidirectional microphone unit is S _S and the angle of the sound source with respect to the front of the microphone unit is δ, the sensitivity S (δ) is

[0021]

(Equation 2) It is expressed as In some cases, S (δ) <0, which indicates that the input is an anti-phase input. FIGS. 4A and 4B show the directivity patterns of the unidirectional microphone unit 38 _M and the bidirectional microphone unit, respectively.

[0022] In the microphone 38, for a single directional microphone unit 38 _M bidirectional microphone unit 38 _S 9
They are arranged at an angle of 0 °. That is, the maximum sensitivity directions of both microphone units 38 _M and 38 _S are perpendicular to each other. Then, the front direction of the unidirectional microphone unit 38 _M is adapted substantially coincides with the optical axis of each video camera unit 10 _L, 10 _R. FIG. 4C shows the directivity patterns of the two microphone units 38 _M and 38 _S when the front direction of the unidirectional microphone unit 38 _M is the 0 ° direction.

The audio processing unit 37 includes a phase dividing section 37 _a which entered speech signal from the bi-directional microphone unit 38 _S divides the audio signal into a positive phase component and negative phase component, unidirectional microphones It is constituted by the gain controller 37 _b for amplifying / attenuating the speech signal input voice signal from unit 37 _M. The amplification / attenuation rate of the gain variable section 37 _b is controlled by a control signal from the control section 15. The output of the gain controller 37 _b is a positive phase output from the phase dividing section 37 _a, adapted to be respectively added to the negative-phase output. The positive-phase output after the addition becomes the left output, and the negative-phase output becomes the right output. These left and right outputs are transmitted to the terminal 52.

Note that the maximum sensitivity directions of both microphone units 38 _M and 38 _S are orthogonal, and
_{Assuming that} the amplification factor (gain) of 7b is k (attenuation if k <1), the right output and the left output are respectively

[0025]

(Equation 3) Can be expressed as The directional axis θ on one side of the microphone 38 as a whole (the direction in which the sensitivity becomes maximum) is

[0026]

(Equation 4) Therefore, by changing the amplification factor k,
The pointing axis θ can be changed.

Next, audio processing in an actual camera operation will be described. As described above, the compound-eye imaging device is provided with the stereoscopic imaging mode and the panoramic imaging mode. First, the stereoscopic imaging mode will be described.

The three dimensional mode, in order to obtain a so-called stereo image is a mode for photographing a same object by the right video camera portion 10 _R and the left video camera portion 10 _L. At this time, as shown in FIG. 5A, the optical axes 14 _R and 14 _L of both the video camera units 10 _R and 10 _L intersect in front of the cameras. Further, since the lens groups 1 _R and 1 _L are zoom lenses, the focal length f changes. In the three dimensional mode, by changing the gain k in the gain varying section 37 _b in conjunction with the focal length f, corresponding to the imaging angle (audio) can be obtained directed shaft. Specifically, the gain k should be minimized on the telephoto side, maximized on the wide-angle side, and the gain k should correspond to the focal length f during that time.

If the object distance changes even if the focal length does not change, the directions of the optical axes 14 _R and 14 _L are changed so that the intersection position of the optical axes 14 _R and 14 _L coincides with the object. There is a need. For example, when the subject approaches the camera, the video camera units 10 _R and 10 _L must be turned inward so that the intersection angle between the optical axes 14 _R and 14 _L becomes large. Here, the value of the gain k corresponding to the focal length f is, for example, based on the value of the gain k when the subject distance is set to 5 m, in the direction of decreasing the gain k when the subject distance is smaller than 5 m. When the object distance is larger than 5 m, the correction may be made in the direction of increasing the gain k. By performing such audio processing, audio corresponding to the imaging screen can be obtained in the stereo shooting mode.

On the other hand, in the panoramic photographing mode, an image having an extremely large angle of view is obtained by combining images photographed by the left and right video camera units 10 _R and 10 _{L. In} this case, FIG. As shown in (b), the optical axes 14 _R and 14 _L
Crosses on the rear side of the camera, and the left and right video camera sections 10
The images at _R and 10 _L partially overlap each other. In the panoramic photography mode, the optical axis 14 _R , 1 on the rear side of the camera for the gain k corresponding to the focal length f described above.
It may be corrected so as to increase in accordance with the intersection angle of 4 _L. The video camera units 10 _R and 10 _L are compared with the subject distance.
Is assumed to be sufficiently small, this interval is ignored, and the gain corrected according to the intersection angle is set to k ′. As shown in FIG. 6, the intersection angle between the optical axes 14 _R and 14 _L is γ, and the directional axis at this time is β. In FIG. 6, the dotted line indicates the front direction of the camera. Then

[0031]

(Equation 5) Becomes Furthermore, the directional axis β corrected to the panorama size
+ Γ is

[0032]

(Equation 6) It is expressed as k, S _S, a value S _M h is set in advance, gamma is determined from the output of the left and right convergence angle motor 17 _R, 17 _L rotary angle detector 19 is a value that varies with _R, 19 _L Value. Therefore, the corrected gain k ′ can be obtained from the above equation. As described above, even at the time of panoramic shooting, it is possible to obtain sound corresponding to the angle of view of the image.

While the embodiment of the present invention has been described above, in the above embodiment, the microphone 38 itself is fixed to the front of the camera, and the microphone units 38 _M and 38 _{M having} different directivities are used.
38 by changing the directional axes by changing the synthesis method of a signal from _S, won the sound corresponding to the angle of view. However, in multi-lens imaging apparatus described above, the left and right of the video camera unit 10 _R, 10 _L since the it is possible to rotationally drive each independently, each video camera unit 10 _R to the object that is not located in front of the apparatus , 10 _L. In such a case, the microphone itself may be rotatably attached to the camera body, and a drive mechanism for rotating the microphone may be provided. In the stereoscopic photography mode, the microphones are controlled so as to point in the direction of intersection of the optical axes 14 _R and 14 _L of the left and right video camera units 10 _R and 10 _{L. In} the panoramic photography mode, the optical axes 14 _R and 14 _R are controlled.
The rotation angle of the microphone may be controlled so as to face the subject on a line connecting the intersection of _L and the rotation center of the microphone.

Further, two microphones are prepared and the left and right microphones are provided.
Video camera unit 10_R, 10_LTo each of the
It is also possible to correspond to a subject not in front of the camera. Figure
7 is each video camera unit 10_R, 10_LMy to each of
Crohon 38_R, 38_LSound processing unit 6 when attaching
FIG. 3 is a block diagram showing a configuration of a 0. Each microphone 38
_R, 38_LAre each a single finger, as in the previous embodiment.
Directional microphone unit 38_MAnd bidirectional microphone unit 3
8_SAnd a combination thereof.

The audio processing unit 60 includes each microphone 38
_R, 38 2 pieces of a phase dividing section 61 _R, 61 _L of the audio signal from the bi-directional microphone unit 38 _S separates the speech signal input to the positive phase component and negative phase component of the _L, the control It is composed of three gain variable units 62 _R , 62 _L , 63 whose gains are controlled by the unit 15. Positive-phase output of the phase dividing section 61 _R corresponding to the right side is input to gain controller 62 _R, reverse phase output of the left phase dividing section 61 _L is input to gain controller 62 _L. Then, the third to the gain controller 63, gain controller 62 _R and the output of the right microphone 38 _R unidirectional microphone unit 3
The sum of the audio signal from the unidirectional microphone unit 38 _M audio signals and the gain controller 62 _L and the output of the left microphone 38 _L from 8 _M inputs. Then, the output of the third gain variable section 63 is divided into two, one is added to the negative phase output of the right phase dividing section 61 _R to become a right output, and the other is the positive phase output of the left phase dividing section 61 _L. Is added to the left output.

The positive phase component inputted from the phase dividing section 61 _R to the variable gain unit 62 _R corresponding to the right side, the unidirectional microphone unit 38 _M of the variable gain portion 61 _R and the output of the right microphone 38 _R since the voice is added from the directional axis in the left direction from the front for the right side of the microphone 38 _R is variable. This allows
Even right video camera portion 10 _R is directed toward the deviating from the subject of the front of the device, the gain variable unit 62 _R
If the gain as such is directed axis in the apparatus front from the rotation angle of the right of the video camera unit 10 _R, it is possible to obtain a sound from the device front direction. Similarly, since the signal from a single directional microphone units 38 _M of the variable gain section 62 _L left microphone 38 _L to the output of the reverse-phase component is input from the phase dividing section 61 _L is added, the left microphone For 38 _L , the directivity axis in the right direction from the front is variable. Therefore, by setting the gain of the gain controller 62 _L, the left video camera portion 10 _L
Can be obtained from the front of the apparatus even when the player is not facing the front of the apparatus. In this way, if the sound converted from the front of the device is treated as the output of the unidirectional microphone unit and input to the gain variable unit 63, the gain of the gain variable unit 63 is changed according to the focal length. gain in each of the positive-phase output of the phase dividing section 61 _L of the negative-phase output and the left of the phase dividing section 61 _R variable section 63
By adding the above outputs, even in the case of shooting a panoramic video with an extremely wide angle of view, it is possible to acquire the sound corresponding to the imaging screen, as in the above-described embodiment.

Here, the microphones 38 _L and 38 _R
Microphone There was to be a zoom microphone that combines the unidirectional microphone units 38 _M and bi-directional microphone unit 38 _S, consisting of only a single directional microphone units to each of the video camera unit 10 _R, 10 _L Is rotatably mounted and rotated in accordance with the intersection angle of the optical axes 14 _R and 14 _L of the video camera units 10 _R and 10 _L , so that sound according to the angle of view is obtained without using a variable gain unit. Can be acquired.

In the above embodiment, the camera is rotated as a means for changing the crossing angle of the optical axis of each camera. However, the crossing angle of the optical axis can also be obtained by using an optical element such as a variable apex prism. Can be changed. In addition, the use of an optical element can also simplify the mechanical system, which is useful for reducing the size and weight of the camera.

[0039]

As described above, according to the present invention, the microphone and the sound processing means for changing the directivity of the microphone in accordance with the direction of the optical axis of each video camera unit are provided, so that the image pickup in the compound-eye imaging apparatus can be performed. There is an effect that it is possible to acquire a sound corresponding to a change in the angle of view.

[Brief description of the drawings]

FIG. 1 is an external perspective view showing a compound-eye imaging apparatus according to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating a configuration of the compound-eye imaging device of FIG. 1;

FIG. 3 is a block diagram illustrating a configuration of an audio processing unit.

FIGS. 4A to 4C respectively show the directivity pattern of the unidirectional microphone unit, the directivity pattern of the bidirectional microphone unit, and the unidirectional microphone unit and the bidirectional microphone unit; It is a graph which shows the directivity pattern at the time of combination.

5A is a diagram illustrating a relationship between optical axes in a stereoscopic imaging mode, and FIG. 5B is a diagram illustrating a relationship between optical axes in a panoramic imaging mode.

FIG. 6 is a diagram illustrating an intersection angle and a directivity axis in a panoramic shooting mode.

FIG. 7 is a block diagram illustrating a configuration of an audio processing unit according to another embodiment of the present invention.

[Explanation of symbols]

1 _R, 1 _L lens group 2 _R, 2 _L image sensor 10 _R, 10 _L video camera unit 13 camera 14 _R, 14 _L optical axis 15 control unit 24 viewfinder 37, 67 sound processing section 37 _a, 61 _R, 61 _L phase division unit 37 _b , 62 _R , 62 _L , 63 Gain variable unit 38, 38 _L , 38 _R microphone 38 _M unidirectional microphone unit 38 _S bidirectional microphone unit 46 image correlation processing unit 47 image synthesis processing Department

Continuation of the front page (56) References JP-A-63-266980 (JP, A) JP-A-63-116590 (JP, A) JP-A-7-38924 (JP, A) JP-A-52-71209 (JP) , A) (58) Field surveyed (Int. Cl. ⁷ , DB name) H04N 5/222-5/253 H04N 13/02

Claims (4)

(57) [Claims] 1. A compound eye comprising: first and second video camera units; and driving means for rotating the first and second video camera units to change the direction of the optical axis of each of the video camera units. An image pickup apparatus comprising: a microphone; and audio processing means for changing a directivity of the microphone according to a direction of an optical axis of each video camera unit. 2. The compound eye imaging apparatus according to claim 1, wherein a microphone is provided commonly to the first and second video camera units. 3. The microphone is configured by combining a unidirectional microphone and a bidirectional microphone having a maximum sensitivity direction orthogonal to the maximum sensitivity direction of the unidirectional microphone, and the sound processing unit includes: 2. The apparatus according to claim 1, wherein the outputs of the unidirectional microphone and the bidirectional microphone are combined in accordance with an intersection angle of an optical axis.
Or the compound-eye imaging device according to 2. 4. A microphone rotating means for rotating the microphone so as to orient a directional axis of the microphone in a direction of an intersection of an optical axis of the first video camera unit and an optical axis of the second video camera unit. The compound eye imaging device according to claim 1, further comprising: