JP6057627B2 - Stereoscopic image capturing apparatus, camera system, control method for stereoscopic image capturing apparatus, program, and storage medium - Google Patents

Description

  The present invention relates to an imaging apparatus and a control method thereof, and more particularly to an imaging apparatus capable of capturing a stereoscopic image and a control method thereof.

  Patent Document 1 discloses the use of a single imaging optical system and a pair of right-eye detectors and left-eye detectors provided on a single detector array in order to capture a stereoscopic image. Has been.

  By the way, for stereoscopic vision, the relative distance between the stereoscopic view obtained by increasing the base line length and the convergence angle in the stereo camera system and the fusion tolerance range that should be taken into consideration when people actually appreciate it. There is a problem to do. Patent Document 2 discloses a method for controlling the convergence angles and the base line lengths of a plurality of imaging units so that both the closest subject and the farthest subject fall within the fusion allowable range from the selected composition. ing.

  On the other hand, there is also a demand for obtaining a stereoscopic image according to the type, size, shooting scene, and observer (user) preference of the display device. In Japanese Patent Laid-Open No. 2004-318, in response to such a problem, a plurality of multi-viewpoint images in which at least one of convergence angles and base line lengths of a plurality of imaging units is different are continuously photographed by a single shutter release operation (3D bracket photography). ) A technique for performing control is disclosed.

JP-A-58-24105 JP-A-7-167633 JP 2008-312058 A

  According to Patent Document 2, when viewing a stereoscopic image of the entire selected composition, a contradiction between the observer's convergence and adjustment (eye focus adjustment) is unlikely to occur, and therefore, it can be viewed without fatigue.

  However, with the progress of digital image technology, it is said that the usage of stereoscopic images is diversified, and it is not always appreciated according to the selected composition. For example, the use of a stereoscopic image has come out, such as cutting out and saving a main subject from an obtained stereoscopic image and viewing it. Here, when the depth of the entire composition is large, the stereoscopic effect of the main subject tends to become poorer as the distance increases. Therefore, when the main subject is cut out, stored, and viewed, it cannot always be said that a sufficient stereoscopic effect is obtained with respect to the main subject, and the observer may not be able to view a desired stereoscopic image. there were.

  In this regard, according to Patent Literature 3, there is a possibility that photography can be performed without being constrained by the stereoscopic effect of the entire composition, but Patent Literature 3 has no suggestion regarding the range of 3D bracket photography.

  An object of the present invention is to shoot a stereoscopic image that can meet the demands of various users during viewing.

A stereoscopic image capturing apparatus according to an aspect of the present invention includes a plurality of imaging units that capture a subject from a plurality of different viewpoints, a convergence angle setting unit that sets a convergence angle between the plurality of imaging units, and the imaging unit. A subject distance measuring unit that measures a distance from the subject to the subject, and a control unit that performs a first shooting and a second shooting using the imaging unit, and the plurality of imaging units include: A plurality of photoelectric conversion units provided in one pixel, wherein the convergence angle setting unit is the first closest to the imaging unit among the distances measured by the subject distance measuring unit in the first imaging. The first convergence angle is set so that the amount of parallax formed on the imaging surface by a point at a distance of 1 is equal to or less than a predetermined first threshold value. The first of the distances measured A parallax variation in which a plurality of points forming on the imaging surface of the object at a distant second distance than away, so that the above predetermined second threshold value, and the maximum light incident on the photoelectric conversion portion An angle is calculated, and a second convergence angle different from the first convergence angle is set so that the maximum light incident angle is smaller than a predetermined third threshold value .

  Further objects and other features of the present invention will become apparent from the preferred embodiments described below with reference to the accompanying drawings.

  According to the present invention, it is possible to perform the first stereoscopic shooting that allows the entire shooting composition to be viewed without a sense of incongruity and the second stereoscopic shooting that does not impair the stereoscopic effect of the subject of interest or a distant subject. Therefore, it is possible to meet the demands of various users at the time of viewing by using the stereoscopic image captured by the first stereoscopic shooting and the stereoscopic image captured by the second stereoscopic shooting.

It is a block diagram showing the structure of the stereo image imaging device which concerns on this invention. It is a top view showing the imaging surface composition of the solid-state image sensing device concerning the present invention. It is a figure which shows an example of the ray diagram which concerns on this invention. It is a graph showing the image formation relational expression concerning this invention. It is a flowchart showing control of the three-dimensional image pick-up device concerning the 1st example of the present invention. It is a flowchart showing control of the stereo image imaging device which concerns on 2nd Example of this invention. It is sectional drawing and the pupil intensity distribution figure of the unit pixel of the solid-state image sensor concerning this invention. It is a flowchart showing control of the three-dimensional image pick-up device concerning the 3rd example of the present invention. It is a flowchart showing control of the stereo image imaging device which concerns on the 4th Example of this invention. It is a figure which shows an example of the stereo image imaging device which concerns on the 5th Example of this invention. It is a flowchart showing control of the stereo image imaging device which concerns on the 5th Example of this invention.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  First, the overall configuration of a stereoscopic image capturing apparatus according to an embodiment of the present invention will be described. FIG. 1 is an example of a block diagram illustrating an overall configuration of a stereoscopic image capturing apparatus according to an embodiment of the present invention. In the following, a lens-integrated stereoscopic image capturing apparatus including a photographing optical system will be described. However, the present invention is a so-called interchangeable lens type in which a photographing optical system (lens apparatus) is detachable from the stereoscopic image capturing apparatus. Needless to say, it can be applied to other camera systems. In FIG. 1, reference numeral 1 denotes a photographing optical system such as a lens including a diaphragm and a mechanical shutter, and 2 denotes a solid-state imaging device that photoelectrically converts a subject image formed by the photographing optical system 1 and takes it out as an electrical signal. In order to perform stereoscopic image shooting according to the present invention, it is necessary to control an imaging unit capable of imaging from a plurality of viewpoints and a convergence angle between the imaging units. The imaging from a plurality of viewpoints and the control of the convergence angle are concretely realized by the characteristics of the imaging optical system 1 and the solid-state imaging device 2, but since it is a structure and control unique to the present invention, details will be described later. I will decide.

  Reference numeral 3 denotes a correlated double sampling (CDS) circuit for sampling an analog electric signal of the solid-state imaging device 2, and reference numeral 4 denotes an A / D converter (A / D converter) for converting the sampled analog signal into a digital signal. ). The digitized image signal is subjected to various signal processing such as white balance correction and gamma correction by the signal processing circuit 7 via the image memory 8, and is recorded on the recording medium 10 as an image signal having parallax. The The recording circuit 9 indicates an interface circuit with the recording medium 10. Further, the image signal subjected to the signal processing can be directly displayed on the image display device 12 such as a liquid crystal display through the display circuit 11 which is an interface circuit. In order to confirm or appreciate the stereoscopic image obtained by the present invention, the image display device 12 may be configured as described below. That is, the image display device 12 updates the image according to a specific cycle. The update of the even line display is started at a certain time, and the update of the even line display is completed when half of the update cycle has elapsed. Next, the update of the odd-numbered line display is started from the time when half of the update cycle has elapsed, and the update of the odd-numbered line display is completed when the entire update period has elapsed. While updating the odd line display, the even line display is held. On the contrary, while the even line display is being updated, the odd line display is held. For example, an image source for even-numbered line display is set as a viewpoint picked up corresponding to the right eye, and an image source for odd-numbered line display is set as a viewpoint picked up corresponding to the left eye. Then, the observer covers the left eye during the period in which the even-line display corresponding to the right eye is held, looks at the image from the right eye, and observes the right eye in the period in which the odd-line display corresponding to the left eye is held. If you can see the image from your left eye while obscuring it, you can see stereoscopically. In other words, the observer can check or appreciate such a stereoscopic image by wearing glasses equipped with a liquid crystal shutter that covers the left and right fields of view in synchronization with the update cycle of the image display device 12. Become. Note that the image display device 12 is not limited to stereoscopic image display, and can also perform live view display for continuously displaying a screen to be captured in the future, and playback display of recorded moving images.

  The timing generation circuit 5 drives an imaging system such as a lens diaphragm, a mechanical shutter, and the solid-state imaging device 2 through a driving circuit 6. Further, the CDS circuit 3 and the A / D converter 4 are driven and controlled in synchronization with the driving of the imaging system and, in turn, the output signal of the solid-state imaging device 2.

  The system control unit 13 controls the entire stereoscopic image capturing apparatus with a program temporarily stored in a volatile memory (RAM) 14.

  The stereoscopic image capturing apparatus further includes a convergence angle setting unit 16 unique to the present invention, and also sets the convergence angles of the photographing optical system 1 and the solid-state image sensor 2 via the drive circuit 6. The convergence angle setting unit 16 determines the distribution of the convergence angle corresponding to the subject distance information from the subject distance measurement unit 17 via the system control unit 13, and sets the convergence angle before photographing by comparing with a predetermined threshold value. To do. Such a series of functions of the convergence angle setting means 16 has a structure and control unique to the present invention, and details will be described later in each embodiment.

  The subject detection means 18 searches for a main subject from the captured image or the live view display image. Depending on the embodiment of the present invention, such a function of searching for a main subject may be used.

  Reference numeral 15 denotes a non-volatile memory (ROM) storing a program and various data to be transferred when the system control unit 13 executes control, and also stores various threshold values and functions used for setting a convergence angle, which will be described later.

  Next, details of the photographing optical system 1 and the solid-state imaging device 2 used in the present invention will be described. These provide a plurality of viewpoints necessary for the stereoscopic image capturing described above. For simplification, a system having two viewpoints will be described first.

  FIG. 2 is a plan view of the imaging surface of the solid-state imaging device 2. The solid-state imaging device 2 uses a pixel including a single microlens 21 that covers a pair of photoelectric conversion units (a plurality of imaging units) 20L and 20R that are independent on the left and right as unit pixels, and these are horizontal and vertical. Are arranged in N rows and M columns (N = 4, M = 6 in FIG. 2). Each photoelectric conversion unit included in each pixel (within one pixel) receives an optical image formed by the photographing optical system 1 as one set of viewpoints. For example, the left photoelectric conversion unit 20L and the right photoelectric conversion unit Two sets of viewpoints can be obtained by 20R. The photoelectric conversion unit may be configured by, for example, a photodiode (PD). In the present embodiment, it is described that two sets of viewpoints are obtained by two independent photoelectric conversion units, left and right, but the left and right directions are not necessarily required. The number is not limited to two. If N (N ≧ 2) independent photoelectric conversion units are used under the micro lens 21, N viewpoints can be provided.

  FIG. 3 is a ray diagram illustrating a mechanism for obtaining a plurality of viewpoints together with the photographing optical system 1 in the configuration of the solid-state imaging device 2 as described above. The photographing optical system 1 includes a diaphragm 10 having an aperture D. In FIG. 3A, the light emitted from the point A on the object side on the optical axis is incident on the solid-state image sensor 2 with the exit angle φ on the object side by the aperture D of the photographing optical system defined by the diaphragm 10 or the like. The image is limited to the angle θ and forms an image at a point B on the image side. Assuming that the focal length of the photographing optical system 1 is f, a point A separated by a distance a on the object side forms an image at a point B advanced by a distance b on the image side, and satisfies the following relational expression.

1 / a + 1 / b = 1 / f (Equation 1)
Such an imaging state is defined by the focusing distance of the photographing optical system 1 and is determined by performing an automatic focus detection (AutoFocus = AF) operation for each photographing scene, or set by a user by focusing or the like. . In FIG. 3A, the point A is drawn so that the point B comes to the imaging surface on the microlens of the solid-state imaging device 2 while satisfying (Equation 1). In focus distance. Of the light from the point A at the in-focus distance, the upper limit light beam that passes through the upper end of the aperture D from the light beam on the optical axis is incident on the left photoelectric conversion unit 20L, and the light beam with the aperture D from the light beam on the optical axis. The lower limit light beam passing through the lower end part enters the right photoelectric conversion unit 20R included in the same pixel. In this way, the point A at the in-focus distance is imaged on the right photoelectric conversion unit 20R at the same pixel position as the left photoelectric conversion unit 20L, and therefore, the electric signal image indicated by 20L and 20R. Are in a state without any deviation.

  By the way, the combination of the point A on the object side and the point B on the image side that satisfies (Equation 1) is not limited to the above-described in-focus state. In FIG. 3B, the light emitted from the point A ′ farther from the photographing optical system 1 than the point A does not form an image on the imaging surface, and forms an image at a point B ′ closer to the photographing optical system 1 than the point B. In the ray diagram, (a) in (Equation 1) is replaced with a ′, and b is replaced with b ′. Although not shown again, conversely, light emitted from one point on the optical axis closer to the photographing optical system 1 than the point A forms an image at a point farther from the photographing optical system 1 than the point B. . Therefore, the point on the optical axis at a so-called out-of-focus distance that is out of focus from the imaging surface is imaged by shifting the imaging point forward and backward of the imaging surface according to (Equation 1). To do. Accordingly, an image shift occurs between one of the left and right photoelectric conversion units that collects light rays on the optical axis to the upper limit light beam and the other photoelectric conversion unit that collects light rays on the optical axis to the lower limit light. This image shift is called parallax, and its absolute amount is called parallax amount.

  Further, in FIG. 3C, the aperture D of the photographing optical system 1 is larger than that in FIG. 3A, and the maximum angle θ of the light incident on the photoelectric conversion units 20L and 20R is large. The photoelectric conversion units 20L and 20R show a state where the emission angle φ at which the point A is desired is also large. The relationship between φ and D is roughly expressed by the following equation using the distance a to the point A.

tanφ = D / 2a (Equation 2)
Here, a virtual aperture D0 when an average ray passes through the stop 10 out of all rays from the ray on the optical axis to the ray passing through the end of the aperture D is defined. An angle φ0 obtained by replacing D in Equation 2 with D0 is an angle corresponding to the convergence angle in the stereo camera system. (Equation 2), the convergence angle φ0 is monotonically increasing with respect to the virtual aperture D0, and considering that D0 is monotonically increasing with respect to the aperture D defined by the aperture 10 or the like, the convergence angle φ0 is It can be seen that the diameter increases monotonously with the aperture D of the diaphragm 10. Further, in FIG. 3C, if a light beam emitted from a point A ′ farther from the photographing optical system 1 than the point A is plotted as shown in FIG. 3B, the aperture of the photographing optical system 1 or the diaphragm 10 is used. It can also be seen that the amount of parallax at a point at the non-focus distance increases. As described above, the aperture or aperture 10 of the photographing optical system 1 is a parameter that can change the stereoscopic effect of the subject.

  By the way, the photographing optical system 1 forms an image of a light beam from an object existing at infinity at a point advanced by the focal length f to the image side. FIG. 4 is a graph showing this by taking the subject distance on the horizontal axis and the distance to the imaging point on the vertical axis. The farther the subject is imaged near the focal length f, the left and right parallax amounts of the farther subject are less likely to change with a constant amount, and the perspective difference is less than that of the subject at a closer distance. Hard to occur. Such a tendency becomes more prominent as the aperture D of the photographing optical system 1 is smaller.

  Heretofore, the basic mechanism for obtaining a plurality of viewpoints, which is configured by the combination with the imaging optical system 1 in addition to the configuration of the solid-state imaging device 2 has been described. In addition, about the structure for the electrical signal transfer in the solid-state image sensor 2, the CCD type with charge transfer, the Active-CMOS type having an amplification amplifier for each pixel, and the like can be considered. However, any other configuration is applicable as long as it can output electrical signals from the photoelectric conversion units for a plurality of viewpoints of the present invention.

  In the present embodiment, the first shooting is performed to limit the amount of parallax of the subject that is closer than the in-focus distance, while the amount of parallax of the subject in the vicinity of the in-focus distance is increased as much as possible. A second image is taken.

  FIG. 5 is a flow showing detailed control at this time.

  The main power supply is turned on by a switch not shown in FIG. 1, and then the power supply of the system control unit 13 is turned on (step S501).

  Next, the mechanical shutter is opened, and a drive setting signal is given to the solid-state imaging device 2 (step S502). As a result, a mode in which a live image is continuously displayed on the image display device 12 can be executed, and at the same time, photometry is performed using continuously acquired images and exposure used for photographing is adjusted, so-called automatic exposure adjustment. Operation (AE) is also possible (step S503). That is, the system control unit 13 acquires the digitized image signal from the imaging system in the image memory 8 and causes the signal processing circuit 7 to perform an exposure calculation. For example, the luminance information extracted from the image signal is weighted for each area of the screen to calculate how much brighter (or darker) the current information should be. The system control unit 13 receives the calculation result and drives the aperture of the photographing optical system 1 to obtain an image with appropriate brightness from the next frame. When the solid-state imaging device 2 has an electronic shutter function, the shutter speed may be changed at the same time. In this AE, a plurality of combinations of aperture values, shutter speeds, and ISO sensitivity as necessary are prepared for shooting, and prepared for first and second shooting described later. In this embodiment, a combination as shown in (Table 1) is detected as a candidate.

  Next, it is determined whether or not a first stroke of a shutter release button (not shown) having a two-stage stroke has been performed (step S504). When it is determined that the first stroke has been performed, focus position adjustment, so-called autofocus (AF), is automatically started to automatically drive the focus adjustment lens so that the subject is in focus (step S505). On the other hand, if it is determined that the first stroke is not performed, the process returns to step S503, and the operations of steps S503 to S504 are repeated. In AF, first, the focus adjustment lens of the photographing optical system 1 is driven in a plurality of steps to obtain a plurality of image signals. The subject distance measuring means 17 performs arithmetic processing on each image signal to determine the position of the focus adjustment lens that is most focused on the subject. In response to this detection result, the system control unit 13 drives the focus adjustment lens of the photographic optical system 1 to obtain an optimum focus image from the next frame. Note that a method of measuring a subject distance from a signal of a distance measuring sensor serving as a subject distance measuring unit 17 provided separately from the imaging system and driving a focus adjustment lens may be used. In either method, the distance information of the subject can be acquired, and this is used for the first shooting and the second shooting described later.

Prior to the first photographing, the system control unit 13 confirms that the subject closest to the photographing optical system 1 can be fused. Specifically, the calculation is performed as follows while checking against the exposure condition candidates shown in (Table 1). In the calculation, (Equation 1) is used, the focal length of the photographic optical system 1 is f, the in-focus distance is a, and the distance to the subject closest to the photographic optical system 1 (first distance) is s (s < Assuming that a), the imaging position of this subject is expressed by (Equation 1) as follows: sf / (s−f) (Equation 3)
Is calculated. Since the in-focus distance is a, the imaging surface (the surface of the microlens 21) of the solid-state imaging device 2 is away from the imaging optical system 1 by af / (af). On the imaging surface, the parallax amount of the photoelectric conversion units 20L and 20R formed by the light rays from the point of the subject distance s is D0 ((s−f) / sf) [(sf / (s−f)) − (af / (Af))] (Equation 4)
It is expressed. Here, as described above, D0 is a virtual aperture when an average light beam passes through the aperture 10, and has a substantially constant correlation with the aperture D of the aperture 10.

  Then, the parallax amount of (Equation 4) is calculated from the aperture candidates present in (Table 1) (step S506). For example, the aperture D corresponding to the stop F4 in (Table 1) corresponds to about ¼ of the focal length. If the virtual aperture D0 when the average light beam passes through the stop 10 is estimated to be about 1/8 of the focal length, for example, it becomes 12.5 mm under the imaging condition of the focal length of 100 mm. From (Expression 4), the parallax amount of the subject with s = 2100 mm in the state where the focusing distance a = 3100 mm is estimated to be approximately 0.2 mm. This corresponds to about 40 pixels when the pixel interval of the solid-state imaging device 2 is 5 μm.

  When such a parallax amount is larger than a preset threshold value (predetermined first threshold value) (No in step S507), a diaphragm with a smaller aperture than the diaphragm F4 (second table) is selected from (Table 1). Is applied again to estimate the amount of parallax (Yes in step S508). If it is equal to or less than the predetermined first threshold, it is determined that fusion is possible (Yes in step S507). Then, the convergence angle setting means 16 sets the aperture value, that is, the first convergence angle, and proceeds to the release standby in preparation for the first imaging.

  When calculating the amount of parallax when the second candidate is applied, it is convenient to use the fact that the amount of parallax is proportional to D0 in (Equation 4) while referring to the estimated value of the first candidate. If it is not determined that fusion is possible even with the second candidate (No in step S507), a third candidate may be applied (Yes in step S508). If there is no condition that satisfies the threshold value among the diaphragms designated as candidates in advance (No in step S508), the imaging conditions are re-established so that a diaphragm with a smaller aperture can be used using the photometric data. It may be set (step S509).

  After the appropriate aperture and focus distance are determined as described above, it is determined whether or not the shutter release button has been pressed (ON) (step S510). If it is determined that the shutter release button has not been pressed (No in step S510), step S510 is repeated until it is determined that the shutter release button has been pressed. On the other hand, if it is determined that the shutter release button has been pressed (Yes in step S510), the first shooting is executed (step S511). As described above, the first photographing is performed by determining the possibility of fusing the closest subject and using the fusing stop. When the first photographing operation is completed, the output signal of the solid-state imaging device 2 is read, and A / D conversion is performed by the A / D converter 4 (step S512). Next, the signal processing circuit 7 performs various signal processing such as white balance and gamma correction on each of the image formed by the set of photoelectric conversion units 20L and the image formed by the set of 20R, and is recorded on the recording medium 10 to be recorded. 1 is completed (step S513).

  Next, depending on whether or not to allow bracket shooting, if there is permission, the process proceeds to the second shooting (Yes in step S514). If there is no permission, the flow ends (No in step S514). The second imaging is performed by setting the angle of convergence as large as possible by paying attention to the subject existing at the farthest distance.

Prior to the second shooting, the system control unit 13 checks how much the amount of parallax change occurs in the subject farthest from the shooting optical system 1 (step S515). Specifically, the calculation is performed while checking against the exposure condition candidates shown in (Table 1). For the calculation, when (Equation 1) is used and the distance (second distance) from the photographing optical system 1 to the farthest subject is s (s> a), the parallax amount of the subject is
−D0 ((s−f) / sf) [(sf / (s−f)) − (af / (a−f))] (Equation 5)
It is represented by Here, “−” corresponds to the fact that the direction of the parallax between the left and right photoelectric conversion units is opposite to that in the case of (Equation 4).

  The amount of change in parallax in the subject farthest from the photographic optical system 1 can be determined by confirming the depth of the farthest subject. Most simply, a plurality of points on the object farthest from the photographing optical system 1, that is, a distance s1 to the front end and a distance s2 to the rear end may be confirmed.

In that case, the parallax amounts at the front end s1 and the rear end s2 of the farthest subject are respectively
-D0 ((s1-f) / s1f) [(s1f / (s1-f))-(af / (af))] (Equation 6)
-D0 ((s2-f) / s2f) [(s2f / (s2-f))-(af / (af))] (Equation 7)
The change in parallax amount is obtained by the difference between (Equation 6) and (Equation 7).

  For example, it is assumed that one of the farthest subjects is distributed in s = 5100 mm to 6100 mm. Then, the parallax amount corresponding to each is 0.164 mm to 0.205 mm, and the change in the parallax amount from the front end to the rear end is about 0.04 mm even though the object is 1000 mm in the depth direction. . Therefore, if the pixel interval of the solid-state imaging device 2 is 5 μm, there is only a difference of 8 pixels. In other words, although it is expressed that such a subject is a subject that is deeper than the in-focus distance under the first shooting, the stereoscopic effect of the subject itself is sufficiently expressed compared to the actual shooting. There was no. Therefore, when the amount of parallax change is smaller than a preset threshold (predetermined second threshold), it is determined that the stereoscopic effect of the distant subject is insufficient (Yes in step S516). Then, the convergence angle setting means 16 sets the fourth or fifth candidate of (Table 1), that is, the second convergence angle, and proceeds to the release standby in preparation for the second imaging. On the other hand, if it is determined that the disparity amount change of the distant subject is sufficient (the disparity change amount is equal to or larger than the predetermined second threshold value), the flow ends (No in step S516). In the case of F5, which is the fifth candidate, the aperture is twice that of F4. Therefore, as can be seen from (Equation 5), the difference in the amount of parallax between the distant subjects is doubled to 0.08 mm.

  When the second imaging (step S517) is completed, the output signal of the solid-state imaging device 2 is read, and A / D conversion is performed by the A / D converter 4 (step S518). Next, the signal processing circuit 7 performs various signal processing such as white balance and gamma correction on each of the image formed by the set of photoelectric conversion units 20L and the image formed by the set of 20R, and is recorded on the recording medium 10 to be recorded. 2 is completed (step S519).

  As described above, in the second photographing, in addition to setting the aperture D in consideration of the fact that the convergence angle is small for a far subject that is a long distance from the photographing optical system 1, it is further commensurate with the depth amount of the far subject. The diameter D can be set so that the amount of parallax change is generated. In the present embodiment, an example in which the amount of change in parallax of a subject farthest from the photographing optical system 1 is calculated in the second shooting is shown, but the present invention is not limited to this, and the focus distance or focus is calculated. You may calculate the variation | change_quantity of the parallax of the to-be-distant subject from distance.

  According to the first embodiment described above, it is possible to perform the first stereoscopic shooting that allows the user to appreciate the entire shooting composition without feeling uncomfortable, and the second stereoscopic shooting that does not impair the stereoscopic effect of the subject of interest or a distant subject. Therefore, for example, when a distant subject is cut out and saved from the obtained stereoscopic image and viewed, it can be viewed without impairing the stereoscopic effect. Therefore, it is possible to meet the demands of various users during viewing.

  As a use of the stereoscopic image by the second photographing in the first embodiment, it is conceivable to view and view a subject at a focal distance or farther away. If a main subject to be cut out and stored is identified and an optimal change in parallax amount can be given according to the depth amount, the effect can be further exerted.

  Therefore, in this embodiment, subject detection means 18 is further provided to determine whether or not there is a main subject at the in-focus distance or further away. If it is determined that there is a main subject, the amount of change in parallax is estimated from the depth of the main subject, and shooting is performed so that this is equal to or greater than a preset threshold.

  FIG. 6 is a flow showing detailed control at this time. Since the first imaging is the same as in the first embodiment, it will be described in a simplified manner. When the first imaging (step S601) is completed, the output signal of the solid-state imaging device 2 is read, and A / D conversion is performed by the A / D converter 4 (step S602). Next, the signal processing circuit 7 performs various signal processing such as white balance and gamma correction on each of the image formed by the set of photoelectric conversion units 20L and the image formed by the set of 20R, and is recorded on the recording medium 10 to be recorded. 1 is completed (step S603).

  Next, depending on whether or not to allow bracket shooting, if there is permission, the process proceeds to the second shooting (Yes in step S604). If there is no permission, the flow ends (No in step S604).

  Prior to the second photographing, first, the subject detection means 18 searches for a main subject from subjects at or far from the focusing distance a (step S605). Then, the system control unit 13 confirms the depth amount of the main subject from the distance (second distance) to the subject determined to be main (step S606). Most simply, a plurality of points on the main subject from the photographing optical system 1, that is, a distance s1 to the front end and a distance s2 to the rear end may be confirmed.

  Next, the amount of parallax and how much the change occurs in the main subject is calculated (step S607). The parallax amount of the main subject is expressed by the above (Equation 6) and (Equation 7), and the change in the parallax amount is given by the difference between (Equation 6) and (Equation 7). When the amount of parallax change is smaller than a preset threshold (predetermined second threshold), it is determined that the stereoscopic effect of the main subject is insufficient (Yes in step S608). Then, the convergence angle setting means 16 sets the photographing condition (second convergence angle) with the aperture opened, and proceeds to the release standby in preparation for the second photographing. On the other hand, if it is determined that the parallax amount change of the main subject is sufficient (the parallax change amount is greater than or equal to the threshold value), the flow ends (No in step S608). The threshold value at this time may be given as a table corresponding to the distance or depth of the subject.

  When the second imaging (step S609) is completed, the output signal of the solid-state imaging device 2 is read, and A / D conversion is performed by the A / D converter 4 (step S610). Next, the signal processing circuit 7 performs various signal processing such as white balance and gamma correction on each of the image formed by the set of photoelectric conversion units 20L and the image formed by the set of 20R, and is recorded on the recording medium 10 to be recorded. 2 is completed (step S611).

  According to the second embodiment described above, in the second stereoscopic photographing, in addition to considering that the main subject having a long distance from the photographing optical system 1 has a smaller convergence angle, the depth of the main subject is also considered. It becomes possible to bring about a parallax change amount according to the amount.

  Also in the present embodiment, the first stereoscopic shooting that allows the entire shooting composition to be viewed without a sense of incongruity, and the second stereoscopic shooting that does not impair the stereoscopic effect of the focused main subject or the distant main subject can be performed. Therefore, for example, when a distant main subject is cut out and saved from the obtained stereoscopic image and viewed, it can be viewed without impairing the stereoscopic effect. Therefore, it is possible to meet the demands of various users during viewing.

  Note that the second shooting in the second embodiment may be executed for a plurality of subjects considered to be main.

  FIG. 7B illustrates the sensitivity for each light receiving angle of the photoelectric conversion unit 20L on the left side of the solid-state imaging device 2 that is the partial schematic cross-sectional view illustrated in FIG. 7A, that is, the pupil intensity distribution. An angle θ formed with a perpendicular line standing at the apex of the microlens 21 in a plane parallel to the paper surface of FIG. 7A is a light incident angle having the same meaning as θ in FIG. In FIG. 7B, the light incident angle θ is the horizontal axis, the sensitivity for each light incident angle θ is the vertical axis, and the pupil intensity distribution of the left photoelectric conversion unit 20L is displayed. It is expressed as If the highly sensitive angle range θ1 ≦ θ ≦ θ2 is as wide as the angle range θmax that can be calculated from the aperture D of the photographing optical system 1 limited by the aperture 10 or the like as in (Equation 8), the convergence angle is defined. The angle range to be applied is not limited by the photoelectric conversion unit 20L on the left side. Therefore, the theory of this embodiment holds.

tan θmax = D / 2b (Equation 8)
Here, b represents the imaging distance. The same applies to the photoelectric conversion unit 20R on the right side.

Conversely,
θ2 << θmax (Equation 9)
As described above, when the angular range in which the photoelectric conversion unit can receive light is significantly smaller than θmax, the parallax change amount cannot be increased as calculated. Furthermore, as estimated, photographing on the wide open side has many disadvantages in that the degree of freedom is reduced in terms of exposure range. This is because the signal level is supplemented separately by a normal sensitivity gain in response to the limited light incident angle range, so that only high-speed shutters or shooting with a low ISO sensitivity can be performed after all. .

  Therefore, in this embodiment, a step of limiting the light incident angle in consideration of the pupil intensity distribution of the photoelectric conversion unit is added when setting the aperture D in the second imaging.

  FIG. 8 is a flow showing detailed control at this time.

  Since the first imaging is the same as in the first embodiment, it will be described in a simplified manner. When the first imaging (step S801) is completed, the output signal of the solid-state imaging device 2 is read, and A / D conversion is performed by the A / D converter 4 (step S802). Next, the signal processing circuit 7 performs various signal processing such as white balance and gamma correction on each of the image formed by the set of photoelectric conversion units 20L and the image formed by the set of 20R, and is recorded on the recording medium 10 to be recorded. 1 is completed (step S803).

  Next, depending on whether or not to allow bracket shooting, if there is permission, the process proceeds to the second shooting (Yes in step S804). If there is no permission, the flow ends (No in step S804).

  Regarding the second imaging, the flow from step S805 to step S808 is the same as that in the second embodiment. That is, after Yes in step S804, the main subject is searched (step S805), and the distance and depth amount of the main subject are confirmed (step S806). Next, the parallax change amount of the main subject is calculated (step S807), and the parallax change amount is compared with a preset threshold value (step S808). If the parallax change amount is smaller than a preset threshold value, it is determined that the stereoscopic effect of the main subject is insufficient (Yes in step S808). Then, the convergence angle setting means 16 sets the photographing condition (second convergence angle) with the aperture opened, and proceeds to the release standby in preparation for the second photographing. On the other hand, if it is determined that the change in the amount of parallax of the main subject is sufficient (the amount of change in parallax is greater than or equal to the threshold value), the flow ends (No in step S808). The threshold value at this time may be given as a table corresponding to the distance or depth of the subject.

  After Yes in step S808, the maximum light incident angle θmax passing through the aperture D of the photographing optical system 1 is obtained according to (Equation 8) (step S809). θmax is compared with a threshold value (predetermined third threshold value) θ2 set in advance according to the sensor characteristics. If θmax is smaller than θ2, the process proceeds to the second imaging (Yes in step S810). On the other hand, when θmax exceeds θ2 (or when it significantly exceeds θ2) (No in step S810), the imaging condition is reset (step S811). After step S811, the process returns to step S807.

  When the second imaging (step S812) is completed, the output signal of the solid-state imaging device 2 is read, and A / D conversion is performed by the A / D converter 4 (step S813). Next, the signal processing circuit 7 performs various signal processing such as white balance and gamma correction on each of the image formed by the set of photoelectric conversion units 20L and the image formed by the set of 20R, and is recorded on the recording medium 10 to be recorded. 2 is completed (step S814).

  In the present embodiment, in order to clarify the difference from the second embodiment, it has been described as a feature that a determination step such as step S809 or step S810 in FIG. 8 is provided. However, this embodiment is characterized in that the aperture D is set while adjusting so as not to significantly exceed θ2 according to the sensor characteristics, and is not limited to the configuration of the determination step as described above. For example, if the sensor is fixed, the condition for defining the aperture D from (Equation 8) and (Equation 9) is only the imaging distance b of the photographing optical system 1. In such a case, it is only necessary to set the aperture D so as not to exceed a predetermined value determined by the photographing optical system 1.

  According to the third embodiment described above, the amount of parallax change according to the distance and depth of the main subject from the photographing optical system 1 without inadvertently applying an open aperture in the second stereoscopic photographing. It becomes possible to bring

  In the first to third embodiments, there is a possibility that the depth of field becomes shallow due to the large aperture used in the second shooting, and the main subject farther than the in-focus distance cannot be focused. is there.

  Therefore, in this embodiment, a step of resetting the focusing distance in consideration of the depth of field of the photographing optical system 1 is added when the aperture D is set in the second photographing.

  FIG. 9 is a flow showing detailed control at this time.

  Since the first imaging is the same as in the first embodiment, it will be described in a simplified manner. When the first imaging (step S901) is completed, the output signal of the solid-state imaging device 2 is read, and A / D conversion is performed by the A / D converter 4 (step S902). Next, the signal processing circuit 7 performs various signal processing such as white balance and gamma correction on each of the image formed by the set of photoelectric conversion units 20L and the image formed by the set of 20R, and is recorded on the recording medium 10 to be recorded. 1 is completed (step S903).

  Next, depending on whether or not the bracket shooting is permitted, if there is permission, the process proceeds to the second shooting (Yes in step S904). If there is no permission, the flow ends (No in step S904).

  Regarding the second imaging, the flow from step S905 to step S908 is the same as that in the second or third embodiment. That is, after Yes in step S904, the main subject is searched (step S905), and the distance and depth amount of the main subject are confirmed (step S906). Next, a parallax change amount of the main subject is calculated (step S907), and the parallax change amount is compared with a preset threshold value (step S908). If this parallax change amount is smaller than a preset threshold value, it is determined that the stereoscopic effect of the main subject is insufficient (Yes in step S908). Then, the convergence angle setting means 16 sets the photographing condition (second convergence angle) with the aperture opened, and proceeds to the release standby in preparation for the second photographing. On the other hand, if it is determined that the change in the amount of parallax of the main subject is sufficient (the amount of change in parallax is greater than or equal to the threshold value), the flow ends (No in step S908). The threshold value at this time may be given as a table corresponding to the distance or depth of the subject.

  Next, the rear depth of field of the photographing optical system 1 is obtained according to (Equation 10) (step S909).

(Af) ² / (fD / δ-a) (Equation 10)
Here, a is the focusing distance, f is the focal length of the photographing optical system 1, D is the aperture of the photographing optical system 1, and δ is an allowable circle of confusion.

  If it can be determined that the main subject is within the depth of field, the process proceeds to the second shooting (Yes in step S910). If it is determined that it does not fall within the depth of field (or does not fall significantly within the depth of field) (No in step S910), the in-focus distance is reset (step S911), and the next step S912 is performed. Proceed to the second shooting. The threshold for determining whether or not the depth of field is within the range may be increased or decreased in consideration of how much high frequency components are included in the main subject. If the main subject contains a lot of high frequency components, it is particularly effective to reset the in-focus distance.

  When the second imaging (step S912) is completed, the output signal of the solid-state imaging device 2 is read, and A / D conversion is performed by the A / D converter 4 (step S913). Next, the signal processing circuit 7 performs various signal processing such as white balance and gamma correction on each of the image formed by the set of photoelectric conversion units 20L and the image formed by the set of 20R, and is recorded on the recording medium 10 to be recorded. 2 is completed (step S914).

  According to the fourth embodiment described above, in the second stereoscopic shooting, it is possible to maintain the focus state of the main subject for which the amount of parallax change is to be enlarged. Furthermore, if the in-focus distance is reset and the aperture is opened, the subject close to the photographic optical system 1 will not be able to focus in the first place. Therefore, even if the amount of parallax becomes excessive, it is possible to view without discomfort. There are also positive effects.

  The third embodiment obtains a photographing condition that restricts the aperture D to a predetermined value or less, and the fourth embodiment has a side surface that resets the in-focus distance so that the aperture D can be enlarged to a predetermined value or more. For the purpose of enlarging the parallax in the second shooting, it is often desirable to enlarge the aperture D on the basis of the fourth embodiment. However, for the purpose of avoiding light incident at an excessive angle when the aperture is larger than a certain diameter, it is preferable to add a restriction as in the third embodiment.

  Up to now, there have been described problems and solutions for stereoscopic imaging using the imaging optical system 1 and the solid-state imaging device 2 having a plurality of photoelectric conversion units in the unit pixel shown in FIG. However, a part of the present embodiment does not necessarily require such an imaging system, and can also be applied to a so-called stereo camera type stereoscopic image capturing apparatus. In this embodiment, an application example will be described.

  FIG. 10A is a plan view illustrating an example of imaging by a stereo camera-type stereoscopic image capturing apparatus. 1000 to 1002 represent subjects included in the object scene and dispersed in the depth direction. Reference numerals 1003 and 1004 denote imaging units corresponding to the right and left viewpoints, and each independently includes a photographing optical system and a solid-state imaging element. The solid-state imaging device in the present embodiment does not necessarily have the configuration shown in FIG. The distance D between 1003 and 1004 may be referred to as the baseline length. Further, in FIG. 10A, the viewpoints from 1003 and 1004 intersect just at the subject 1001. Such an intersection may be referred to as a convergence point. In addition, an angle α at which the line of sight from 1003 and the line of sight from 1004 intersect may be referred to as a convergence angle. Further, the surface R including the convergence point may be referred to as a reference surface. Reference numeral 1005 schematically shows an imaging screen from the right viewpoint.

  Although the magnification of the subject varies depending on the focal length of the lens used, the subject 1000 is viewed from the right viewpoint 1003 due to the base line length D and the convergence angle φ between the two imaging units shown in FIG. Are imaged as being separated by an arrow line 1007. Further, the image is formed as being separated from the subject 1002 by an arrow line 1006 when viewed from the right viewpoint 1003. Taking advantage of the fact that these parallax directions are opposite from the viewpoint on the left side, stereoscopic image capturing is performed. Although not shown in FIG. 10, in the stereo camera-type stereoscopic image capturing apparatus of the present embodiment, the angle of convergence of the plurality of image capturing units is controlled by rotating the image capturing unit 1003 and the image capturing unit 1004. A convergence angle setting means for adjusting and setting is provided. In addition, interval setting means for adjusting and setting the interval between the imaging unit 1003 and the imaging unit 1004 is provided.

  FIG. 10B is a plan view illustrating an imaging example in which the base line length D of FIG. 10A is increased and the convergence angle φ is increased with the subject 1001 as a reference plane. It can be seen that the arrows 1007 and 1006 representing the amount of parallax between the subject 1000 in front of the reference plane and the subject 1002 behind are longer. As already described in other embodiments, the shooting conditions with a large amount of parallax and the shooting conditions with a small amount of parallax have advantages. If the amount of parallax is relatively small, both the subject 1000 in the foreground and the subject 1002 in the back can reasonably perform stereoscopic viewing, whereas the subject 1001 near the reference plane tends to have poor stereoscopic effect. . Conversely, if the amount of parallax is increased, the stereoscopic effect of the subject 1001 can be expanded, but the amount of parallax of the subject 1000 in the foreground and the subject 1002 in the back may be too large, and fusion may not be possible during stereoscopic viewing. There is also.

  Therefore, in the present embodiment, the first shooting for limiting the amount of parallax of the subject included in the shooting composition is performed, while the second for increasing the amount of parallax of the subject near the reference plane as much as possible. Take a picture.

  FIG. 11 is a flow showing detailed control at this time.

  The main power supply is turned on by a switch not shown in FIG. 1, and then the power supply of the system control unit 13 is turned on (step S1101).

  Next, the mechanical shutter is opened, and a drive setting signal is given to the solid-state imaging device 2 (step S1102). As a result, a mode in which a live image is continuously displayed on the image display device 12 can be executed, and at the same time, photometry is performed using continuously acquired images and exposure used for photographing is adjusted, so-called automatic exposure adjustment. Operation (AE) is also possible (step S1103). Details of AE are omitted because they are common to the other embodiments.

  Next, it is determined whether or not a first stroke of a shutter release button (not shown) having a two-stage stroke has been performed (step S1104). If it is determined that the first stroke has been performed, focus position adjustment for automatically driving the focus adjustment lens so as to focus on the subject, so-called autofocus (AF) is started (step S1105). On the other hand, if it is determined that the first stroke is not performed, the process returns to step S1103, and the operations of steps S1103 to S1104 are repeated. The details of AF are also omitted since they are common to the other embodiments. In AF, since distance information of a subject can be acquired, this is used for first and second imaging described later.

Prior to the first imaging, the system control unit 13 calculates a settable convergence angle range while confirming the depth of the subject included in the imaging composition. First, subject distance information and optical information such as the diagonal size y, the focal length f, and the distance s to the reference plane of the solid-state imaging device 2 are acquired (step S1106), and the diagonal images taken by the left and right imaging units 1003 and 1004 are captured. The size of the space is calculated (step S1107). The size of the diagonal space is approximately y × s / f (Equation 11)
It is represented by If an excessive amount of parallax is provided as compared to such a diagonal space, fusion cannot be performed. Therefore, the maximum allowable maximum parallax amount Dif1 which is a function of (Equation 11) is obtained (step S1108).

  Next, the convergence angle setting means 16 calculates and sets the convergence angle (step S1109). Assuming that the distance from the midpoint of the imaging units 1003 and 1004 to the closest subject is s1, and the distance to the farthest subject is s2, the maximum amount of parallax in the shooting space is the sine of the convergence angle (s2-s1) Therefore, the settable convergence angle is calculated as follows.

φ = Arcsin [Dif1 / (s2-s1)] (Equation 12)
After the convergence angle setting means 16 sets the convergence angle, the process proceeds to a release standby in preparation for photographing.

  In the next step, it is determined whether or not the shutter release button has been pressed (ON) (step S1110). If it is determined that the shutter release button has not been pressed (No in step S1110), step S1110 is repeated until it is determined that the shutter release button has been pressed. On the other hand, if it is determined that the shutter release button has been pressed (Yes in step S1110), the first shooting is executed (step S1111). As described above, the first imaging is performed by determining the possibility of fusion of all the subjects included in the imaging composition and using the convergence angle at which fusion is possible. When the first photographing operation is completed, the output signal of the solid-state imaging device 2 is read, and A / D conversion is performed by the A / D converter 4 (step S1112). Next, the signal processing circuit 7 performs various signal processing such as white balance and gamma correction on each of the image captured by the image capturing unit 1003 and the image captured by the image capturing unit 1004, and is recorded on the recording medium 10 to be subjected to the first photographing. Is completed (step S1113).

  Next, depending on whether or not the bracket shooting is permitted, if there is permission, the process proceeds to the second shooting (Yes in step S1114). If there is no permission, the flow ends (No in step S1114). The second imaging is performed by searching for the main subject from the vicinity of the reference plane and setting the convergence angle as large as possible.

  Prior to the second shooting, first, the subject detection means 18 searches for a main subject from subjects near the reference plane (step S1115). Then, the system control unit 13 confirms the depth amount of the main subject from the distance to the subject determined to be main (step S1116). Most simply, the difference Δs between the distance from the midpoint of the imaging units 1003 and 1004 to the front end of the main subject 1001 and the distance to the rear end may be confirmed.

  Next, the range of the convergence angle that can be set is estimated from the change in the amount of parallax generated in the imaging space for the main subject, and the convergence angle is determined (step S1117). In the following equation, Dif2 is given as the minimum amount of parallax necessary to expand the stereoscopic effect, unlike Dif1, and does not necessarily depend on (Equation 11) representing the imaging diagonal space. The convergence angle setting means 16 calculates the convergence angle φ according to (Equation 12) as shown in (Equation 13).

φ = Arcsin [Dif2 / Δs] (Equation 13)
When the second imaging (step S1118) is completed, the output signal of the solid-state imaging device 2 is read, and A / D conversion is performed by the A / D converter 4 (step S1119). Next, the signal processing circuit 7 performs various signal processing such as white balance and gamma correction on each of the image captured by the image capturing unit 1003 and the image captured by the image capturing unit 1004, and is recorded on the recording medium 10 to be second photographed. Is completed (step S1120).

  According to the fifth embodiment described above, it is possible to perform the first stereoscopic shooting that allows the user to appreciate the entire shooting composition without feeling uncomfortable and the second stereoscopic shooting that does not impair the stereoscopic effect of the main subject near the reference plane. Therefore, for example, when a subject near the reference plane is cut out and saved from the obtained stereoscopic image and viewed, it can be viewed without impairing the stereoscopic effect. Therefore, it is possible to meet the demands of various users during viewing.

The preferred embodiments have been described above, but the present invention is not limited to these embodiments and can be applied. Various modifications and changes can be made within the scope of the gist.
(Other embodiments)
The object of the present invention can also be achieved as follows. That is, a storage medium in which a program code of software in which a procedure for realizing the functions of the above-described embodiments is described is recorded is supplied to the system or apparatus. The computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program code stored in the storage medium.

  In this case, the program code itself read from the storage medium realizes the novel function of the present invention, and the storage medium and program storing the program code constitute the present invention.

  Examples of the storage medium for supplying the program code include a flexible disk, a hard disk, an optical disk, and a magneto-optical disk. Further, a CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-R, magnetic tape, nonvolatile memory card, ROM, or the like can also be used.

  Further, by making the program code read by the computer executable, the functions of the above-described embodiments are realized. Furthermore, when the OS (operating system) running on the computer performs part or all of the actual processing based on the instruction of the program code, the functions of the above-described embodiments are realized by the processing. Is also included.

  Furthermore, the following cases are also included. First, the program code read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instruction of the program code, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing.

  The present invention can be suitably used for a stereoscopic image capturing apparatus such as a stereo camera type or a single-lens reflex camera.

DESCRIPTION OF SYMBOLS 13 ... System control part 16 ... Convergence angle setting means 17 ... Subject distance measurement means 20L ... Left photoelectric conversion part 20R contained in the unit pixel of the solid-state image sensor 2 ... Solid-state image sensor 2 Right-side photoelectric conversion unit 1203... Right-side imaging unit 1204.

Claims (11)

A plurality of imaging means for imaging a subject from a plurality of different viewpoints;
A convergence angle setting means for setting a convergence angle between the plurality of imaging means;
Subject distance measuring means for measuring the distance from the imaging means to the subject;
A control unit that performs first imaging and second imaging using the imaging unit;
The plurality of imaging means are a plurality of photoelectric conversion units provided in one pixel of the imaging element,
The convergence angle setting means includes:
In the first photographing, a parallax amount formed on the imaging surface by a point at the first distance closest to the imaging means among the distances measured by the subject distance measuring means is a predetermined first threshold value. Set the first convergence angle to be
In the second shooting, the amount of parallax change made on the imaging surface by a plurality of points on the subject at a second distance farther than the first distance among the distances measured by the subject distance measurement unit Is equal to or greater than a predetermined second threshold, and the maximum light incident angle on the photoelectric conversion unit is calculated, and the maximum light incident angle is smaller than a predetermined third threshold. A stereoscopic image capturing apparatus, wherein a second convergence angle different from the first convergence angle is set.   The stereoscopic image capturing apparatus according to claim 1, wherein the second distance is a distance farthest from the imaging unit. Further comprising subject detection means for detecting the subject,
The convergence angle setting means includes:
In the second shooting, a parallax change amount formed on the imaging surface by a plurality of points on the subject at the second distance detected by the subject detection unit is set to be equal to or larger than a predetermined second threshold value. The stereoscopic image capturing apparatus according to claim 1, wherein the second convergence angle is set.   4. The control unit according to claim 3, wherein when the subject detection unit detects a plurality of subjects, the control unit performs a plurality of second photographing using the imaging unit with respect to the plurality of subjects. The three-dimensional image pickup device described.   5. The stereoscopic image capturing apparatus according to claim 1, wherein the plurality of points on the subject are a front end and a rear end of the subject. The stereoscopic image capturing apparatus according to claim 1, wherein the convergence angle setting unit is an aperture setting unit. The convergence angle setting means includes:
Calculating the depth of field of the imaging optical system, so that the subject fits into the depth of field in depth, any one of claims 1 to 6, characterized in that setting the second angle of convergence The three-dimensional image pickup device described in 1. Photographic optics,
Camera system characterized by having a a stereoscopic image pickup apparatus according to any one of claims 1-7. A method for controlling a stereoscopic image capturing apparatus having a plurality of image capturing means for capturing a subject from a plurality of different viewpoints,
A convergence angle setting step of setting a convergence angle between the plurality of imaging means;
A subject distance measuring step for measuring a distance from the imaging means to the subject;
A control step of performing a first photographing and a second photographing using the imaging means,
The plurality of imaging means are a plurality of photoelectric conversion units provided in one pixel of the imaging element,
The convergence angle setting step includes:
In the first photographing, a parallax amount formed on the imaging surface by a point at the first distance closest to the imaging unit among the distances measured in the subject distance measurement step is a predetermined first threshold value. Set the first convergence angle to be
In the second shooting, the amount of parallax change made on the imaging surface by a plurality of points on the subject at a second distance farther than the first distance among the distances measured in the subject distance measurement step Is equal to or greater than a predetermined second threshold, and the maximum light incident angle on the photoelectric conversion unit is calculated, and the maximum light incident angle is smaller than a predetermined third threshold. A control method for a stereoscopic image capturing apparatus, wherein a second convergence angle different from the first convergence angle is set. A computer-executable program in which a procedure of a control method for a stereoscopic image capturing apparatus according to claim 9 is described. A computer-readable storage medium storing a program for causing a computer to execute each step of the control method of the stereoscopic image capturing apparatus according to claim 9 .