JP2010231192A - Stereoscopic imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a stereoscopic imaging apparatus for creating a stereoscopic image having a deep depth of field, without reducing the quantity of light from an object. <P>SOLUTION: A pan focus processing circuit 18R obtains the synthesized image data of a large number of image surfaces of which the position is deviated little by little, from an arithmetic processing circuit 15R. For each of the plurality of synthesized image data, a difference in brightness value between an attention pixel and a pixel in the vicinity of it is calculated to calculate the contrasts of each pixel included in the synthesized image data. Then, the contrasts of each pixel existing in the same position between the plurality of synthesized image data are compared with each other, to take out the pixel whose contrast is maximum and create pan focus image data. A control circuit 101 creates a pair of image data having parallax, based on two pan focus image data. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a stereo imaging device.

  A stereo imaging device that captures a stereo image including a right-eye image and a left-eye image using two imaging optical systems is known (for example, Patent Document 1). Such a stereo imaging device causes parallax to occur in two images obtained by imaging the same subject by arranging two imaging optical systems at a predetermined interval.

  It is desirable that the stereo image has a deep depth of field. This is because the stereoscopic effect is often more clearly felt when the subject is sharper than the perspective, compared with the case where the subject is not. For example, it is possible to increase the depth of field of a stereo image by reducing the aperture diameter of the imaging optical system.

  On the other hand, Patent Document 2 describes an imaging apparatus that synthesizes image data of an image on an arbitrary image plane. This imaging apparatus is focused on a subject located at an arbitrary shooting distance by taking out an output of a predetermined photoelectric conversion element from a pixel array arranged behind each microlens constituting the microlens array. It has a function to synthesize image data.

Japanese Patent Laid-Open No. 8-47001 JP 2007-4471 A

  When the aperture diameter of the imaging optical system is reduced in order to increase the depth of field of the stereo image, there is a problem in that the amount of light from the subject decreases and image noise increases.

The invention according to claim 1 is a first image creation for creating a pair of images having parallax based on at least two imaging units that output an image signal of a subject and the image signals output by the two imaging units. And at least one of the two imaging units is two-dimensionally arranged in the vicinity of the imaging optical system and a predetermined imaging plane of the imaging optical system. A plurality of pixels having a plurality of positive lenses and a photoelectric conversion function that are two-dimensionally arranged behind each of the plurality of positive lenses and receive light beams from different regions of the exit pupil of the imaging optical system. And a pan-focus imaging unit having an output device capable of outputting a pan-focus image signal based on an output of the imaging device.
The invention according to claim 6 has parallax based on at least two imaging units that output an image signal of a subject, distance measuring means that detects a distance to the subject, and image signals output by the two imaging units. First image creating means for creating a pair of images, and second image creating means for creating a pair of images having parallax based on an image signal output from one of the two imaging units. And at least one of the two imaging units includes an imaging optical system and a plurality of positive pixels arranged two-dimensionally in the vicinity of a predetermined imaging plane of the imaging optical system. A plurality of pixels that are two-dimensionally arranged on the rear side of each of the lens and the plurality of positive lenses and that have a photoelectric conversion function of receiving light beams from different regions of the exit pupil of the imaging optical system, Pixel An image pickup unit having an image pickup device for outputting an output as an image signal, wherein the second image creating means creates a pair of images having parallax based on the image signal output by the composite image pickup unit, and ranging means A pair of images having parallax is created by the first image creation means when the distance detected by the distance measurement is greater than or equal to a predetermined value, and the second image creation means when the distance detected by the distance measurement means is less than the predetermined value This is a stereo imaging device characterized in that a pair of images having parallax is created.

  According to the present invention, a stereo image can be obtained using a pan focus imaging unit.

It is a figure which shows the structure of the stereo imaging device by 1st Embodiment. It is a figure which shows the structure of the imaging units 100L and 100R. It is a figure which shows an example of the image composition function of the arithmetic processing circuit 15L. It is a figure which shows the composite image data of the image surface Z1, and the composite image data of the image surface Z2. It is a figure which shows the process which produces pan-focus image data. It is a figure which shows the principle which produces stereo image data only using the imaging unit 100L. 3 is a diagram illustrating a relationship between a pixel array and a pair of image data 124R and 124L created by a control circuit 101. FIG. It is a figure which shows the process which calculates the pixel value of image data from the output of a photoelectric conversion element. It is a figure which shows the process which obtains a lenticular type | mold stereoscopic image using two imaging units. It is a figure which shows the process which acquires a lenticular type | mold stereoscopic image using one imaging unit. It is a figure which shows the structure of the stereo imaging device by 3rd Embodiment. It is a figure which shows the structure of the imaging units 100L and 200R. It is a figure explaining the outline | summary of the adjustment function of a convergence angle. It is an enlarged view of the micro lens array 12L and the image sensor 13L in FIG. It is a figure which shows the change of a convergence angle.

-First embodiment-
FIG. 1 is a diagram illustrating a configuration of a stereo imaging device according to the first embodiment. The stereo imaging device 1 includes imaging units 100L and 100R, a control circuit 101, a memory 102, an input device 103, and a distance sensor 104.

  Both of the imaging units 100L and 100R capture image data in which all the subjects from the closest distance to infinity are in focus in one shooting with the shooting optical system focused on the subject located at a predetermined shooting distance. An imaging unit that can be created (hereinafter, such an imaging unit is referred to as a composite imaging unit). Since this image data is image data focused from a near view to a distant view of a subject with a depth, it is called pan focus image data. The control circuit 101 creates a pair of image data having parallax based on the pan focus image data created by the imaging units 100L and 100R, respectively. In the following description, this pair of image data is referred to as stereo image data. The control circuit 101 creates the above-described stereo image data in accordance with an instruction input from the user via the input device 103 and stores it in the memory 102. The distance sensor 104 outputs the result of measuring the distance to the subject to the control circuit 101.

  FIG. 2 is a diagram illustrating the configuration of the imaging units 100L and 100R. The imaging unit 100L includes a photographing lens 11L, a microlens array 12L in which a large number of microlenses ML are two-dimensionally arranged, an imaging element 13L, a drive circuit 14L, an arithmetic processing circuit 15L, a memory 16L, a control circuit 17L, and a pan focus process. A circuit 18L and an output circuit 19L are provided.

  The imaging element 13L is disposed in the vicinity of the focal plane of each microlens ML that constitutes the microlens array 12L. The imaging element 13L is substantially conjugate with the exit pupil of the photographing lens 11L. Note that these positional relationships do not need to be accurate, and correction can be performed in the calculation process of the output of the image sensor 13L. The imaging light flux that has entered the vicinity of the focal plane of the photographic lens 11L is converted into an electrical signal in the image sensor 13L. The image sensor 13L is driven by the drive circuit 14L, and an image signal output from the image sensor 13L is taken into the arithmetic processing circuit 15L via the drive circuit 14L or output to the output circuit 19L. The arithmetic processing circuit 15L synthesizes composite image data based on the image signal. The composite image data is output to the pan focus processing circuit 18L.

The pan focus processing circuit 18L creates pan focus image data based on the composite image data output from the arithmetic processing circuit 15L. The created pan focus image data is stored in the memory 16L. The output circuit 19L outputs the pan focus image data stored in the memory 16L and the image signal output from the drive circuit 14L to the outside of the imaging unit 100L as an image signal. The drive circuit 14L, the arithmetic processing circuit 15L, the memory 16L, the pan focus processing circuit 18L, and the output circuit 19L are all controlled by the control circuit 17L. The control circuit 17L controls each circuit in accordance with a control signal input from the outside of the imaging unit 100L.

  The microlens array 12L is configured by a plurality of microlenses ML arranged in a two-dimensional manner. In the imaging element 13L, a pixel array 130 that receives light that has passed through each of the microlenses ML is arranged in an arrangement pattern corresponding to the microlens ML. Each pixel array 130 includes a plurality of photoelectric conversion elements arranged in a two-dimensional manner.

  Note that the configuration of the imaging unit 100R is the same as that of the imaging unit 100L described above, and a description thereof will be omitted. Further, the principle of combining image data of images formed on arbitrary surfaces by the arithmetic processing circuits 15L and 15R is the same as the content described in Patent Document 2, and thus the description thereof is omitted.

  The imaging units 100L and 100R are installed so that the distance between the photographing lenses 11L and 11R is about 65 mm. This is approximately the same length as the average human eye spacing.

  Next, processing contents of the pan focus processing circuits 18L and 18R will be described. The pan focus processing circuits 18L and 18R create pan focus image data using the composite image data synthesized by the arithmetic processing circuits 15L and 15R. Hereinafter, pan focus image data creation processing executed by the pan focus processing circuit 18L will be described.

  FIG. 3 is a diagram illustrating an example of an image composition function of the arithmetic processing circuit 15L. FIG. 3 illustrates a state in which the small triangular subject 111 and the large quadrangular subject 112 that are present at different positions on the optical axis Z are imaged by the imaging unit 100L. In the following description, the image plane on which the subject image of the subject 111 at the position Zp1 is formed is referred to as Z1, and the image plane on which the subject image of the subject 112 at the position Zp2 is formed is referred to as Z2.

  Based on the principle described in Patent Document 2, the arithmetic processing circuit 15R can synthesize image data of an arbitrary image plane based on an image signal obtained by one imaging. The pan focus processing circuit 18R acquires composite image data of a large number of image planes whose positions are gradually shifted from the arithmetic processing circuit 15R. In other words, for any subject located between the close range and infinity, there is composite image data in which the subject is in focus. In this case, for the sake of simplicity, only the composite image data of the image plane Z1 on which the subject image of the subject 111 is formed and the composite image data of the image plane Z2 on which the subject image of the subject 112 is formed are acquired. Shall.

  FIG. 4 is a diagram illustrating composite image data of the image plane Z1 and composite image data of the image plane Z2. The composite image data 121 shown in FIG. 4A corresponds to the image plane Z1, and the composite image data 122 shown in FIG. 4B corresponds to the image plane Z2. In FIG. 4, the contour of the subject drawn with a solid line indicates that the subject is in focus. Similarly, the contour of the subject drawn with a broken line indicates that the subject is not in focus and the subject image is a blurred image.

  FIG. 5 is a diagram showing a process of creating pan focus image data. The pan focus processing circuit 18L calculates the contrast of each pixel included in the composite image data for each of the plurality of composite image data acquired from the arithmetic processing circuit 15L. The contrast is calculated by calculating a difference in luminance value between the pixel of interest and its neighboring pixels. In FIG. 5, the contrast of the pixel P1 is C1 for the composite image data 121 of the image plane Z1. Similarly, in the image data 122 of the image plane Z2, the contrast of the pixel P2 existing at the same position as the pixel P1 is C2.

  The control circuit 18L calculates the contrast for each pixel of all the composite image data, and then compares the contrast of each pixel existing at the same position between the composite image data. Of these pixels, the pixel having the maximum contrast is extracted. For example, in FIG. 5, since the subject 112 is not in focus on the image plane Z1, the contrast C1 of the pixel P1 that forms the outline of the subject 112 is the pixel P2 at the same position on the image plane Z2 that is in focus on the subject 112. Less than the contrast C2. Accordingly, the pan focus processing circuit 18L takes out the pixel P2 instead of the pixel P1.

  The pan focus processing circuit 18L creates pan focus image data by collecting the pixels extracted by the above processing. For example, in FIG. 5, the pan-focus image data 123 created by the above-described processing is focused on both the subject on which the subject image is formed on the image plane Z1 and the subject on which the subject image is formed on the image plane Z2. Matched image data with deep depth of field. If this process is performed on image data of a large number of image planes that are slightly different in position, pan-focus image data in which all subjects are in focus, from subjects located at close distances to subjects located at infinity. can get. The pan focus processing circuit 18R performs the same processing as described above.

  Next, processing executed by the control circuit 101 shown in FIG. 1 will be described. The control circuit 101 switches processing to be executed between a case where the subject is located at a close distance and a case other than that based on a distance signal indicating the distance to the subject output by the distance sensor 104. In the following, first, a process executed by the control circuit 101 when the subject is farther than the closest distance will be described, and then a process executed by the control circuit 101 when the subject is positioned near the close distance will be described.

  When the subject is located farther than the vicinity of the closest distance, the control circuit 101 outputs an instruction to create pan focus image data to the imaging units 100L and 100R. As a result, pan focus image data is output as an image signal from each of the imaging units 100L and 100R. The control circuit 101 creates two pan focus image data based on the image signals output from these two imaging units. Then, the pair of pan focus image data is stored in the memory 102 as stereo image data.

  Next, a process executed by the control circuit 101 when the subject is located near the close distance will be described. When the subject is located at a close distance, the imaging range of the subject is significantly different between the imaging unit 100L and the imaging unit 100R. As a result, it becomes difficult for an observer of the stereo image to recognize the subject as a stereoscopic image from the stereo image, or the subject is captured only on one of the imaging units. Therefore, when the distance to the subject output by the distance sensor 104 is less than a predetermined length, the control circuit 101 creates stereo image data using only the imaging unit 100L as described below.

  FIG. 6 is a diagram illustrating the principle of creating stereo image data using only the imaging unit 100L. FIG. 6 illustrates a state in which the imaging unit 100L is viewed from above. The pixel array 130 is actually two-dimensional, but only the vertical array is considered here for the sake of simplicity.

In FIG. 6, the area occupied by the projection image on the pupil plane of the photographing lens 11L by the microlens ML corresponding to the photoelectric conversion element a1 is a partial area A1 on the pupil of the photographing lens 11L. Although this projected image is actually a blurred image, even if it is a blurred image, the same thing as the case where there is no blurred is true. Therefore, in the following description, the blurred image is ignored. Like the partial area A1, the partial areas A2, A3,... Correspond to the photoelectric conversion elements a2, a3,. The light beams transmitted through the partial areas A1, A2, A3,... On the pupil of the photographing lens 11L are transmitted through the microlenses and are incident on the photoelectric conversion elements a1, a2, a3,. Partial areas E1, E2, E3
.. Also corresponds to the photoelectric conversion elements e1, e2, e3,.

  The control circuit 101 creates right-eye image data based on the outputs of the photoelectric conversion elements a1, a2, a3,..., And similarly controls the left-eye image based on the outputs of the photoelectric conversion elements e1, e2, e3,. Create image data. These image data creation methods will be described in detail later. Outputs of these photoelectric conversion elements are output to the control circuit 101 via the drive circuit 14L and the output circuit 19L. These two image data are image data obtained by dividing the subject image into pupils on the left and right sides of the photographing lens 11L. That is, these two image data are a pair of image data having a parallax with the interval L between the areas A1 and E1 as a base line length.

  Since the partial areas A1 and E1 are within the pupil of the photographing lens 11L, the baseline length L is shorter than the average human eye interval. If the baseline length is short, the perspective obtained when viewing stereo image data is reduced. However, when the subject is at a close distance, if a pair of image data has a parallax with an average human eye interval, the difference between the left and right images becomes too large to be recognized as a three-dimensional image. Moreover, even when the image can be recognized as a stereoscopic image, the sense of perspective is often felt excessively. Therefore, it is convenient for creating the stereo image data of the subject located at a close distance that the baseline length L is shorter than the average human eye interval.

  With reference to FIGS. 7 and 8, a method of creating image data when a subject at a close distance is photographed will be described in detail. FIG. 7 is a diagram showing the relationship between the pixel array 130 and a pair of image data 124R and 124L created by the control circuit 101. As shown in FIG. For each pixel array 130, the control circuit 101 associates the output of the photoelectric conversion elements e1, e2, e3,... Positioned at the right end in the pixel array with each pixel of the left-eye image data 124L. Similarly, the output of the photoelectric conversion elements a1, a2, a3,... Positioned at the left end in the pixel array is made to correspond to each pixel of the right-eye image data 124R.

  FIG. 8 is a diagram illustrating processing for calculating the pixel value of the image data based on the output of the photoelectric conversion element. In the present embodiment, in the pixel array 130 in which the photoelectric conversion elements are two-dimensionally arranged, a calculation based on the outputs of the three photoelectric conversion elements a1 in the center of the rightmost row indicated by diagonal lines in the drawing is performed, and the left-eye image The luminance value of one pixel of the data 124L is calculated. Specifically, the output values of these three photoelectric conversion elements are added to obtain the luminance value of the pixel L1 shown in the figure. The same applies to a2, a3,..., e1, e2, e3,.

  The correspondence between the pixel array and one pixel in the image data is based on the position of the pixel array on the image sensor 13R. In FIG. 8, the luminance values of the third pixel from the left and the third pixel from the top among all the pixels of the image data 124L for the left eye are shown in the pixel array located in the third column from the left and the third row from the top. Calculation is performed from the output of the photoelectric conversion element a1.

According to the stereo imaging device according to the first embodiment described above, the following operational effects can be obtained.
(1) The control circuit 101 creates a pair of pan focus image data having parallax based on the pan focus image data output from the pan focus processing circuits 18L and 18R. Thereby, as compared with stereo image data obtained by normal imaging, stereo image data in a desirable state as a stereo image can be obtained in which all subjects existing in a wide range of distance are clearly captured.

(2) When the distance to the subject detected by the distance sensor 104 is less than a predetermined value, the control circuit 101 generates a pair of image data having parallax based only on the image data output from the imaging unit 100L. create. As a result, a sufficient three-dimensional effect can be obtained from a subject located at a close distance.

(3) The pan focus processing circuits 18L and 18R create pan focus image data based on image data focused on a large number of image planes whose positions are gradually shifted. Thereby, pan-focus image data with less noise can be obtained as compared with the method of reducing the aperture diameter of the lens.

  In the above description, a stereo image composed of a right-eye image and a left-eye image has been described. Such a stereo image can be viewed by various known methods. On the other hand, as a lenticular three-dimensional image, a method of obtaining a three-dimensional effect using not only two left and right viewpoint images but also three or more viewpoint images is known. In the second embodiment described in detail below, a multi-viewpoint image for such a method can be created.

-Second embodiment-
FIG. 9 is a diagram illustrating processing for obtaining a lenticular stereoscopic image using two imaging units. As shown in FIG. 9, by changing the relative position of the photoelectric conversion element that extracts the output from each pixel array 130, the same image data 221R to 230L as when the subject is imaged from different viewpoints can be obtained. In FIG. 9, image data corresponding to five different viewpoints is acquired for each imaging unit by this method. This corresponds to a case where 10 cameras are arranged side by side to image a subject. A lenticular stereoscopic image can be created by arranging the ten pieces of image data 221R to 230L thus obtained in a vertical stripe shape by a known method.

  Note that the parallax of the ten pieces of image data 221R to 230L obtained by the above processing is not constant. Specifically, only the parallax between the image data 225R and the image data 226L is larger than the parallax between the other image data. However, if there is a viewer of a lenticular stereoscopic image at a position where any one of the image data 226L to 230L can be viewed with the left eye and any one of the image data 221R to 225R with the right eye, it is the same as a normal lenticular stereoscopic image. The three-dimensional effect can be obtained.

  FIG. 10 is a diagram illustrating processing for obtaining a lenticular stereoscopic image using one imaging unit. Similarly to the case of using the two imaging units shown in FIG. 9, image data of different viewpoints can be obtained by changing the relative position of the photoelectric conversion element used for creating the image data. In FIG. 10, image data 221R to 225R corresponding to five different viewpoints are acquired. As in the first embodiment, a lenticular stereoscopic image created using two imaging units is larger than a lenticular stereoscopic image created using only one imaging unit. A feeling is obtained. This is because the latter has a larger parallax of images captured by the left and right eyes.

According to the imaging apparatus according to the second embodiment described above, the following operational effects can be obtained.
(1) The imaging apparatus uses two imaging units to create image data corresponding to three or more different viewpoints. Thereby, it is not necessary to prepare a large number of image pickup units, and an increase in size of the image pickup apparatus can be avoided.

--Third embodiment--
FIG. 11 is a diagram illustrating a configuration of a stereo imaging device according to the third embodiment. The stereo imaging device 2 according to this embodiment includes imaging units 100L and 200R, a control circuit 201, a memory 102, an input device 103, a distance sensor 104, and a display device 205. That is, the stereo imaging device 2 is newly provided with a display device 205 and an imaging unit 200R in place of the imaging unit 100R in the first embodiment. The display device 205 is a scan backlight type display device and has a function of displaying an image in a planar manner based on one image data and a function of displaying a stereoscopically viewable image based on stereo image data. The configuration of the imaging unit 200R will be described in detail later.

  FIG. 12 is a diagram illustrating the configuration of the imaging units 100L and 200R. Since the configuration of the imaging unit 100L is the same as that of the first embodiment, the description thereof is omitted. Unlike the imaging unit 100L and the imaging unit 100R, the imaging unit 200R does not have a function of synthesizing a pan-focus image. That is, the imaging unit 200R is not a composite imaging unit. Specifically, the microlens array is not disposed in the vicinity of the focal plane of the photographing lens 11R, and only the image sensor 213R is disposed. In addition, you may arrange | position the micro lens for raising the condensing rate to the said photoelectric conversion element with respect to each photoelectric conversion element. This microlens is for increasing the light collection rate, and is not used for the purpose of synthesizing a pan focus image.

  The image sensor 213R is driven by the drive circuit 14R in the same manner as the image sensor 13L of the image pickup unit 100L. The image signal output from the image sensor 213R is taken into the image processing circuit 215R via the drive circuit 14R or output to the output circuit 19R. The image processing circuit 215R creates image data by applying known image processing to the image signal. The image processing circuit 215R stores this image data in the memory 16R.

  The number of pixels of pan focus image data created by the imaging unit 100L is smaller than the number of pixels of image data created by the imaging unit 200R. This is because the imaging unit 100L outputs one pixel for one microlens ML. That is, the number of pixels of pan focus image data is limited by the number of microlenses ML included in the microlens array 12L.

  In the present embodiment, the number of pixels that can be displayed on the display screen of the display device 205 is larger than the number of pixels of pan-focus image data and smaller than the total number of images of image data created by the imaging unit 200R.

  The control circuit 201 (FIG. 11) creates stereo image data based on the pan focus image data created by the imaging unit 100L and the image data created by the imaging unit 200R. The control circuit 201 is configured to be able to create stereo image data having an arbitrary number of pixels. However, the number of pixels of image data created by the imaging unit 200R is the upper limit of the number of pixels of stereo image data. Hereinafter, the number of pixels of pan focus image data created by the imaging unit 100L is called the number of left pixels, the number of pixels of image data created by the imaging unit 200R is called the number of right pixels, and the number of pixels of stereo image data is called the target number of pixels.

  As described above, the number of pixels is different between the pan focus image data created by the imaging unit 100L and the image data created by the imaging unit 200R. That is, the left pixel number and the right pixel number are different. Therefore, the control circuit 201 executes a predetermined enlargement process or reduction process on these two image data, thereby matching the number of pixels of the two image data with the target number of pixels to obtain stereo image data. Note that the reduction process executed by the control circuit 201 is a known image reduction process, and thus description thereof is omitted.

  As described above, the target pixel number does not exceed the right pixel number. Accordingly, the enlargement process is not executed on the image data created by the imaging unit 200R. On the other hand, enlargement processing may be performed on pan-focus image data created by the imaging unit 100L. When executing the enlargement process, the control circuit 201 performs a blurring process on the image data after executing a known image enlargement process. This is because a simple enlargement results in a rough mosaic image.

  The control circuit 201 stores the created stereo image data in the memory 102. Thereafter, the control circuit 201 stores the stereo image data in a storage medium (not shown) or displays the stereo image data on the display device 205, for example. When storing stereo image data in a storage medium (not shown), the user can designate a target resolution to the control circuit 101 via the input device 103.

  When displaying stereo image data, the control circuit 201 creates stereo image data with the number of pixels that can be displayed on the display device 205 as the target number of pixels. When the stereo image data is displayed, the user can transmit an instruction to enlarge the stereo image data to the control circuit 201 via the input device 103. Upon receiving the instruction, the control circuit 201 increases the target number of pixels in accordance with the enlargement instruction and cuts out a region for the number of pixels that can be displayed from the created stereo image data to obtain new stereo image data. The new stereo image data is displayed on the display device 205.

  As described above, when the enlargement process or the blurring process is applied to the pan focus image data, the pan focus image data becomes a slightly unclear image as compared to before applying these processes. However, even if one of the image data constituting the stereo image data is an unclear image, if the other image data is clear, the stereo image data can be fully appreciated. Needless to say, a clearer image can be created by interpolating one of the unclear image data based on the other clear image data.

  Next, the convergence angle adjustment function will be described. The control circuit 201 of this embodiment has a selection function for selecting the exit pupil area of the photographic lens 11L. When imaging using the stereo imaging device 2, the user can designate the exit pupil region of the photographic lens 11 </ b> L with the input device 103. When this designation is made, the control circuit 201 selects the area designated by the above selection function. Then, the image pickup unit 100L is caused to create image data synthesized using only the light flux from the region selected by the selection function, not the pan focus image data.

  FIG. 13 is a diagram for explaining the outline of the convergence angle adjustment function. FIG. 13 shows a state in which the imaging units 100L and 200R are viewed from above, as in FIG. When the convergence angle adjustment function is not used, the base line length Ls of the stereo image data created by the control circuit 201 is the distance between the optical axis of the photographing lens 11R and the optical axis of the photographing lens 11L.

  On the other hand, when the selection function of the control circuit 201 selects the region 208 shown in FIG. 13 which is a region far from the photographic lens 11R from the entrance pupil of the photographic lens 11L, the imaging unit 100L moves from this region 208 to the imaging element 13L. Image data is synthesized using only incident light beams. Thereby, the base line length of the stereo image data created by the control circuit 201 becomes the base line length L longer than Ls. That is, by using this selection function, the base line length of the created stereo image data can be made longer than usual. Conversely, if the region 209 is selected, the baseline length can be shortened.

  FIG. 14 is an enlarged view of the microlens array 12L and the image sensor 13L in FIG. When combining image data using only the light flux from the region 208 shown in FIG. 13, the arithmetic processing circuit 15L combines the image data using only the output of the pixel closer to the photographic lens 11R in each pixel array 130. . That is, the image data is synthesized using only the outputs of the pixels e1, e2, e3, e4,.

  FIG. 15 is a diagram illustrating changes in the convergence angle. When stereo image data is captured on the subject 210 without using the convergence angle adjustment function, the convergence angle with respect to the subject 210 is an angle θs shown in FIG. On the other hand, when the image data is synthesized using only the light flux from the region 208 shown in FIG. 13, the convergence angle with respect to the subject 210 is the angle θ shown in FIG. Thus, by selecting the exit pupil area of the photographic lens 11L used for the synthesis of the image data, the parallax for the same subject in the image data created by the imaging unit 100L is different from the normal parallax. As a result, the stereo image data created by the control circuit 201 has a convergence angle different from the normal convergence angle for the same subject.

According to the imaging apparatus according to the third embodiment described above, the following operational effects can be obtained.
(1) The imaging unit 200R includes an imaging lens 11R, an imaging element 213R including a plurality of pixels having a photoelectric conversion function arranged two-dimensionally in the vicinity of the imaging surface of the imaging lens 11R, and an output of the imaging element 213R. And an image processing circuit 215R that creates image data having a larger number of pixels than the imaging unit 100L. The control circuit 201 performs a predetermined enlargement process on the pan focus image data created by the imaging unit 100L so that the number of pixels is the same as the image data created by the imaging unit 200R. Thereafter, predetermined blurring processing is performed to create stereo image data. Since it did in this way, stereo image data with more pixels than the image data which the imaging unit 100L produces can be obtained.

(2) The control circuit 201 has a selection function for selecting the exit pupil area of the photographic lens 11L. The imaging unit 100L is an image in which the parallax with respect to the same subject is different from that of the pan focus image data, instead of the pan focus image data, based on the outputs of the plurality of pixels that receive the light flux from the exit pupil area selected by the selection function. Create data. Since it did in this way, even if it is a case where the space | interval of imaging unit 100L, 200R is made into the length different from a normal stereo imaging device, the parallax equivalent to a normal stereo imaging device can be obtained.

The following modifications are also within the scope of the present invention, and one or a plurality of modifications can be combined with the above-described embodiment.
(1) In the first embodiment, pan focus image data may be created based on the result of focus detection. That is, instead of using image data of a large number of image planes whose positions are gradually shifted, focus detection is performed by a known method, and only the image data of the image plane where it is detected that a subject image is formed is used. Also good.

(2) In the first embodiment, if the subject is sufficiently bright, the pan-focus image data is obtained by collecting only the outputs of one photoelectric conversion element at the same relative position from each pixel array 130 and collecting them. You may create it.

(3) The pixel value of the image data shown in FIG. 8 may be calculated by a calculation other than adding the output of the photoelectric conversion element. For example, a value obtained by averaging the outputs of the photoelectric conversion elements may be used as the pixel value of the image data. Further, the selection pattern of the photoelectric conversion elements may be different from the pattern shown by hatching in FIG.

(4) The optical characteristics of the two imaging units may not be completely the same. In addition, one of the two imaging units can create image data in which all subjects from close range to infinity are in focus with one shooting with the state of the shooting optical system fixed. It may not be an imaging unit.

(5) The exit pupil plane of the photographic lens and the image sensor may be conjugate or there may be a shift.

(6) Three or more imaging units may be prepared, and stereo image data may be created from any two of them. Moreover, any of these imaging units may be used as the imaging unit used when imaging a subject existing at a close distance. For example, only one predetermined imaging unit may be used, or an optimal imaging unit may be selected according to the position of the subject.

(7) As the distance sensor, distance detection devices of various principles that have been used in conventional cameras and the like can be used. In addition, focus detection is performed by the output of either the imaging unit 100L or 100R using a technique known in Japanese Patent Application Laid-Open No. 2007-316521 and the distance of the subject from the obtained imaging position and the focus lens position of the photographing lens. May be calculated. In this way, although the detection means for the focus lens position of the photographic lens is required, it is not necessary to provide any other mechanism for detecting the distance of the subject, and the overall structure is simplified and the cost can be reduced.

(8) In the stereo imaging device 1 according to the first embodiment, it is not always necessary to create pan-focus image data. That is, using either one of the imaging units 100R and 100L at a short distance and both at a long distance, the pan focus image data and / or one of the image data focused on a desired position is created. Can do. As a stereo imaging device, stereo image data can be output based on one of the image data.

(9) The convergence angle adjustment function described in the third embodiment can also be applied to the case where the two imaging units are both pan-focus imaging units as in the first embodiment. . In this case, since the convergence angle can be adjusted in both of the two imaging units, the range of adjustment is wider than that of the stereo imaging device 2 shown in the third embodiment.

(10) In the third embodiment, the number of pixels of the stereo image data created by the control circuit 201 may be automatically changed based on the detected various information. For example, in the case of moving image shooting, the number of pixels may be made smaller than in the case of still image shooting, or the number of pixels may be reduced when the free capacity of the storage medium becomes a predetermined amount or less. .

  As long as the characteristics of the present invention are not impaired, the present invention is not limited to the above-described embodiments, and other forms conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention. .

1, 2 Stereo imaging devices 18L, 18R Pan focus processing circuits 100L, 100R, 200R Imaging unit 101 Control circuit 130 Pixel array ML Micro lens

Claims (6)

At least two imaging units for outputting an image signal of a subject;
First image creating means for creating a pair of images having parallax based on image signals output by the two imaging units;
A stereo imaging device comprising:
At least one of the two imaging units includes an imaging optical system, a plurality of positive lenses arranged two-dimensionally in the vicinity of a predetermined imaging surface of the imaging optical system, and the plurality of positive lenses. An image sensor that includes a plurality of pixels that are arranged two-dimensionally on the rear side of each of the imaging optical systems and that receives light beams from different regions of the exit pupil of the imaging optical system, and an output of the image sensor A stereo image pickup apparatus, comprising: a pan-focus image pickup unit having an output unit capable of outputting a pan-focus image signal. The stereo imaging device according to claim 1,
The two image pickup units are both pan focus image pickup units, and a stereo image pickup apparatus is characterized. The stereo imaging device according to claim 1,
Ranging means for detecting the distance to the subject;
Second image creating means for creating a pair of images having parallax based on the image signal output by the pan focus imaging unit;
Further comprising
When the distance detected by the distance measuring means is greater than or equal to a predetermined value, a pair of images having parallax is created by the first image creating means, and when the distance detected by the distance measuring means is less than a predetermined value A stereo imaging apparatus, wherein a pair of images having parallax is created by the second image creating means. The stereo imaging device according to claim 2,
Distance measuring means for detecting a distance between a subject and the stereo imaging device;
Third image creating means for creating a pair of images having parallax based on an image signal output by either one of the two image synthesizing imaging units;
Further comprising
A stereo which operates by switching between the first image creating means and the third image creating means depending on whether the distance detected by the distance measuring means is greater than a predetermined value or less than a predetermined value. Imaging device. The stereo imaging device according to claim 1 or 3,
One of the two imaging units is the pan-focus imaging unit, and the other is a second imaging optical system and a photoelectric element arranged two-dimensionally in the vicinity of a predetermined imaging surface of the second imaging optical system. A second imaging device including a plurality of pixels having a conversion function; and second output means for outputting an image signal having a larger number of pixels than an image signal output from the pan-focus imaging unit based on an output of the second imaging device. A standard imaging unit having
Enlarging means that performs a predetermined enlargement process on the image signal output from the pan focus imaging unit, and makes the number of pixels the same as the image signal output from the standard imaging unit;
A blurring unit that performs a predetermined blurring process on the image signal that has been subjected to the magnification process by the magnification unit;
The first image creating unit is a pair of images having parallax based on the image signal subjected to the blurring process by the blurring unit and the image signal output by the standard imaging unit, and the pan-focus imaging A stereo imaging device that creates a pair of images having a larger number of pixels than an image signal output by a unit. At least two imaging units for outputting an image signal of a subject;
Ranging means for detecting the distance to the subject;
First image creating means for creating a pair of images having parallax based on image signals output by the two imaging units;
Second image creation means for creating a pair of images having parallax based on an image signal output by one of the two imaging units;
A stereo imaging device comprising:
At least one of the two imaging units includes an imaging optical system, a plurality of positive lenses arranged two-dimensionally in the vicinity of a predetermined imaging surface of the imaging optical system, and the plurality of positive lenses. Each pixel includes a plurality of pixels arranged two-dimensionally on the rear side and having a photoelectric conversion function of receiving light beams from different regions of the exit pupil of the imaging optical system, and outputs the plurality of pixels as image signals. An image pickup unit having an image pickup device that outputs as:
The second image creation means creates a pair of images having parallax based on the image signal output by the composite imaging unit,
When the distance detected by the distance measuring means is greater than or equal to a predetermined value, a pair of images having parallax is created by the first image creating means, and when the distance detected by the distance measuring means is less than a predetermined value A stereo imaging apparatus, wherein a pair of images having parallax is created by the second image creating means.

Images