JP2013211790A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To adjust differences in sensitivity between plural types of pixels.SOLUTION: On the imaging face 23a of a color imaging device 23, a first group of pixels R, G, Bwith high sensitivity to a subject light 50L and a second group of pixels R, G, Bwith high sensitivity to a subject light 50R are arrayed two-dimensionally. On the light paths of the subject lights 50L and 50R which enter the imaging face 23a through an imaging optical system 17, a liquid crystal shutter 22 is disposed. The light quantities of the subject lights 50L and 50R are individually adjusted by adjusting the light transmissivity of the left and right light transmission areas 52L and 52R of the liquid crystal shutter 22, which adjusts differences in sensitivity between the first group of pixels R, G, Band the second group of pixels R, G, B.

Description

  The present invention relates to an imaging apparatus that obtains a plurality of images with parallax by imaging the subject light that has passed through a photographing optical system by dividing the pupil light.

  2. Description of the Related Art Digital cameras that are equipped with color image sensors such as CCD image sensors and CMOS image sensors and acquire digital images have become widespread. Particularly in recent years, a monocular 3D digital camera that obtains a stereoscopically viewable parallax image composed of multiple viewpoint images with parallax by imaging subject light that has passed through a photographing optical system by dividing the pupil is known. Yes.

  A color image pickup device of such a monocular 3D digital camera transmits subject light to an image pickup surface on which a first group of RGB pixels and a second group of RGB pixels are arranged, and a photoelectric conversion element (PD) of each RGB pixel. And a condensing microlens. The first group of RGB pixels is arranged in a predetermined pattern on the imaging surface. The second group of RGB pixels has the same arrangement pattern as the first group of RGB pixels, and is disposed on the imaging surface so as to be adjacent to each of the first group of RGB pixels (see Patent Documents 1 and 2). .

  The first group of RGB pixels receives light incident through the first eccentric opening that is eccentric in the first direction with respect to the center position of the light receiving surface of the PD. The second group of RGB pixels receives light incident through a second eccentric opening that is decentered in a second direction opposite to the first direction with respect to the center position of the light receiving surface of the PD. Therefore, the PD of the first group of RGB pixels and the PD of the second group of RGB pixels selectively receive subject light that has passed through different regions in the left-right direction of the photographing optical system. Accordingly, two viewpoints configured by an L viewpoint image configured by output signals output from the first group of RGB pixels and an R viewpoint image configured by output signals output from the second group of RGB pixels. The parallax image can be acquired.

  In general, in order to obtain a parallax image, it is necessary to prepare two sets of photographing lenses and color image pickup elements and arrange them in parallel to perform photographing. On the other hand, in the digital camera provided with the color image sensor described in Patent Documents 1 and 2, a pair of photographing lens and color image sensor may be used, so that a parallax image can be acquired with a simple configuration.

JP 2011-203331 A JP 2011-223562 A

  However, in the color imaging elements described in Patent Documents 1 and 2, the center (optical axis) position of each microlens is slightly shifted from the center position of the PD of the RGB pixels of the first group and the second group. (See FIG. 10). Thus, when the microlens deviates from the predetermined design position, the amount of subject light received by the first group of RGB pixels is different from the amount of subject light received by the second group of RGB pixels. A sensitivity difference occurs between the first group of RGB pixels and the second group of RGB pixels. As a result, the image quality of the parallax image is degraded.

  The objective of this invention is providing the imaging device which can adjust the sensitivity difference of the multiple types of pixel contained in an image pick-up element.

  An imaging apparatus for achieving the object of the present invention is a plurality of types of pixels including a photographing optical system and photoelectric conversion elements, and subject light that has passed through a plurality of different areas of the photographing optical system is divided by pupils. An image sensor in which a plurality of types of pixels that selectively receive light are two-dimensionally arranged, and the amount of light of each subject light is individually adjusted according to the difference in sensitivity between the plurality of types of pixels, and the sensitivity difference between the plurality of types of pixels And a light amount adjusting means for adjusting the light amount by the light amount.

  According to the present invention, the sensitivity difference between a plurality of types of pixels can be adjusted by the amount of light by individually adjusting the amount of light of subject light that has passed through a plurality of different regions of the photographing optical system.

  Preferably, the light amount adjusting means has an opening through which each subject light passes, and the light amount of each subject light can be individually adjusted for each of a part of the regions in the opening. It is possible to adjust the sensitivity difference between plural types of pixels.

  The light amount adjusting means is provided in the opening and has a light-shielding portion having a plurality of small openings through which a part of each subject light passes, and a subject that is individually provided in each small opening and passes through each small opening. It is preferable to have an individual light amount adjustment unit that adjusts the amount of light. As a result, the separability of each subject light that has passed through a plurality of different areas of the photographing optical system is improved, so that good parallax image data with more enhanced parallax can be obtained.

  The light amount adjusting means is provided in the opening and is capable of adjusting the light transmittance at each position where the subject light is transmitted, and the light transmittance is controlled from the center side to the outside by controlling the light transmitting portion. The first light transmission control means is a first light transmission control unit that changes in stages to form a first light transmission region in a multiple ring zone and a second light transmission region located inside the first light transmission region. In addition, it is preferable to include first light transmission unit control means for performing individual adjustment by decentering the position of the second light transmission region with respect to the center of the light transmission unit. Thereby, the sensitivity difference of multiple types of pixels can be adjusted. In addition, it is possible to adjust the light amount of the direct incident light and the oblique incident light, to uniformly inject the subject light on the entire imaging surface, and to enhance the parallax.

  The first light transmission part control means preferably increases the light transmittance of the light transmission part stepwise from the center side toward the outside. As a result, shading correction can be performed.

  The light amount adjusting means is provided in the opening and is capable of adjusting the light transmittance for each position where the subject light is transmitted, and the light transmittance of each light transmitting region of the light transmitting portion through which each subject light is transmitted. A second light transmission part control means for individually controlling the light transmission part, wherein individual adjustment is performed by individually changing the light transmittance of each light transmission region stepwise from the center side to the outside of the light transmission part. And a two-light transmission part control means. Thereby, the sensitivity difference of multiple types of pixels can be adjusted. In addition, it is possible to adjust the light amount of the direct incident light and the oblique incident light, to uniformly inject the subject light on the entire imaging surface, and to enhance the parallax.

  It is preferable that the second transmission part control means has a different number of steps when changing the light transmittance stepwise for each light transmission region. This makes it possible to more accurately control the amount of each subject light that has passed through a plurality of different areas of the photographing optical system.

  It is preferable that the second light transmission part control means increases the light transmittance of each light transmission region in a stepwise manner from the center side toward the outside. As a result, shading correction can be performed.

  The plurality of different regions of the photographic optical system are two regions that are symmetrical with respect to the first straight line perpendicular to the optical axis of the photographic optical system, and the second light transmission unit control means passes through the center of the light transmission unit. In addition, it is preferable that the light transmittances of the first divided light transmission region and the second divided light transmission region obtained by dividing the light transmission portion along a second straight line parallel to the first straight line are changed stepwise. Thereby, parallax image data including viewpoint images of two favorable viewpoints (L and R viewpoints) can be obtained.

  The imaging element preferably has a microlens for condensing each subject light on the photoelectric conversion elements of a plurality of types of pixels.

  The image sensor is a two-dimensional array of pixel blocks composed of a plurality of types of pixels, each of which is a microlens provided on each pixel block, and receives each subject light that has passed through each small aperture. It is preferable to provide a microlens that collects light on a plurality of types of pixel photoelectric conversion elements. Thereby, favorable multi-viewpoint parallax image data with more enhanced parallax can be obtained.

The pixel block, N 2 ^(N is a natural number of 2 or more) types of pixels are arranged in a square matrix shape, the light shielding part, N ² pieces of small openings and the individual light quantity adjusting unit arranged in a square matrix It is preferable that Thereby, favorable multi-viewpoint parallax image data with more enhanced parallax can be obtained.

  It is preferable that the difference in sensitivity between the plurality of types of pixels is caused by the positional deviation of the microlens.

  The light amount adjusting means is preferably a liquid crystal shutter.

  It is preferable to include image generation means for generating a parallax image including a plurality of images respectively constituted by output signals of a plurality of types of pixels.

  The image pickup apparatus of the present invention can individually adjust the amount of light of subject light that has passed through a plurality of different regions of the photographing optical system to adjust the sensitivity difference between a plurality of types of pixels by the amount of light. A good parallax image can be obtained even when a positional shift or the like occurs.

It is a front perspective view of the digital camera of a 1st embodiment. It is a rear perspective view of a digital camera. It is a block diagram which shows the electric constitution of a digital camera. It is a front view of the imaging surface of a color image sensor. It is explanatory drawing for demonstrating the pixel arrangement | sequence of a color image sensor. It is sectional drawing which follows the X1-X1 'line | wire in FIG. It is sectional drawing which follows the X2-X2 'line | wire in FIG. 1 is a schematic diagram of a photographing optical system. It is explanatory drawing for demonstrating a state when the position shift of a micro lens has not generate | occur | produced. It is explanatory drawing for demonstrating a state when the position shift of the micro lens has generate | occur | produced. It is the schematic of a liquid-crystal shutter. It is a functional block diagram of CPU. It is explanatory drawing for demonstrating the measuring method of incident angle dependence information. It is a graph showing the measurement result of incident angle dependence information. 12 is a graph showing a change in light transmittance of the liquid crystal shutter along the line X3-X3 ′ in FIG. 11. It is a flowchart for demonstrating the effect | action (imaging process) of a digital camera. It is explanatory drawing for demonstrating light quantity adjustment by a liquid-crystal shutter. It is the schematic of the liquid-crystal shutter of the digital camera of 2nd Embodiment. 19 is a graph showing a change in light transmittance of the liquid crystal shutter along the line X4-X4 ′ in FIG. 18. It is explanatory drawing for demonstrating the effect of the liquid-crystal shutter of 2nd Embodiment. It is the schematic of the liquid-crystal shutter of the digital camera of 3rd Embodiment. It is the graph showing the change of the light transmittance of the liquid-crystal shutter in alignment with the X5-X5 'line | wire in FIG. It is explanatory drawing for demonstrating the light quantity adjustment of the to-be-photographed object light by a liquid-crystal shutter. 24 is a graph showing a change in light transmittance of the liquid crystal shutter along the line X5a-X5a ′ in FIG. 23. It is the schematic of the liquid-crystal shutter of the digital camera of 4th Embodiment. It is the graph showing the change of the light transmittance of the liquid-crystal shutter in alignment with the X6-X6 'line | wire in FIG. It is a front view of the image pick-up surface of the color image pick-up element of the digital camera of 5th Embodiment. It is the schematic of the liquid-crystal shutter of 5th Embodiment. It is a graph showing the change of the light transmittance of the liquid-crystal shutter along the X7-X7 'line | wire in FIG. It is a graph showing the change of the light transmittance of the liquid-crystal shutter along the X8-X8 'line | wire in FIG. It is a front view of the image pick-up surface of the color image pick-up element of the digital camera of 6th Embodiment. It is the schematic of the liquid-crystal shutter of 6th Embodiment. It is a perspective view of a smart phone. It is a block diagram which shows the electric constitution of a smart phone.

[Overall Configuration of Digital Camera of First Embodiment]
As shown in FIG. 1, on the front surface of a camera body 3 of a digital camera 2 corresponding to the imaging apparatus of the present invention, an imaging unit 4 including a lens barrel, an imaging optical system, and the like, a strobe light emitting unit 5 and the like Is provided. On the upper surface of the camera body 3, a shutter button 6, a power switch 7, and the like are provided.

  As shown in FIG. 2, a display unit 8 and an operation unit 9 are provided on the back surface of the camera body 3. The display unit 8 functions as an electronic viewfinder in a shooting standby state, and displays a live view image (also referred to as a through image). At the time of image reproduction, the image is reproduced and displayed on the display unit 8 based on the image data recorded on the memory card 10.

  The operation unit 9 includes a mode switch, a cross key, an execution key, and the like. The mode switch is operated when switching the operation mode of the digital camera 2. The digital camera 2 has a normal shooting mode for obtaining a normal shot image, a 3D shooting mode for obtaining a stereoscopically viewable parallax image, a playback mode for playing back and displaying each image obtained in each shooting mode, and the like.

  The cross key and the execution key are used to display a menu screen or a setting screen on the display unit 8, move a cursor displayed in the menu screen or the setting screen, or confirm various settings of the digital camera 2. It is manipulated.

  Although not shown, a card slot in which the memory card 10 is detachably loaded and a loading lid for opening and closing the opening of the card slot are provided on the bottom surface of the camera body 3.

  As shown in FIG. 3, the CPU 11 of the digital camera 2 executes various programs and data read from the memory 13 sequentially based on a control signal from the operation unit 9 to control each unit of the digital camera 2 in an integrated manner. . The RAM area of the memory 13 functions as a work memory for the CPU 11 to execute processing and a temporary storage destination for various data.

  The imaging unit 4 incorporates a photographing optical system 17 including a zoom lens 15 and a focus lens 16, a mechanical shutter 18, and the like. The zoom lens 15 and the focus lens 16 are driven by a zoom mechanism 19 and a focus mechanism 20, respectively, and are moved back and forth along the optical axis O1 of the photographing optical system 17.

  The mechanical shutter 18 has a movable portion (not shown) that moves between a closed position where the object light is prevented from entering the liquid crystal shutter 22 and the color image sensor 23 and an open position where the object light is allowed to enter. The mechanical shutter 18 opens / blocks the optical path from the photographing optical system 17 to the color image sensor 23 by moving the movable part to each position. In addition, the mechanical shutter 18 includes a diaphragm for controlling the amount of subject light incident on the liquid crystal shutter 22 and the color image sensor 23. The mechanical shutter 18, zoom mechanism 19, and focus mechanism 20 are controlled by the CPU 11 via a lens driver 25.

  A liquid crystal shutter (light amount adjusting means) 22 and a color image sensor 23 are disposed behind the image pickup unit 4. The liquid crystal shutter 22 individually adjusts the amount of subject light that has passed through different areas of the photographing optical system 17. The shutter driver 26 controls individual adjustment of the light amount by the liquid crystal shutter 22 under the control of the CPU 11.

  The color image sensor 23 converts the subject light that has passed through the photographing optical system 17 and the liquid crystal shutter 22 into an electrical output signal and outputs it. The color image sensor 23 may be any type of image sensor such as a CCD (Charge Coupled Device) color image sensor or a CMOS (Complementary Metal Oxide Semiconductor) color image sensor. The image sensor driver 27 controls driving of the color image sensor 23 under the control of the CPU 11.

  The signal adjustment circuit 28 performs various signal adjustment processes on the output signal output from the color image sensor 23. The signal adjustment circuit 28 includes a CDS / AGC circuit and an A / D conversion circuit when the color image sensor 23 is a CCD type, and an amplifier when the color image sensor 23 is a CMOS type. The The configuration of the signal adjustment circuit 28 may be changed as appropriate.

  The image processing circuit 29 performs various processing such as gradation conversion, white balance correction, and γ correction processing on the output signal input from the color image sensor 23 via the signal adjustment circuit 28 to generate image data. The image processing circuit 29 includes a normal image generation unit 29a and a parallax image generation unit (image generation unit) 29b. The normal image generation unit 29a operates in the normal shooting mode and generates normal shooting image data. The parallax image generation unit 29b operates in the 3D shooting mode to generate parallax image data.

  The compression / decompression processing circuit 31 performs compression processing on each image data processed by the image processing circuit 29. The compression / decompression processing circuit 31 performs decompression processing on the compressed image data obtained from the memory card 10 via the media I / F 32. The media I / F 32 performs recording and reading of each image data with respect to the memory card 10.

  The display unit 8 displays a through image, a reproduced image, and the like. As the display unit 8, various monitors capable of observing a stereoscopic image based on parallax image data are used. Note that various methods such as a lenticular method, a parallax barrier method, a parallax barrier method, an anaglyph method, a frame sequential method, and a light direction method may be used as a stereoscopic image display method.

  Although not shown, the digital camera 2 is provided with an autofocus AF detection circuit, an AE detection circuit, and the like. The CPU 11 executes the AF process by driving the focus mechanism 20 via the lens driver 25 based on the detection result of the AF detection circuit. Further, the CPU 11 executes the AE process by driving the mechanical shutter 18 via the lens driver 25 based on the detection result of the AE detection circuit.

[Configuration of color image sensor]
As shown in FIGS. 4 and 5, on the imaging surface 23a of the color imaging device 23, a first group of pixels R _A , G _A , B _A and a second group of pixels corresponding to a plurality of types of pixels of the present invention. Pixels R _B , G _B , and B _B are two-dimensionally arranged. The first group of pixels R _A , G _A and B _A includes a first photoelectric conversion element (hereinafter simply referred to as a first PD) 37A, and the second group of pixels R _B , G _B and B _B A photoelectric conversion element (hereinafter simply referred to as a second PD) 37B is included. The first PD 37A and the second PD 37B are basically the same.

The first group of pixels R _A , G _A , and B _A are arranged in a matrix at a pitch P in the horizontal and vertical directions on the odd-numbered pixel lines of the color image sensor 23. Specifically, the GR pixel row 38a formed by arranged at a pitch P alternately pixel G _A and the pixel R _A of the first group in the horizontal direction, the pixel B _A and the pixel G _A of the first group in the horizontal direction BG pixel rows 38b alternately arranged at a pitch P are arranged at a pitch P in the vertical direction.

The second group of pixels R _B , G _B , and B _B are arranged on the even pixel lines of the color image sensor 23 with the same arrangement pattern (pitch in the horizontal and vertical directions) as the first group of pixels R _A , G _A , and B _A. P is arranged in a matrix. Specifically, the GR pixel row 39a formed by arranged at a pitch P alternately pixel G _B and the pixel R _B of the second group in the horizontal direction, the pixel B _B and the pixel G _B of the second group in the horizontal direction BG pixel rows 39b alternately arranged at a pitch P are arranged at a pitch P in the vertical direction. However, the pixels of the pixel arrays 39a and 39b are arranged at positions shifted by 1 / (2P) in the horizontal direction with respect to the pixels of the pixel arrays 38a and 38b.

As described above, the imaging surface 23a is provided with the GR pixel array 38a, the GR pixel array 39a, the BG pixel array 38b, and the BG pixel array 39b alternately and repeatedly in the vertical direction. Incidentally, FIGS. 4 and 5, there is shown a pixel of the first group _{R A,} G A, and _{B A,} pixels _R B of the second _group, G B, an example of a pixel array with _{B B,} The arrangement of each pixel may be changed as appropriate. In the imaging surface 23a, an area in which the first group of pixels R _A , G _A , and B _A are arranged is referred to as an “A plane”, and the second group of pixels R _B , G _B , and B _B are arranged. The region that is made is also referred to as “B-plane”.

  6 showing a cross section of the GR pixel array 38a (X1-X1 ′ line in FIG. 5) and FIG. 7 showing a cross section of the GR pixel array 39a (X2-X2 ′ line in FIG. 5), The first PD 37A and the second PD 37B are formed on the surface layer of the sub) 42, respectively. Although not shown, the semiconductor substrate 42 is provided with various circuits used for driving each pixel and outputting signals.

  A light-transmissive insulating film 43 is provided on the semiconductor substrate 42. Here, “up” refers to the direction from the semiconductor substrate 42 toward the microlens 48 (upward in the figure). A light shielding film 44 is provided on the insulating film 43. The light shielding film 44 has a first eccentric opening 44a formed on the first PD 37A and a second eccentric opening 44b formed on the second PD 37B.

  The first eccentric opening 44a is formed at a position shifted in the right direction in the drawing with respect to the center of the first PD 37A. Thereby, the substantially left half area (hereinafter simply referred to as the left area) of the first PD 37A is covered with the light shielding film 44, and conversely, the substantially right half area (hereinafter simply referred to as the right area) is exposed. On the other hand, the second eccentric opening 44b is formed at a position shifted in the left direction in the figure with respect to the center of the second PD 37B. As a result, the right region of the second PD 37B is covered with the light shielding film 44, and conversely, the left region is exposed.

  On the light shielding film 44, a light-transmitting flattening layer 45 having a flat surface is provided. On the planarization layer 45, color filters 46 of R, G, and B colors are provided at positions corresponding to pixels of the colors R, G, and B, respectively.

  A micro lens 48 is provided on each color filter 46 and above the first and second PDs 37A and 37B. Various layers such as a light-transmitting flat layer may be provided between the color filter 46 and the microlens 48. The symbol “O2” is the optical axis of the microlens 48.

  The subject light 50L incident on the microlens 48 from the left oblique direction in the drawing is condensed on the right region of the first PD 37A through the first eccentric opening 44a by the microlens 48, but the second PD 37B covered with the light shielding film 44 is collected. It is not collected in the right area. Conversely, the subject light 50R incident on the microlens 48 from the right oblique direction in the drawing is condensed by the microlens 48 on the left region of the second PD 37 through the second eccentric opening 44b, but is covered with the light shielding film 44. The first PD 37 is not focused on the left region. Therefore, the light shielding film 44 functions as a pupil division unit that performs pupil division.

  As shown in FIG. 8, the subject lights 50L and 50R are symmetric with respect to the first straight line L1 perpendicular to the optical axis O1 of the photographing optical system 17 (the zoom lens 15 and the focus lens 16), that is, the photographing optical system. The subject light has passed through the left region 17L and the right region 17R which are different in the left-right direction. In FIG. 8, both lenses 15 and 16 are shown in an integrated manner in order to prevent the drawing from becoming complicated.

Returning to FIG. 6 and FIG. 7, subject light incident on the color image sensor 23 is pupil-divided by the light shielding film 44, so that the first PD 37 </ b> A (first group of pixels R _A , G _A , B _A ) The sensitivity is increased with respect to 50L, and the second PD 37B (second group of pixels R _B , G _B , B _B ) has higher sensitivity with respect to the subject light 50R.

The normal image generation unit 29a illustrated in FIG. 1 is output from, for example, the first group of pixels R _A , G _A , and B _A (or the second group of pixels R _B , G _B , and B _B are possible) in the normal shooting mode. Based on the output signal, normal captured image data is generated.

In addition, the parallax image generation unit 29b generates L viewpoint image data when the subject is viewed from the L viewpoint side based on output signals from the first group of pixels R _A , G _A , and B _A in the 3D shooting mode. R viewpoint image data when the subject is viewed from the R viewpoint side is generated based on output signals from the second group of pixels R _B , G _B , and B _B. Since an image based on the L viewpoint image data and an image based on the R viewpoint image data have parallax, stereoscopic viewing is possible.

  At this time, as shown in FIG. 9, the microlens 48 is formed such that its center position (optical axis O2) is positioned on the center of each PD 37A, 37B. However, as shown in FIG. 10, the center position (optical axis O2) of the microlens 48 is slightly shifted from the center of each PD 37A, 37B due to errors in the manufacturing process. . In addition, since each micro lens 48 is collectively formed using, for example, a photolithographic method or an etching method, all the micro lenses 48 are shifted in the same direction.

Since each micro lens 48 is displaced from a predetermined design position, there is a difference in the amount of subject light 50L and 50R incident on each PD 37A and 37B. As a result, the sensitivity of the first group of pixels R _A , G _A and B _A (hereinafter simply referred to as first group pixel sensitivity) and the sensitivity of the second group of pixels R _B , G _B and B _B (hereinafter simply referred to as “first group pixel sensitivity”). A second group pixel sensitivity). For this reason, the digital camera 2 uses the liquid crystal shutter 22 so that the sensitivity difference between the first group pixel sensitivity and the second group pixel sensitivity becomes 0 (including almost 0), and the left and right regions of the photographic optical system 17. The light amounts of the subject lights 50L and 50R transmitted through 17L and 17R are individually adjusted.

  The structure of the color image sensor 23 is not limited to the examples shown in FIGS. 6 and 7 and may be changed as appropriate. For example, the light shielding film 44 may be provided in the upper layer of the color filter 46. Moreover, although the 1st and 2nd eccentric opening 44a, 44b is formed in the triangle shape, opening shape is not specifically limited. Further, a pupil division unit other than the light shielding film 44 may be used.

[Configuration related to liquid crystal shutter and light amount adjustment thereof]
As shown in FIG. 11A, the liquid crystal shutter 22 includes a frame body 51 and a light transmission portion 22 a provided in the opening 51 a of the frame body 51. The light transmission part 22a is comprised with the various liquid crystal panel which has a light transmittance. Since the light transmission part 22a can adjust the light transmittance (transmitted light amount) for each liquid crystal element, the light transmittance can be adjusted for each position where the subject light is transmitted.

  For example, as shown in FIG. 11B, the transmittance of the left light transmission region 52L of the light transmission portion 22a through which the subject light 50L is transmitted is set to the transmission of the right light transmission region 52R of the light transmission portion 22a through which the subject light 50R is transmitted. By making the rate lower, the amount of the subject light 50L incident on the color image sensor 23 can be made lower than the amount of the subject light 50R. Conversely, by making the light transmittance of the right light transmission region 52R lower than the light transmittance of the left light transmission region 52L, the light amount of the subject light 50R incident on the color image sensor 23 is made larger than the light amount of the subject light 50L. Can be lowered. That is, it is possible to individually adjust the light amounts of the subject lights 50L and 50R for each part of the area of the opening 51a (the left and right light transmission areas 52L and 52R in the present embodiment). Such adjustment of the light transmittance of the liquid crystal shutter 22 is executed by the CPU 11.

  As illustrated in FIG. 12, the CPU 11 functions as the light amount control unit 54 by executing a program read from the memory 13. The light amount control unit 54 adjusts the light transmittance of the light transmission unit 22 a via the shutter driver 26 based on the light amount correction data 55 stored in the memory 13 in advance. Thereby, the liquid crystal shutter 22 functions as the light amount adjusting means of the present invention.

  The light amount correction data 55 is data for adjusting the light transmittance of the light transmitting portion 22a so that the sensitivity difference between the first group pixel sensitivity and the second group pixel sensitivity becomes 0 (including almost 0). The light amount correction data 55 is obtained by the external element inspection device 56 at the time of shipping inspection of the color image sensor 23, for example.

As shown in FIG. 13, while irradiating inspection light (such as telecentric light) from the inspection light source 57 toward the imaging surface 23a of the color image sensor 23, the incident angle of the inspection light (specifically, the incident angle in the left-right direction). The inspection light source 57 is moved so that changes from −θ to + θ, for example, from −30 ° to + 30 °. At the same time, the element inspection apparatus 56 measures the magnitudes of the output signals output from the first group of pixels R _A , G _A , B _A and the second group of pixels R _B , G _B , B _B. Thereby, the incident angle dependency information 58 (see FIG. 14) indicating the difference in sensitivity between the pixels of the present invention is obtained.

As shown in FIG. 14, the incident angle dependency information 58 indicates a sensitivity difference between the first group pixel sensitivity and the second group pixel sensitivity (hereinafter simply referred to as a pixel sensitivity difference) caused by a positional deviation of the microlens 48 or the like. It is information to represent. Waveform WA shows the measurement results of the output signals of the first group of pixels R _A , G _A and B _A , and waveform WB shows the measurement result of the output signals of the second group of pixels R _B , G _B and B _B Yes.

  When the position of the micro lens 48 is shifted, the peak heights of both waveforms WA and WB are shifted, and the intermediate position of the peaks of both waveforms WA and WB is shifted from 0 °. In the example shown in FIG. 14, since the peak height of the waveform WA is higher than the peak height of the waveform WB, it is determined that the first group pixel sensitivity is higher than the second group pixel sensitivity. . The pixel sensitivity difference can be obtained based on such incident angle dependency information 58.

  Based on the pixel sensitivity difference obtained from the incident angle dependency information 58, the element inspection device 56 can adjust the left and right light transmission regions 52L as the light amount correction data 55 so that the pixel sensitivity difference becomes 0 (including almost 0). , 52R. Specifically, the pixel sensitivity difference is adjusted by reducing the amount of subject light incident on the pixel with higher sensitivity than the amount of subject light incident on the pixel with lower sensitivity. The light transmittances of the left and right light transmission regions 52L and 52R are determined so that the amount of subject light incident on the higher pixel decreases. For example, various information (data table or arithmetic expression) indicating the relationship between the pixel sensitivity difference and the optimal light transmittance of the left and right light transmission regions 52L and 52R is obtained in advance through experiments, simulations, etc. The light transmittance of the right and left light transmission regions 52L and 52R can be determined. In this way, the light quantity correction data 55 is obtained.

  The light amount correction data 55 obtained by the element inspection device 56 is stored in the memory 13 of the digital camera 2 on which the color image sensor 23 is mounted. Instead of acquiring the light amount correction data 55 from the element inspection device 56, the incident angle dependency information 58 (pixel sensitivity difference) is acquired from the element inspection device 56, and the light amount control unit is based on the incident angle dependency information 58. 54 may obtain the light amount correction data 55.

  As shown in FIG. 15, the light amount control unit 54 individually adjusts the light amounts of the subject lights 50L and 50R by adjusting the light transmittances of the left and right light transmission regions 52L and 52R based on the light amount correction data 55, respectively. Here, the individual adjustment includes not adjusting the amount of subject light incident on the pixel having the lower sensitivity.

  For example, when the first group pixel sensitivity is higher than the second group pixel sensitivity, the light amount control unit 54 reduces the light transmittance of the left light transmission region 52 </ b> L according to the light amount correction data 55. FIG. 15 shows a change in the light transmittance of the liquid crystal shutter 22 along the line X3-X3 ′ shown in FIG. Conversely, when the second group pixel sensitivity is higher than the first group pixel sensitivity, the light transmittance of the right light transmission region 52 </ b> R is lowered according to the light amount correction data 55.

[Operation of Digital Camera of First Embodiment]
Next, the operation of the digital camera 2 configured as described above will be described using the flowchart shown in FIG. When the shooting mode is set to the normal shooting mode or the 3D shooting mode by the operation unit 9 (step S1), the light amount control unit 54 of the CPU 11 operates to read the light amount correction data 55 from the memory 13 (step S2).

  Next, the light amount control unit 54 individually adjusts the light transmittance of the left and right light transmission regions 52L and 52R of the liquid crystal shutter 22 (light transmission unit 22a) via the shutter driver 26 based on the light amount correction data 55 read from the memory 13. To do. For example, as shown in FIG. 14, when the first group pixel sensitivity is higher than the second group pixel sensitivity due to the positional deviation of the micro lens 48, the light transmittance of the left light transmission region 52L is corrected by the light amount. It is lowered by the amount determined by the data 55 (step S3).

In this way, by adjusting the light transmittance of the left light transmission region 52L to be low, the light amount of the subject light 50L incident on the first PD 37A of the first group of pixels R _A , G _A , and B _A can be adjusted to be low. . On the other hand, since the light transmittance of the right light transmitting region 52R is set higher than the left light transmitting region 52L of the light transmittance, the object entering the pixel _R B of the second _group, G B, to the 2PD37B of _{B B} The amount of light 50R is greater than the amount of subject light 50L incident on the first PD 37A.

  As shown in FIG. 17, since the first group pixel sensitivity is relatively lowered by making the light amount of the subject light 50L lower than the light amount of the subject light 50R, the pixel sensitivity difference is adjusted. When the second group pixel sensitivity is higher than the first group pixel sensitivity, the imaging plane 23a is set by making the light transmittance of the right light transmission region 52R lower than the light transmittance of the left light transmission region 52L. The amount of the subject light 50R incident on the light is made lower than the amount of the subject light 50L. Thereby, since the second group pixel sensitivity is relatively low, the pixel sensitivity difference is adjusted.

Returning to FIG. 16, the CPU 11 controls the operation of the mechanical shutter 18 through the lens driver 25 and drives the color image pickup device 23 through the image pickup device driver 27 simultaneously with or after the light amount adjustment by the liquid crystal shutter 22. (Step S4). The mechanical shutter 18 is opened and closed at a predetermined shutter speed, and signal charges are accumulated in each pixel of the color image sensor 23. Then, under the control of the image sensor driver 27, output signals are output from the first group of pixels R _A , G _A , B _A and the second group of pixels R _B , G _B , _BB , respectively.

<Normal shooting mode>
When the normal shooting mode is set as the shooting mode, the normal image generation unit 29a operates. The normal image generation unit 29a generates normal captured image data based on the output signals from the first group of pixels R _A , G _A , B _A and outputs this to the display unit 8 at a fixed timing. Thereby, a through image is displayed on the display unit 8. At the same time, shooting preparation processing such as AF processing and AE processing is performed simultaneously.

  When the shutter button 6 is pressed, normal captured image data for one frame is generated by the normal image generation unit 29a. The normal photographed image data is compressed by the compression / decompression processing circuit 31 and then recorded on the memory card 10 via the media I / F 32 (step S5).

<3D shooting mode>
When the 3D shooting mode is set as the shooting mode, the parallax image generation unit 29b is operated, and the first group of pixels R _A , G _A , B _A and the second group of pixels R _B , G _B , based on the output signals from the respective B _B, L viewpoint image data, and generates an R view image data. The parallax image generation unit 29b outputs at least one of the L and R viewpoint image data to the display unit 8 at a constant timing. Thereby, a through image is displayed on the display unit 8.

  When the shutter button 6 is pressed, L and R viewpoint image data for one frame is generated by the parallax image generation unit 29b. These L and R viewpoint image data are compressed as parallax image data by the compression / decompression processing circuit 31, and then recorded on the memory card 10 via the media I / F 32 (step S6). The parallax image data recorded in the memory card 10 is displayed on the display unit 8 so as to be stereoscopically viewable by switching the operation mode of the digital camera 2 to the playback mode. Hereinafter, when the shooting is continued, the above-described processing is repeatedly executed. At the time of moving image shooting, a moving image file composed of a plurality of frames of L and R viewpoint image data (referred to as L and R frame image data) is generated and recorded in the memory card 10.

<Effects of digital camera>
Even when the position of the micro lens 48 is misaligned, the pixel sensitivity difference can be adjusted by the individual adjustment of the light amounts of the subject lights 50L and 50R by the liquid crystal shutter 22, so that good parallax image data can be obtained.

  In the first embodiment, the digital camera that obtains the left and right viewpoint parallax image data has been described. However, when the parallax image of three viewpoints or more is obtained, the light transmission area of the liquid crystal shutter 22 is set to 3 correspondingly. The light transmittance of each region may be adjusted by dividing into two or more regions.

[Configuration of Digital Camera (Liquid Crystal Shutter) of Second Embodiment]
Next, a digital camera according to the second embodiment of the present invention will be described. The liquid crystal shutter 22 of the digital camera 2 of the first embodiment transmits subject light 50L and 50R over the entire surface of the left and right light transmission regions 52L and 52R. On the other hand, the liquid crystal shutter of the digital camera of the second embodiment transmits only a part of each of the subject lights 50L and 50R.

  The digital camera of the second embodiment has basically the same configuration as that of the first embodiment except that the liquid crystal shutter 60 is different from that of the digital camera 2 of the first embodiment. For this reason, the same functions and configurations as those of the first embodiment are denoted by the same reference numerals, and description thereof is omitted.

  As shown in FIG. 18, the liquid crystal shutter 60 is provided in the opening 51 a of the frame 51, and includes a light shielding unit 62 having small openings 61 </ b> L and 61 </ b> R that transmit part of subject light 50 </ b> L and 50 </ b> R, and a small opening. 61L and 61R are provided with individual light quantity adjustment units 63L and 63R, respectively. The liquid crystal shutter 60 has a so-called double pole shape.

  The small openings 61 </ b> L and 61 </ b> R are provided in the central part (substantially central part) of the left and right regions of the light shielding part 62. For this reason, in the subject light 50L and 50R incident on the liquid crystal shutter 60, light incident on portions other than the small openings 61L and 61R is blocked by the light blocking portion 62. Thereby, only a part of each of the subject lights 50L and 50R is transmitted.

  The individual light amount adjusting units 63L and 63R are configured by various liquid crystal panels having light transmittance like the light transmitting unit 22a of the first embodiment, and the light transmittance is adjusted by the light amount control unit 54, respectively.

  As shown in FIG. 19, by making the transmittance of the individual light amount adjustment unit 63R lower than the transmittance of the individual light amount adjustment unit 63L, the light amount of the subject light 50R can be made lower than the light amount of the subject light 50L. . Although not shown, the light amount of the subject light 50L is made lower than the light amount of the subject light 50R by making the light transmittance of the individual light amount adjustment unit 63L lower than the light transmittance of the individual light amount adjustment unit 63R. Can do. That is, it is possible to individually adjust the light amounts of the subject lights 50L and 50R for each part of the regions in the opening 51a (individual light amount adjusting units 63L and 63R). Thereby, as in the first embodiment, the pixel sensitivity difference can be adjusted by individually adjusting the light amounts of the subject lights 50L and 50R, so that favorable parallax image data can be obtained.

  Further, in the liquid crystal shutter 60 having a double pole shape, the subject light that has entered other than the small openings 61L and 61R is shielded, that is, the subject light that has entered the central portion in the left-right direction of the liquid crystal shutter 60 shown in FIG. Is shielded from light. As a result, as shown in FIG. 20, the separability of the subject lights 50L and 50R is improved, so that parallax image data with more enhanced parallax can be obtained.

  Instead of providing the individual light amount adjustment units 63L and 63R in the small openings 61L and 61R, the light amounts of the subject lights 50L and 50R may be individually adjusted by adjusting the opening diameters of the small openings 61L and 61R. (The same applies to fifth and sixth embodiments described later).

[Configuration of Digital Camera (Liquid Crystal Shutter) of Third Embodiment]
Next, a digital camera according to a third embodiment of the present invention will be described. In the digital camera 2 of the first embodiment, the liquid crystal shutter 22 is formed with semicircular left and right light transmission regions 52L and 52R. However, in the digital camera of the third embodiment, the liquid crystal shutter 22 has a multi-annular shape. A light transmission region is formed.

  The digital camera of the third embodiment has basically the same configuration as that of the first embodiment except that the shape of the light transmission region formed on the liquid crystal shutter 22 is different. For this reason, the same functions and configurations as those of the first embodiment are denoted by the same reference numerals, and description thereof is omitted.

  In FIG. 21 and FIG. 22 showing the change in the light transmittance of the light transmission part 22a along the X5-X5 ′ direction in this figure, the digital camera of the third embodiment is a light amount adjustment part ( First light transmitting portion control means) 66 is provided. The light quantity control unit 66 adjusts the light transmittance of the light transmission unit 22 a via the shutter driver 26.

  The light quantity control unit 66 of the third embodiment controls the light transmission part 22a to increase the light transmittance stepwise from the center side toward the outside. As a result, the light transmitting portion 22a includes a multiple annular light transmitting region (hereinafter referred to as an annular light transmitting region) 68, and a disk-shaped light transmitting region (hereinafter referred to as an annular light transmitting region 68). 69 (referred to as a central light transmission region). The annular light transmission region 68 corresponds to the first light transmission region of the present invention, and the central light transmission region 69 corresponds to the second light transmission region of the present invention.

  By forming a multi-annular ring-shaped light transmission region 68 in the light transmission part 22a, the amount of vertically incident light that is perpendicularly incident on the imaging surface 23a of the image sensor 23 and the oblique incidence that is obliquely incident on the imaging surface 23a. The amount of light can be adjusted. Thereby, subject light can be uniformly incident on the entire imaging surface 23a. Moreover, parallax image data with more enhanced parallax can be obtained by adjusting the amount of subject light incident on the inner side (center side) and the outer side of the imaging surface 23a. Furthermore, the shading correction can be performed by increasing the light transmittance outside the light transmitting portion 22a and decreasing the light transmittance inside.

  Note that the number of the ring-shaped light transmission regions 68 and the width of each region (ring zone width) are not particularly limited, and may be determined as appropriate through, for example, experiments or simulations.

  When the position of the micro lens 48 is displaced, the light amount control unit 66 in FIG. 23 and FIG. 24 showing the change in the light transmittance of the light transmission unit 22a along the X5a-X5a ′ direction in FIG. The center position of the central light transmission region 69 is decentered in the left-right direction with respect to the center of the light transmission part 22a. As a result, the light amounts of the subject light 50L and 50R can be individually adjusted for each part of the area in the opening 51a.

  For example, when the second group pixel sensitivity is higher than the first group pixel sensitivity, the center position of the central light transmission region 69 is decentered in the right direction. Thereby, since the light quantity of the subject light 50R can be reduced, the second group pixel sensitivity can be lowered. Conversely, when the first group pixel sensitivity is higher than the second group pixel sensitivity, the center position of the central light transmission region 69 is decentered leftward. Thereby, since the light quantity of the subject light 50L can be reduced, the first group pixel sensitivity can be lowered. Note that the central light transmission region 69 may be decentered in the upper right direction, the lower right direction, the upper left direction, and the lower left direction in FIG.

  The decentering direction and the decentering amount for decentering the central light transmission region 69 are determined based on the light amount correction data 70 shown in FIG. The light amount correction data 70 is obtained by the element inspection device 56 in the same manner as the light amount correction data 55 of the first embodiment.

  Based on the incident angle dependency information 58 shown in FIG. 14, the above-described element inspection device 56 determines the eccentric direction and the amount of eccentricity of the central light transmission region 69 such that the difference in pixel sensitivity is 0 (including almost 0). The determination result is stored in the memory 13 as the light amount correction data 70. For example, various information (data table and arithmetic expression) indicating the relationship between the pixel sensitivity difference obtained from the incident angle dependency information 58 and the optimum eccentric direction and the amount of eccentricity of the central light transmission region 69 are obtained in advance. The eccentric direction and the amount of eccentricity of the central light transmission region 69 can be determined. At this time, the number, size (ring zone width), and light transmittance of each region of the annular light transmitting region 68 and the size and light transmittance of the central light transmitting region 69 may be determined at the same time. Good.

  The light amount control unit 66 individually adjusts the light amounts of the subject lights 50L and 50R by decentering the central light transmission region 69 according to the eccentric direction and the eccentric amount determined by the light amount correction data 70 stored in the memory 13. Thereby, as in the first and second embodiments, the difference in pixel sensitivity can be adjusted, so that good parallax image data can be obtained. Furthermore, the parallax can be further enhanced by forming a light transmission region having a multi-annular zone shape on the liquid crystal shutter 22.

[Configuration of Digital Camera (Liquid Crystal Shutter) of Fourth Embodiment]
Next, a digital camera according to a fourth embodiment of the present invention will be described. In the digital camera of the third embodiment, the central light transmission region 69 is decentered in order to adjust the pixel sensitivity difference. However, in the digital camera of the fourth embodiment, the light transmission part 22a is divided into left and right regions. A half-ring-shaped light transmission region is formed.

  The digital camera of the fourth embodiment has basically the same configuration as that of the first embodiment, as in the third embodiment. For this reason, the same functions and configurations as those of the first embodiment are denoted by the same reference numerals, and description thereof is omitted.

  In FIG. 25 and FIG. 26 showing the change of the light transmittance of the light transmitting portion 22a along the X6-X6 ′ direction in this figure, the digital camera of the fourth embodiment is different from the light intensity adjusting portion ( Second light transmission part control means) 73. The light quantity control unit 73 adjusts the light transmittance of the light transmission unit 22 a via the shutter driver 26.

  The light quantity control unit 73 controls the light transmittance for each of the left and right light transmission regions 52L and 52R of the light transmission unit 22a. Here, the left and right light transmission regions 52L and 52R are divided regions obtained by dividing the light transmission portion 22a along a second straight line L2 that passes through the center of the light transmission portion 22a and is parallel to the first straight line L1 shown in FIG. These correspond to the first divided light transmission region and the second divided light transmission region of the present invention.

  The light quantity control unit 73 increases the light transmittance of the left and right light transmission regions 52L and 52R stepwise from each other toward the outside from the center side of the light transmission unit 22a (the number of steps is not particularly limited). Thus, the left light transmission region 52L includes a multiple and half-ring-shaped light transmission region (hereinafter referred to as a half-ring-shaped light transmission region) 75L, and a semicircular disk located inside the half-ring-shaped light transmission region 75L. A light transmitting region (hereinafter referred to as a central light transmitting region) 76L is formed. Further, in the right light transmission region 52R, a multiple half-ring-shaped light transmission region 75R and a semi-disc-shaped central light transmission region 76R located inside the half-ring-shaped light transmission region 75R are formed.

  As described above, the light amounts of the subject light 50L and 50R can be individually adjusted by making the change in the light transmittance different between the left and right light transmission regions 52L and 52R. Thereby, the light amounts of the subject lights 50L and 50R can be individually adjusted more accurately than in the third embodiment.

  In the example shown in FIGS. 25 and 26, the number and size of each of the half-annular light transmission regions 75L and 75R (ring width) are different, but the number and size of each region are different. Only the light transmittance may be varied in the same manner. In addition, the central light transmission regions 76L and 76R may have the same number of regions and light transmittance. That is, the number, size, and light transmittance (hereinafter, referred to as a first parameter) of each of the half-ring light transmitting regions 75L, 75R, the size of each of the central light transmitting regions 76L, 76R, and The light transmittance (hereinafter referred to as the second parameter) is appropriately set according to the pixel sensitivity difference.

  The first parameter of each of the semi-annular light transmission regions 75L and 75R and the second parameter of each of the central light transmission regions 76L and 76R are determined based on the light amount correction data 78. The light quantity correction data 78 is obtained by the element inspection device 56 in the same manner as the light quantity correction data 55 and 70 of the first and second embodiments. Based on the pixel sensitivity difference obtained from the incident angle dependency information 58 shown in FIG. 14, the element inspection device 56 uses a data table, an arithmetic expression, or the like obtained in advance, so that the pixel sensitivity difference is 0 (including almost 0). First and second parameters are determined such that

  The light quantity control unit 73 forms the half-ring-like light transmission areas 75L and 75R and the central light transmission areas 76L and 76R in the left and right light transmission areas 52L and 52R, respectively, according to the first and second parameters determined by the light quantity correction data 78. . As a result, the pixel sensitivity difference can be adjusted in the same manner as in the above-described embodiments, so that favorable parallax image data can be obtained.

  Further, by forming the semi-annular light transmission regions 75L and 75R and the central light transmission regions 76L and 76R in the left and right light transmission regions 52L and 52R, respectively, the multiple annular light transmission regions described in the third embodiment and Although the shape is slightly different, a light transmission region having a substantially multiple annular zone is formed in the light transmission part 22a. Therefore, as described in the third embodiment, it is possible to adjust the light amount of vertically incident light and obliquely incident light, to uniformly inject subject light on the entire imaging surface 23a, to emphasize parallax, and to perform shading correction.

  When obtaining parallax image data of three or more viewpoints, the number of divisions of the light transmission region of the liquid crystal shutter 22 (light transmission unit 22a) is increased accordingly, and the light transmittance is adjusted for each region. Good.

[Configuration of Digital Camera (Liquid Crystal Shutter) of Fifth Embodiment]
Next, a digital camera according to a fifth embodiment of the present invention will be described. In each of the above embodiments, the digital camera that obtains the left and right LR viewpoint images has been described. However, the digital camera of the fifth embodiment obtains the four viewpoint images of the top, bottom, left, and right.

  The digital camera of the fifth embodiment has basically the same configuration as that of the first embodiment except that the color image sensor and the liquid crystal shutter are different. For this reason, the same functions and configurations as those of the first embodiment are denoted by the same reference numerals, and description thereof is omitted.

  As shown in FIG. 27, on the imaging surface 80a of the color imaging device 80 of the fifth embodiment, a G pixel block 83G that selectively receives green subject light and a B that selectively receives blue subject light. A pixel block 83B and an R pixel block 83R that selectively receives red subject light are two-dimensionally arranged.

  The G pixel block 83G includes a total of four types of G pixels, a first to fourth pixels G1, G2, G3, and G4, arranged in a square matrix. The first pixel G1 is provided at the upper left in the figure. The second pixel G2 is provided at the upper right in the figure. The third pixel G3 is provided at the lower left in the figure. The fourth pixel G4 is provided at the lower right in the figure.

  The B pixel block 83B is formed by arranging a total of four types of B pixels of first to fourth pixels B1, B2, B3, and B4 in a square matrix. The R pixel block 83R includes a total of four types of R pixels, a first to fourth pixels R1, R2, R3, and R4, arranged in a square matrix. The arrangement of the B and R pixel blocks 83B and 83R is the same as that of the G pixel block 83G.

  One microlens 85 is provided on each of the pixel blocks 83R, 83G, and 83B. The left region of the microlens 85 in the drawing condenses the subject light that has passed through the left region of the photographing optical system 17 on the second pixels R2, G2, B2 and the fourth pixels R4, G4, B4, respectively. The right region in the drawing of the microlens 85 condenses the subject light that has passed through the right region of the photographing optical system 17 on the first pixels R1, G1, B1 and the third pixels R3, G3, B3, respectively.

  The upper region of the microlens 85 in the drawing condenses the subject light that has passed through the upper region of the photographic optical system 17 on the third pixels R3, G3, B3 and the fourth pixels R4, G4, B4, respectively. The lower region of the microlens 85 in the drawing condenses the subject light that has passed through the lower region of the photographing optical system 17 on the first pixels R1, G1, B1 and the second pixels R2, G2, B2, respectively. Thus, in the color image sensor 80, the microlens 85 functions as a pupil division unit.

  The parallax image generation unit 29b according to the fifth embodiment forms viewpoint image data of four viewpoints (up, down, left, and right) based on output signals output from the first to fourth pixels. The viewpoint image data of the left viewpoint is generated based on output signals from the second pixels R2, G2, B2 and the fourth pixels R4, G4, B4. The viewpoint image data for the right viewpoint is generated based on output signals from the first pixels R1, G1, B1 and the third pixels R3, G3, B3.

  The viewpoint image data of the upper viewpoint is generated based on output signals from the third pixels R3, G3, B3 and the fourth pixels R4, G4, B4. The viewpoint image data of the lower viewpoint is generated based on output signals from the first pixels R1, G1, B1 and the second pixels R2, G2, B2. These four viewpoint image data are compressed as parallax image data by the compression / decompression processing circuit 31 and then recorded on the memory card 10 via the media I / F 32.

  As shown in FIG. 28, the liquid crystal shutter 87 is provided in the opening 51a and has a light shielding unit 89 having four first to fourth small openings 88a, 88b, 88c, and 88d arranged in a square matrix. Individual light quantity adjustment units 90a, 90b, 90c, and 90d provided in the small openings 88a to 88d, respectively. For this reason, the liquid crystal shutter 87 has a so-called quadrupole shape.

  The first small opening 88a passes through the left region of the photographic optical system 17 and passes through a part of the subject light condensed on the second pixels R2, G2, and B2 and the upper region of the photographic optical system 17. Part of the subject light condensed on the three pixels R3, G3, and B3 is transmitted. The second small opening 88b passes through the right region of the photographic optical system 17 and part of the subject light condensed on the first pixels R1, G1, and B1 and the upper region of the photographic optical system 17 through the first region. A part of the subject light condensed on the four pixels R4, G4, B4 is transmitted.

  The third small opening 88c passes through the left region of the photographic optical system 17 and passes through a part of subject light condensed on the fourth pixels R4, G4, and B4 and the lower region of the photographic optical system 17 to pass through the third small opening 88c. A part of subject light condensed on one pixel R1, G1, B1 is transmitted. The fourth small opening 88d passes through the right region of the photographic optical system 17 and part of the subject light condensed on the third pixels R3, G3, and B3 and the lower region of the photographic optical system 17 to pass through the fourth small opening 88d. Part of the subject light condensed on the two pixels R2, G2, and B2 is transmitted.

  The individual light amount adjusting units 90a to 90d are configured by a liquid crystal panel having light transmittance and adjustable light transmittance, like the individual light amount adjusting units 63L and 63R of the second embodiment. The light transmittance of the individual light amount adjustment units 90 a to 90 d is adjusted by the light amount control unit 91. Light amount correction data 92 is used for adjusting the light transmittance by the light amount control unit 91.

  The light quantity correction data 92 is obtained by the element inspection device 56 in the same manner as the light quantity correction data 55 of the first embodiment. The element inspection device 56 calculates the sensitivity difference of each pixel based on the result of measuring the incident angle dependency of each pixel in the same pixel block so that the sensitivity difference of each pixel becomes 0 (including almost 0). The light transmittance (light amount correction data 92) of each individual light amount adjusting unit 90a to 90d is determined.

  In FIG. 29 showing the change of the light transmittance along the X7-X7 ′ line of the liquid crystal shutter 87 in FIG. 28 and FIG. 30 showing the change of the light transmittance along the X8-X8 ′ line, the light quantity control unit 91 is: Based on the light amount correction data 92 stored in the memory 13, the light transmittances of the individual light amount adjustment units 90a to 90d are adjusted. As a result, the light amount of the subject light transmitted through each of the individual light amount adjustment units 90a to 90d is individually adjusted. In other words, the light amount of each subject light can be individually adjusted for each part of the region in the opening 51a (individual light amount adjustment units 90a to 90d). As a result, the sensitivity difference of each pixel in each pixel block 83R, 83G, 83B is adjusted, so that good parallax image data can be obtained.

  In addition, by using the quadrupole liquid crystal shutter 87, the separation of the subject light that has passed through each area of the photographing optical system 17 is improved as in the second embodiment, so that the parallax is more emphasized. Parallax image data is obtained.

  Instead of the liquid crystal shutter 87 having a quadrupole shape, the liquid crystal shutter may be divided into four quarter-circle regions, and the light transmittance may be individually adjusted in each divided region.

[Configuration of Digital Camera (Liquid Crystal Shutter) of Sixth Embodiment]
Next, a digital camera according to a sixth embodiment of the present invention will be described. In the fifth embodiment, each of the pixel blocks 83R, 83G, and 83B includes four types of pixels. In the digital camera of the sixth embodiment, each of the pixel blocks 83R, 83G, and 83B includes nine types of pixels. ing.

  The digital camera of the sixth embodiment has basically the same configuration as that of the fifth embodiment except that the color image sensor and the liquid crystal shutter are different. For this reason, the same functions and configurations as those of the fifth embodiment are denoted by the same reference numerals, and the description thereof is omitted.

  As shown in FIG. 31, in the color image sensor 94 of the sixth embodiment, each pixel block 83R, 83G, 83B has nine pixels arranged in a square matrix (first to ninth pixels R1 to R9, first pixel). 1 to 9th pixels G1 to G9 and 1st to 9th pixels B1 to B9). In addition, as in the fifth embodiment, the microlens 85 functions as a pupil division unit. Therefore, in the sixth embodiment, viewpoint image data (parallax image data) of nine viewpoints configured by output signals respectively output from the pixels of the pixel blocks 83R, 83G, and 83B is obtained.

  As shown in FIG. 32, the liquid crystal shutter 96 is provided in the opening 51a, and includes a light shielding portion 97 having nine first to ninth small openings 88a to 88i arranged in a square matrix, and each small opening. It has a so-called nine-pole shape including individual light quantity adjustment units 90a to 90i provided in 88a to 88i, respectively. The liquid crystal shutter 96 is basically the same as the liquid crystal shutter 87 of the fifth embodiment except that the individual light amount adjustment units 90a to 90i are increased with the increase in the number of pixels constituting each pixel block 83R, 83G, 83B. Therefore, detailed description is omitted.

  By individually adjusting the light amount of the subject light passing through the individual light amount adjustment units 90a to 90i, the sensitivity difference between the pixels in the pixel blocks 83R, 83G, and 83B is adjusted as in the fifth embodiment. . Thereby, good parallax image data is obtained. Further, by using the liquid crystal shutter 96 having a nine-pole shape, parallax image data with more emphasized parallax can be obtained.

Each pixel block 83R, 83G, N ^{2 (N} is a natural number of 4 or larger) to 83B type when the pixel is a square matrix, N ² pieces of small openings and the light quantity adjusting unit arranged in a square matrix it may be used liquid crystal shutter in the form of N ² dipole that is. Further, each pixel block 83R, 83G, 83B may be composed of two or more types of pixels arranged in an arbitrary pattern instead of being composed of pixels arranged in a square matrix.

[Other Embodiments]
In each of the above embodiments, the light amount of each subject light that has passed through a plurality of different regions of the photographing optical system is individually adjusted by the liquid crystal shutter, but instead of the liquid crystal shutter, the light amount of each subject light such as an ND filter is adjusted. Various adjustable members, devices, and mechanisms (light amount adjusting means) may be provided. Further, the liquid crystal shutter is provided between the mechanical shutter 18 (aperture) and the color image sensor 23, but may be provided between the photographing optical system 17 and the mechanical shutter 18, for example.

  In the first embodiment, the individual adjustment of the light amounts of the subject light 50L and 50R is performed to correct the pixel sensitivity difference caused by the positional deviation of the microlens 48. However, as a factor other than the positional deviation of the microlens 48, The present invention can also be applied to correction of the resulting pixel sensitivity difference. The same applies to each embodiment other than the first embodiment.

  In each of the above embodiments, a digital camera has been described as an example of the imaging apparatus of the present invention. For example, the present invention is also applied to a mobile phone, a smartphone, a PDA (Personal Digital Assistants), and a portable game machine having a camera function. Can be applied. Hereinafter, a smartphone will be described as an example, and will be described in detail with reference to the drawings.

<Configuration of smartphone>
FIG. 33 shows the appearance of the smartphone 500. A smartphone 500 shown in FIG. 33 includes a flat housing 502, and a display input in which a display panel 521 as a display unit and an operation panel 522 as an input unit are integrated on one surface of the housing 502. Part 520. The housing 502 includes a speaker 531, a microphone 532, an operation unit 540, and a camera unit 541. Note that the configuration of the housing 502 is not limited thereto, and for example, a configuration in which the display unit and the input unit are independent, or a configuration having a folding structure or a slide mechanism can be employed.

  FIG. 34 is a block diagram showing a configuration of smartphone 500 shown in FIG. As shown in FIG. 34, the main components of the smartphone include a wireless communication unit 510, a display input unit 520, a call unit 530, an operation unit 540, a camera unit 541, a storage unit 550, and an external input / output unit. 560, a GPS (Global Positioning System) receiving unit 570, a motion sensor unit 580, a power supply unit 590, and a main control unit 501. In addition, as a main function of the smartphone 500, a wireless communication function for performing mobile wireless communication via the base station device BS and the mobile communication network NW is provided.

  The radio communication unit 510 performs radio communication with the base station apparatus BS accommodated in the mobile communication network NW according to an instruction from the main control unit 501. Using this wireless communication, transmission and reception of various file data such as audio data and image data, e-mail data, and reception of Web data and streaming data are performed.

  The display input unit 520 displays images (still images and moving images), character information, and the like visually by transmitting information to the user under the control of the main control unit 501, and detects user operations on the displayed information. A so-called touch panel, which includes a display panel 521 and an operation panel 522. When viewing the generated 3D image, the display panel 521 is preferably a 3D display panel.

  The display panel 521 uses an LCD (Liquid Crystal Display), an OELD (Organic Electro-Luminescence Display), or the like as a display device. The operation panel 522 is a device that is placed so that an image displayed on the display surface of the display panel 521 is visible and detects one or a plurality of coordinates operated by a user's finger or stylus. When this device is operated by a user's finger or stylus, a detection signal generated due to the operation is output to the main control unit 501. Next, the main control unit 501 detects an operation position (coordinates) on the display panel 521 based on the received detection signal.

  As shown in FIG. 33, the display panel 521 and the operation panel 522 of the smartphone 500 integrally form the display input unit 520, but the operation panel 522 is disposed so as to completely cover the display panel 521. ing. When this arrangement is adopted, the operation panel 522 may have a function of detecting a user operation even in an area outside the display panel 521. In other words, the operation panel 522 includes a detection area (hereinafter referred to as a display area) for an overlapping portion that overlaps the display panel 521 and a detection area (hereinafter, a non-display area) for an outer edge portion that does not overlap the other display panel 521. May be included).

  Although the size of the display area and the size of the display panel 521 may be completely matched, it is not always necessary to match them. In addition, the operation panel 522 may include two sensitive regions of the outer edge portion and the other inner portion. Further, the width of the outer edge portion is appropriately designed according to the size of the housing 502 and the like. Furthermore, examples of the position detection method employed in the operation panel 522 include a matrix switch method, a resistance film method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, and a capacitance method. You can also

  The call unit 530 includes a speaker 531 and a microphone 532, and converts a user's voice input through the microphone 532 into voice data that can be processed by the main control unit 501, and outputs the voice data to the main control unit 501, or a wireless communication unit 510 or the audio data received by the external input / output unit 560 is decoded and output from the speaker 531. As shown in FIG. 33, for example, the speaker 531 can be mounted on the same surface as the display input unit 520 and the microphone 532 can be mounted on the side surface of the housing 502.

  The operation unit 540 is a hardware key using a key switch or the like, and receives an instruction from the user. For example, as shown in FIG. 34, the operation unit 540 is mounted on the lower and lower side of the display unit of the housing 502 of the smartphone 500, and turns on when pressed with a finger or the like, and restores a spring or the like when the finger is released. It is a push button type switch that is turned off by force.

  The storage unit 550 includes control software and control data for the main control unit 501, application software including an image processing program for generating a left-eye image and a right-eye image according to the present invention, and a first used for generating a stereoscopic image. And the second digital filter group, the parallax map, the address data that associates the name and telephone number of the communication partner, the transmitted / received e-mail data, the web data downloaded by web browsing, and the downloaded content data, It also temporarily stores streaming data and the like. The storage unit 550 includes an internal storage unit 551 with a built-in smartphone and an external storage unit 552 having a removable external memory slot. Each of the internal storage unit 551 and the external storage unit 552 constituting the storage unit 550 includes a flash memory type, a hard disk type, a multimedia card micro type, This is realized using a storage medium such as a card type memory (for example, Micro SD (registered trademark) memory), a RAM (Random Access Memory), a ROM (Read Only Memory), or the like.

  The external input / output unit 560 serves as an interface with all external devices connected to the smartphone 500, and communicates with other external devices (for example, universal serial bus (USB), IEEE1394, etc.) or a network. (For example, the Internet, wireless LAN, Bluetooth (registered trademark), RFID (Radio Frequency Identification), infrared communication (Infrared Data Association: IrDA) (registered trademark), UWB (Ultra Wideband) (registered trademark), ZigBee ( ZigBee) (registered trademark, etc.) for direct or indirect connection.

  As an external device connected to the smartphone 500, for example, a wired / wireless headset, a wired / wireless external charger, a wired / wireless data port, a memory card connected via a card socket, a SIM (Subscriber) Identity Module Card) / UIM (User Identity Module Card) card, external audio / video equipment connected via audio / video I / O (Input / Output) terminal, external audio / video equipment connected wirelessly, yes / no There are a wirelessly connected smartphone, a wired / wireless personal computer, a wired / wireless PDA, a wired / wireless personal computer, an earphone, and the like. The external input / output unit may transmit data received from such an external device to each component inside the smartphone 500, or may allow data inside the smartphone 500 to be transmitted to the external device. it can.

  The GPS receiving unit 570 receives GPS signals transmitted from the GPS satellites ST1 to STn in accordance with instructions from the main control unit 501, performs positioning calculation processing based on the received plurality of GPS signals, the latitude of the smartphone 500, A position consisting of longitude and altitude is detected. When the GPS reception unit 570 can acquire position information from the wireless communication unit 510 or the external input / output unit 560 (for example, a wireless LAN), the GPS reception unit 570 can also detect the position using the position information.

  The motion sensor unit 580 includes, for example, a three-axis acceleration sensor and detects the physical movement of the smartphone 500 in accordance with an instruction from the main control unit 501. By detecting the physical movement of the smartphone 500, the moving direction and acceleration of the smartphone 500 are detected. This detection result is output to the main control unit 501.

  The power supply unit 590 supplies power stored in a battery (not shown) to each unit of the smartphone 500 in accordance with an instruction from the main control unit 501.

  The main control unit 501 includes a microprocessor, operates according to a control program and control data stored in the storage unit 550, and controls each unit of the smartphone 500 in an integrated manner. Further, the main control unit 501 includes a mobile communication control function for controlling each unit of the communication system and an application processing function in order to perform voice communication and data communication through the wireless communication unit 510.

  The application processing function is realized by the main control unit 501 operating according to application software stored in the storage unit 550. Examples of the application processing function include an infrared communication function for controlling the external input / output unit 560 to perform data communication with the opposite device, an e-mail function for sending and receiving e-mails, a web browsing function for browsing web pages, and the present invention. And a function for generating a 3D image from the 2D image according to the above.

  Further, the main control unit 501 has an image processing function such as displaying video on the display input unit 520 based on image data (still image or moving image data) such as received data or downloaded streaming data. The image processing function is a function in which the main control unit 501 decodes the image data, performs image processing on the decoding result, and displays an image on the display input unit 520.

  Further, the main control unit 501 executes display control for the display panel 521 and operation detection control for detecting a user operation through the operation unit 540 and the operation panel 522.

  By executing the display control, the main control unit 501 displays an icon for starting application software, a software key such as a scroll bar, or a window for creating an e-mail. Note that the scroll bar refers to a software key for accepting an instruction to move the display portion of a large image that does not fit in the display area of the display panel 521.

  Further, by executing the operation detection control, the main control unit 501 detects a user operation through the operation unit 540, or accepts an operation on the icon or an input of a character string in the input field of the window through the operation panel 522. Or a display image scroll request through a scroll bar.

  Furthermore, by executing the operation detection control, the main control unit 501 causes the operation position with respect to the operation panel 522 to overlap with the display panel 521 (display area) or other outer edge part (non-display area) that does not overlap with the display panel 521. And a touch panel control function for controlling the sensitive area of the operation panel 522 and the display position of the software key.

  The main control unit 501 can also detect a gesture operation on the operation panel 522 and execute a preset function in accordance with the detected gesture operation. Gesture operation is not a conventional simple touch operation, but an operation that draws a trajectory with a finger or the like, designates a plurality of positions at the same time, or combines these to draw a trajectory for at least one of a plurality of positions. means.

  A camera unit (imaging device) 541 is a digital camera that performs electronic photography using an imaging device such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge-Coupled Device), and is basically the same as the digital camera of each of the above embodiments. It is the same configuration.

  The camera unit 541 converts image data obtained by imaging into compressed image data such as JPEG (Joint Photographic coding Experts Group) under the control of the main control unit 501, and records the data in the storage unit 550, The data can be output through the input / output unit 560 and the wireless communication unit 510. In the smartphone 500 shown in FIG. 33, the camera unit 541 is mounted on the same surface as the display input unit 520, but the mounting position of the camera unit 541 is not limited to this, and the camera unit 541 may be mounted on the back surface of the display input unit 520. Alternatively, a plurality of camera units 541 may be mounted. Note that when a plurality of camera units 541 are mounted, the camera unit 541 used for shooting may be switched to shoot alone, or a plurality of camera units 541 may be used for shooting simultaneously. it can.

  The camera unit 541 can be used for various functions of the smartphone 500. For example, an image acquired by the camera unit 541 can be displayed on the display panel 521, or the image of the camera unit 541 can be used as one of operation inputs of the operation panel 522. Further, when the GPS receiving unit 570 detects the position, the position can also be detected with reference to an image from the camera unit 541. Furthermore, referring to an image from the camera unit 541, the optical axis direction of the camera unit 541 of the smartphone 500 can be determined without using the triaxial acceleration sensor or in combination with the triaxial acceleration sensor, The current usage environment can also be determined. Of course, the image from the camera unit 541 can be used in the application software.

  In addition, the position information acquired by the GPS receiver 570 to the image data of the still image or the moving image, the voice information acquired by the microphone 532 (the text information may be converted into voice information by the main control unit or the like), Posture information and the like acquired by the motion sensor unit 580 can be added and recorded in the storage unit 550 or output through the external input / output unit 560 or the wireless communication unit 510.

  DESCRIPTION OF SYMBOLS 2 ... Digital camera, 11 ... CPU, 17 ... Shooting optical system, 22, 60, 87, 96 ... Liquid crystal shutter, 23, 80, 94 ... Color image sensor, 48 ... Micro lens, 54, 66, 73, 91 ... Light quantity Control unit 55, 70, 78, 92... Light amount correction data, 61L, 61R, 88a to 88d... Small aperture, 63L, 63R, 90a to 90d.

Claims (15)

Photographic optics,
Imaging in which a plurality of types of pixels including a photoelectric conversion element and two-dimensionally arraying a plurality of types of pixels that selectively receive subject light that has passed through a plurality of different regions of the photographing optical system by pupil division Elements,
A light amount adjusting means for individually adjusting the amount of light of the subject light according to the difference in sensitivity of the plurality of types of pixels, and adjusting the sensitivity difference of the plurality of types of pixels by the amount of light;
An imaging apparatus comprising:   The light amount adjusting means has an opening through which the subject light passes, and the amount of light of the subject light can be individually adjusted for each partial region in the opening. Imaging device. The light amount adjusting means is
A light-shielding portion that is provided in the opening and has a plurality of small openings that respectively pass a part of the subject light;
The imaging apparatus according to claim 2, further comprising: an individual light amount adjustment unit that is individually provided in each of the small openings and adjusts the amount of the subject light that passes through each of the small openings. The light amount adjusting means is
A light transmissive portion provided in the opening and capable of adjusting light transmittance for each position through which the subject light is transmitted;
The light transmission part is controlled to change the light transmittance stepwise from the center side to the outside, and the light transmission part includes a first light transmission region having a multi-ring zone shape and a first light transmission region of the first light transmission region. A first light transmission portion control means for forming a second light transmission region located on the inner side, wherein the individual adjustment is performed by decentering a position of the second light transmission region with respect to a center of the light transmission portion. The imaging device according to claim 2, further comprising: a first light transmission unit control unit that performs.   The imaging apparatus according to claim 4, wherein the first light transmission part control unit increases the light transmittance of the light transmission part in a stepwise manner from the center side toward the outside. The light amount adjusting means is
A light transmissive portion provided in the opening and capable of adjusting light transmittance for each position through which the subject light is transmitted;
A second light transmission unit control means for individually controlling the light transmittance of each light transmission region of the light transmission unit through which each subject light is transmitted; The imaging apparatus according to claim 2, further comprising: a second light transmission unit control unit that performs the individual adjustment by individually changing the light transmittance of each light transmission region in a stepwise manner.   The imaging device according to claim 6, wherein the second light transmission unit control unit changes the number of steps when changing the light transmittance stepwise for each light transmission region.   The imaging device according to claim 6 or 7, wherein the second light transmission unit control unit increases the light transmittance of each of the light transmission regions in a stepwise manner from the center side toward the outside. The plurality of different regions of the photographing optical system are two regions that are symmetrical with respect to a first straight line perpendicular to the optical axis of the photographing optical system,
The second light transmission portion control means includes a first divided light transmission region and a second light transmission region formed by dividing the light transmission portion along a second straight line passing through the center of the light transmission portion and parallel to the first straight line. The imaging device according to claim 6, wherein the light transmittance of the divided light transmission region is changed stepwise.   The imaging apparatus according to claim 1, wherein the imaging element includes a microlens that collects the subject light on the photoelectric conversion elements of the plurality of types of pixels. In the imaging device, pixel blocks composed of the plurality of types of pixels are two-dimensionally arranged,
A microlens provided on each of the pixel blocks, the microlens for condensing the subject light that has passed through the small openings on the photoelectric conversion elements of the plurality of types of pixels. Item 4. The imaging device according to Item 3. In the pixel block, N ² (N is a natural number of 2 or more) types of pixels are arranged in a square matrix,
The imaging apparatus according to claim 11, wherein N ² small apertures and the individual light amount adjustment units are arranged in a square matrix in the light shielding unit.   The imaging device according to any one of claims 10 to 12, wherein the difference in sensitivity between the plurality of types of pixels is caused by a positional shift of the microlens.   The imaging apparatus according to claim 1, wherein the light amount adjusting unit is a liquid crystal shutter.   The imaging device according to claim 1, further comprising an image generation unit configured to generate a parallax image including a plurality of images respectively configured by output signals of the plurality of types of pixels.

Images