JP5045007B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging element and an imaging apparatus.

A pixel structure including one microlens and one photoelectric conversion unit disposed behind the microlens is configured so that the first type pixel and the second type pixel receive light incident from different directions. An imaging device that detects the focus adjustment state of an image by a so-called pupil division type phase difference detection method by performing imaging with an imaging device in which a first type pixel and a second type pixel are alternately arranged is known. (For example, refer to Patent Document 1).
In the pupil division type phase difference detection method, an object image is formed using two light beams that have passed through different parts of the pupil of the imaging optical system, and the positional phase difference between the two object images is based on the output of the image sensor. This is detected and converted into a defocus amount of the imaging optical system.

Prior art documents related to the invention of this application include the following.
JP 04-267211 A

  However, the conventional imaging apparatus described above has the following problems. First, if the pixels are simply arranged in a two-dimensional form, the same type of pixels may be concentrated in a specific direction. If the pixel output is used as an imaging output as it is, the same type of pixels will be concentrated. There is a risk that the image quality will deteriorate in the direction in which they are arranged.

  FIG. 28 shows an image formation state in the pupil division type phase difference detection method. As shown in FIG. 28A, regions 192 and 193 (hereinafter referred to as distance measurement pupils 192 and 193) on the exit pupil plane 90 of the optical system. The luminous flux passing through the lens) forms a clear image on the focal plane P0. For example, when a line pattern (contrast pattern of white lines on a black background) on the optical axis 91 of the optical system and perpendicular to the paper surface of FIG. 28 is imaged by the optical system, (c) As shown in FIG. 4, a high-contrast line image pattern is formed at the same position on the optical axis 91. On the other hand, as shown in (b) on the surface P1 in front of the focusing surface P0 and as shown in (d) on the surface P2 behind the focusing surface P0, different positions in the opposite direction to (b). A blurred line image pattern is formed.

  A case where an image sensor as shown in FIG. 29 is used as the pixel configuration of the image sensor for separating and detecting two images of light beams from different distance measurement pupils will be described. 29 receives the light beam from the distance measuring pupil 192 shown in FIG. 28 by the first type of pixel 722 in which the position of the photoelectric conversion unit 712 is biased to the right from the center of the microlens 710. On the other hand, the light beam from the distance measuring pupil 193 shown in FIG. 28 is received by the second type pixel 723 in which the position of the photoelectric conversion unit 713 is deviated to the left from the center of the microlens 710. When the imaging element having such a configuration is used, focus detection can be performed using the pixel output, and image information can be generated using the pixel output.

  However, in the image sensor having the pixel arrangement as shown in FIG. 29, the same type of pixels are arranged in the vertical direction, and the columns are arranged alternately in the horizontal direction. Therefore, when an image of the white line pattern is captured in a state where the image sensor is on the P1 plane and the P2 plane shown in FIG. 28, blurred images are generated at two locations in the vertical direction (like two-line blur) and When this blurred image is enlarged, a white portion and a black portion are alternately observed as a zebra pattern, resulting in an unsightly image.

An image pickup apparatus according to a first aspect of the present invention includes a plurality of first pixels that respectively receive one of a pair of beams passing through a pair of regions of an exit pupil of an optical system, and the other of the pair of beams. A plurality of second pixels that respectively receive light, and an image sensor in which the plurality of first pixels and the plurality of second pixels are two-dimensionally arranged, and an output of the first pixel from the image sensor Read means for reading the output of the second pixel, image generation means for generating a captured image signal based on the output of the first pixel and the output of the second pixel read by the read means, and the read means A phase difference detection type focus detection means for detecting a focus adjustment state of the optical system based on the output of the first pixel and the output of the second pixel read out by The first pixel and the first pixel; The pixels are alternately arranged in a first direction in which the pair of regions are arranged, and the first pixel and the second pixel are within a range of eight adjacent pixels in a second direction perpendicular to the first direction. The pixels are arranged so that the same number exists.

  According to the present invention, a high-quality captured image can be obtained while achieving a focus detection function.

  An embodiment in which the present invention is applied to a digital still camera as an imaging apparatus will be described. FIG. 1 is a diagram illustrating a configuration of a digital still camera according to an embodiment. A digital still camera 201 according to an embodiment includes an interchangeable lens 202 and a camera body 203, and the interchangeable lens 202 is attached to a mount portion 204 of the camera body 203.

  The interchangeable lens 202 includes lenses 205 to 207, a diaphragm 208, a lens drive control device 209, and the like. The lens 206 is for zooming, and the lens 207 is for focusing. The lens drive control device 209 includes a CPU and its peripheral components, and controls the driving of the focusing lens 207 and the aperture 208, detects the positions of the zooming lens 206, the focusing lens 207 and the aperture 208, and communicates with the control device of the camera body 203. Transmit lens information and receive camera information.

  On the other hand, the camera body 203 includes an image sensor 211, a camera drive control device 212, a memory card 213, an LCD driver 214, an LCD 215, an eyepiece 216, and the like. The imaging element 211 is disposed on the planned image plane (planned focal plane) of the interchangeable lens 202, captures the subject image formed by the interchangeable lens 202, and outputs an image signal. Pixels (details will be described later) are two-dimensionally arranged on the image sensor 211.

  The camera drive control device 212 includes peripheral components such as a CPU and a memory, and controls the drive of the image sensor 211, processing of the captured image, focus detection and focus adjustment of the interchangeable lens 202, control of the aperture 208, display control of the LCD 215, lens drive Communication with the control device 209, sequence control of the entire camera, and the like are performed. The camera drive control device 212 communicates with the lens drive control device 209 via an electrical contact 217 provided on the mount unit 204.

The memory card 213 is an image storage that stores captured images. The LCD 215 is used as a display of a liquid crystal viewfinder (EVF: electronic viewfinder), and a photographer can visually recognize a captured image displayed on the LCD 215 via an eyepiece lens 216. The camera body 203 has a shutter button, a focus detection position selection switch, etc.
Various operation members (not shown) are provided, and operation signals of these operation members are sent to the camera drive control device 212, and an imaging operation and a focus detection position setting operation are performed according to the operation signals.

  The subject image that has passed through the interchangeable lens 202 and formed on the image sensor 211 is photoelectrically converted by the image sensor 211, and the image output is sent to the camera drive controller 212. The camera drive control device 212 calculates a defocus amount at a predetermined focus detection position based on the pixel output, and sends this defocus amount to the lens drive control device 209. Further, the camera drive control device 212 sends an image signal generated based on the output of the pixel to the LCD driver 214, displays it on the LCD 215, and stores it in the memory card 213.

  The lens drive control device 209 detects the positions of the zooming lens 206, the focusing lens 207, and the diaphragm 208 and calculates lens information based on the detected positions, or a lens corresponding to the detected position from a lookup table prepared in advance. Information is selected and sent to the camera drive controller 212. Further, the lens drive control device 209 calculates the lens drive amount based on the defocus amount received from the camera drive control device 212, and drives and controls the focusing lens 207 based on the lens drive amount.

  FIG. 2 is a front view showing a schematic configuration of the image sensor 211. In the image sensor 211, first-type pixels 311 and second-type pixels 312 described later are regularly arranged in a two-dimensional manner.

  FIG. 3 is an enlarged view (8 pixels × 8 pixels) of the arrangement of the image sensor 211. The first type pixel 311 includes the microlens 10 and the photoelectric conversion unit 12. The photoelectric conversion unit 12 has a rectangular shape, and the left long side thereof is close to the vertical bisector along the arrangement direction of the microlenses 10 (the vertical direction in the drawing). On the other hand, the second type pixel 312 includes the microlens 10 and the photoelectric conversion unit 13. The photoelectric conversion unit 13 is rectangular, and the right long side thereof is close to the vertical bisector of the microlens 10 in parallel. The photoelectric conversion units 12 and 13 are aligned in the horizontal direction when the microlenses 10 of the pixels 311 and 312 are superimposed and displayed, and have a symmetrical shape with respect to the vertical bisector of the microlens 10.

  In this image sensor 211, pixels 311 and 312 are alternately arranged in the horizontal direction (alignment direction of the photoelectric conversion units 12 and 13) to form a pixel row, and this pixel row is one pixel every two rows in the vertical direction. They are staggered.

  Thus, in a square pixel array in which pixels are densely arranged in the horizontal direction and the vertical direction, the horizontal direction (row direction = direction in which pixels adjacent in the horizontal direction are arranged) and the vertical direction (column direction = adjacent in the vertical direction). 3 pixels or more are not consecutively arranged in a 45 ° oblique direction (upward to the left, upward to the right = diagonal direction of adjacent pixels). Further, the first type pixel 311 and the second type pixel 312 appear at the same ratio (that is, 1: 1) in the horizontal direction, the vertical direction, and the oblique 45 degree direction. That is, the same type of pixels are not arranged in a concentrated manner but are arranged in a distributed manner.

  Thereby, zebra pattern imaging when the line pattern is blurred can be prevented. The layout of the pixel array is not limited to the layout shown in FIG. 3, and various layouts are conceivable as will be described later. However, if there are 9 or more pixels of the same type in the same direction within a minute region (for example, 16 pixels × 16 pixels), If you do n’t, it wo n’t stand out.

  FIG. 4 is a cross-sectional view of the first type of pixel 311. In the first type of pixel 311, the microlens 10 is disposed corresponding to the photoelectric conversion unit 12, and the image of the photoelectric conversion unit 12 is projected forward by the microlens 10. The photoelectric conversion unit 12 is formed on the semiconductor circuit substrate 29, and the microlens 10 is integrally and fixedly formed thereon by the manufacturing process of the semiconductor image sensor. The photoelectric conversion unit 12 is arranged on one side with respect to the optical axis of the microlens 10.

  FIG. 5 is a cross-sectional view of the second type pixel 312. In the second type pixel 312, the microlens 10 is disposed so as to face the photoelectric conversion unit 13, and the image of the photoelectric conversion unit 13 is projected forward by the microlens 10. The photoelectric conversion unit 13 is formed on the semiconductor circuit substrate 29, and the microlens 10 is integrally and fixedly formed thereon by the manufacturing process of the semiconductor image sensor. The photoelectric conversion unit 13 is arranged on one side with respect to the optical axis of the microlens 10 and on the side opposite to the photoelectric conversion unit 12.

  Next, a focus detection method based on a pupil division type phase difference detection method using a microlens will be described with reference to FIG. In FIG. 6, reference numeral 90 denotes an exit pupil set at a distance d4 in front of the microlens arranged on the planned imaging plane of the interchangeable lens 202. This distance d4 is determined according to the curvature and refractive index of the microlens, the distance between the microlens and the photoelectric conversion unit, and is hereinafter referred to as a distance measurement pupil distance. 91 is an optical axis of the interchangeable lens 202, 10a to 10c are microlenses, 12a, 12b, 13a and 13b are photoelectric conversion units, 311a and 311b are first type pixels, and 312a and 312b are second type pixels.

  Reference numerals 72, 73, 82, and 83 denote luminous fluxes, and 92 denotes regions of the photoelectric conversion units 12a and 12b projected by the microlenses 10a and 10c (hereinafter referred to as distance measuring pupils). In FIG. 6, for the sake of clarity, the distance measuring pupil is shown as an elliptical area, but in reality, the shape of the photoelectric conversion unit is an enlarged projection. Reference numeral 93 denotes regions of the photoelectric conversion units 13a and 13b projected by the microlenses 10b and 10d (hereinafter referred to as distance measuring pupils). In FIG. 6, for the sake of clarity, the distance measuring pupil is shown as an elliptical area, but in reality, the shape of the photoelectric conversion unit is an enlarged projection.

  In FIG. 6, four adjacent pixels (pixels 311a, 311b, 312a, and 312b) are schematically illustrated. However, in other pixels, the photoelectric conversion unit arrives at each microlens from the corresponding distance measurement pupil. Receiving the luminous flux.

  The microlenses 10a to 10c are arranged in the vicinity of the planned imaging surface of the interchangeable lens 202, and the photoelectric conversion units 12a, 12b, 13a, and 13b arranged behind the microlenses 10a to 10c have the shapes of the microlenses 10a to 10c. 10c is projected onto the exit pupil 90 separated by a distance measurement pupil distance d4, and the projection shape forms distance measurement pupils 92 and 93. That is, the projection direction of the photoelectric conversion unit in each pixel is determined so that the projection shape (ranging pupils 92 and 93) of the photoelectric conversion unit of each pixel matches on the exit pupil 90 at the projection distance d4.

  The photoelectric conversion unit 12a passes through the distance measuring pupil 92 and outputs a signal corresponding to the intensity of the image formed on the microlens 10a by the light beam 72 directed to the microlens 10a. The photoelectric conversion unit 12b outputs a signal corresponding to the intensity of the image formed on the microlens 10c by the light beam 82 passing through the distance measuring pupil 92 and directed to the microlens 10c. Further, the photoelectric conversion unit 13a outputs a signal corresponding to the intensity of the image formed on the microlens 10b by the light beam 73 passing through the distance measuring pupil 93 and directed to the microlens 10b. The photoelectric conversion unit 13b outputs a signal corresponding to the intensity of the image formed on the microlens 10d by the light beam 83 passing through the distance measuring pupil 92 and directed to the microlens 10d.

  By arranging a large number of first-type pixels and second-type pixels as described above in a straight line and collecting the output of the photoelectric conversion unit of each pixel into an output group corresponding to the distance measuring pupil 92 and the distance measuring pupil 93 Information on the intensity distribution of a pair of images formed on the pixel array by the focus detection light beams passing through the distance measuring pupil 92 and the distance measuring pupil 93 is obtained. By applying image shift detection calculation processing (correlation calculation processing, phase difference detection processing) to be described later to these pieces of information, an image shift amount of a pair of images is detected by a so-called pupil division type phase difference detection method. Further, by multiplying the image shift amount by a predetermined conversion coefficient, the current image plane relative to the planned image plane (the image plane at the focus detection position corresponding to the position of the microlens array on the planned image plane) is changed. A deviation (defocus amount) is calculated.

  FIG. 7 is a front view showing the projection relationship on the exit pupil plane. The distance measuring pupils 92 and 93 obtained by projecting the photoelectric conversion unit of each pixel onto the exit pupil plane 90 by the microlens are set to a size including the exit pupil 89 of the optical system. Therefore, the light beam actually received by the photoelectric conversion unit of the pixel becomes a light beam that passes through a region in which the distance measuring pupils 92 and 93 are narrowed by the exit pupil 89 of the optical system. When the distance measuring pupils 92 and 93 are set in a larger area than the exit pupil 89 of the optical system in this way, the photoelectric conversion unit receives light having a light amount corresponding to the control amount of the aperture by controlling the aperture of the optical system. This makes it possible to use a pixel output as image data.

  FIG. 8 is a front view for explaining the arrangement relationship of distance measuring pupils. In the figure, axes x and y indicate a horizontal bisector and a vertical bisector passing through the center of the exit pupil 90 (intersection with the optical axis). The first type pixel 311 shown in FIG. 3 receives the light beam coming from the distance measuring pupil 92, and the second type pixel 312 receives the light beam coming from the distance measuring pupil 93. The distance measuring pupils 92 and 93 are arranged symmetrically with respect to the axis y. Note that FIG. 8 shows an example in which a gap is provided between the distance measurement pupils 92 and 93 for easy understanding, but the gap may be omitted or slightly overlapped.

  Therefore, in the pixel array shown in FIG. 3, since two types of pixels appear in the same direction in the vertical and horizontal diagonal directions, when the luminous flux received by pixels arranged in the same direction is averaged, the center of gravity of the average luminous flux is It coincides with the center of gravity of the light beam obtained by adding the light beam passing through the distance measuring pupil 92 and the light beam passing through the distance measuring pupil 93, and becomes the center of the exit pupil 90 (intersection with the optical axis). As a result, even when pixel data is used as image data, the image in a specific direction is not formed by a light beam that passes through a biased region in the exit pupil, so image quality does not deteriorate.

  FIG. 12 is a flowchart showing the operation of the digital still camera (imaging device) shown in FIG. When the camera is turned on in step 100, the camera drive control device 212 starts an imaging operation. In step 110, pixel data is read out and displayed on the electronic viewfinder. In step 120, an image shift detection calculation process (correlation calculation process), which will be described later, is performed based on the data of the first type pixel and the data of the second type pixel corresponding to the pixel area corresponding to the focus detection position. The defocus amount is calculated by calculating the amount. The focus detection position is designated by the photographer via an operation member (not shown).

  In step 130, it is checked whether or not the focus is close, that is, whether or not the absolute value of the calculated defocus amount is within a predetermined value. If it is not close to the in-focus state, the process proceeds to step 140, the defocus amount is transmitted to the lens drive control device 209, the focusing lens 207 of the interchangeable lens 202 is driven to the in-focus position, the process returns to step 110, and the above operation is repeated. Even when focus detection is impossible, the process branches to this step, a scan drive command is transmitted to the lens drive control device 209, and the focusing lens 207 of the interchangeable lens 202 is continuously driven from infinity to close, and the process returns to step 110. Repeat the above operation.

  On the other hand, if it is determined that the focus is in the vicinity, the process proceeds to step 150 to determine whether or not a shutter release has been performed by operating an operation member (not shown). If it is determined that the shutter release has not been performed, the process returns to step 110 to perform the above operation. repeat. If the shutter release has been performed, the process proceeds to step 160, where an aperture adjustment command is transmitted to the lens drive control device 209, and the control F value (set by the photographer or automatically set by the camera) is transmitted to the aperture 208 of the interchangeable lens 202. (F value). When the aperture control is completed, the image sensor 211 performs an image capturing operation, reads data from all the pixels of the image sensor 211, and stores the data in the memory card 213. Image processing (interpolation processing based on surrounding pixels, high-frequency component cut processing, etc.) may be performed before saving in the memory card 213.

  Next, details of the image shift detection calculation process (correlation calculation process) in step 120 of FIG. 12 will be described. FIG. 13 is a partially enlarged view of the image sensor 211 shown in FIG. 2 and shows classification of pixel data used for focus detection. In the first pixel row, the first type of pixel data (L1 series data: L11, L12, L13,...) And the second type of pixel data (R1 series data: R11, R12, R13,...・) Is obtained. Further, in the second pixel row, the first type pixel data (L2 series data: L21, L22, L23,...) And the second type pixel data (R2 series data: R21, R22, R23). , ...) is obtained. Similarly, the data of the first type of pixel and the data of the second type of pixel are obtained from the third row. Image shift detection is performed by a combination of a pair of data series (L1 series data, R1 series data), (L2 series data, R2 series data),.

When a pair of data series is expressed by being generalized as (E1 to EL) and (F1 to FL), the data series (F1 to FL) is shifted relative to the data series (E1 to EL) while the following formula ( According to 1), the correlation amount C (k) in the shift amount k between the two data strings is calculated.
C (k) = Σ | En−Fn + k | (1)
In equation (1), the range taken by n in the Σ operation is limited to the range where En and Fn + k data exist according to the shift amount k. The shift amount k is an integer, and is a relative shift amount in units of the detection pitch of a pair of data.

As shown in FIG. 14A, the calculation result of the expression (1) shows that the correlation amount C (k) is the shift amount with high correlation between the pair of data series (k = kj = 2 in FIG. 14A). The minimum (the smaller the value, the higher the degree of correlation). Next, the shift amount x that gives the minimum value C (x) with respect to the continuous correlation amount is obtained by using a three-point interpolation method according to the following equations (2) to (5).
x = kj + D / SLOP (2),
C (x) = C (kj) − | D | (3),
D = {C (kj-1) -C (kj + 1)} / 2 (4),
SLOP = MAX {C (kj + 1) -C (kj), C (kj-1) -C (kj)} (5)

The defocus amount DEF of the subject image plane with respect to the planned image formation plane can be obtained from the following equation (6) from the shift amount x obtained from the equation (2).
DEF = KX · PY · x (6)
In equation (6), PY is a detection pitch (arrangement pitch of the same type of pixels), and KX is a conversion coefficient determined by the size of the opening angle of the center of gravity of a pair of light beams. In addition, when the pair of data series exactly match (x = 0), the data string is actually shifted by half of the detected pitch, so the shift amount obtained by equation (2) is only half the pitch. It is corrected and applied to equation (6). Further, the size of the opening angle of the center of gravity of the pair of light beams changes according to the size of the aperture opening (open aperture value) of the interchangeable lens, and thus is determined according to the lens information.

  Whether or not the calculated defocus amount DEF is reliable is determined as follows. As shown in FIG. 14B, when the degree of correlation between the pair of data series is low, the value of the minimum value C (x) of the interpolated correlation amount is large. Therefore, when C (x) is equal to or greater than a predetermined value, it is determined that the reliability is low. Alternatively, in order to normalize C (x) with the contrast of data, if the value obtained by dividing C (x) by SLOP that is proportional to the contrast is equal to or greater than a predetermined value, it is determined that the reliability is low. Furthermore, when SLOP that is a value proportional to the contrast is equal to or less than a predetermined value, it is determined that the subject has low contrast and the reliability of the calculated defocus amount DEF is low.

  As shown in FIG. 14 (c), when the correlation between the pair of data series is low and there is no drop in the correlation amount C (k) between the predetermined shift ranges kmin to kmax, the minimum value C (x) is set. In such a case, it is determined that the focus cannot be detected. When focus detection is possible, the defocus amount is calculated by multiplying the calculated image shift amount by a predetermined conversion coefficient.

  In the above description, the defocus amount obtained for a pair of data series is adopted, but the defocus amount obtained for a plurality of pairs of data series in the focus detection area is subjected to statistical processing (average, median, etc.). The final defocus amount may be obtained.

<Modification>
FIG. 15 shows a modification of the pixel array of the image sensor 211. The first type pixel 314 includes the microlens 10 and the photoelectric conversion unit 14. The photoelectric conversion unit 14 has a rectangular shape, and one long side of the photoelectric conversion unit 14 is close to the bisector of the microlens 10 at an angle of 45 degrees to the right. The second type pixel 315 includes the microlens 10 and the photoelectric conversion unit 15. The photoelectric conversion unit 15 has a rectangular shape, and one long side of the photoelectric conversion unit 15 is close to the bisector of the microlens 10 at an angle of 45 degrees upward to the right. The photoelectric conversion units 14 and 15 are arranged in a diagonally upward 45 degree leftward direction when the microlenses 10 are superimposed and displayed, and have a symmetrical shape with respect to the bisector of the microlens 10 obliquely upward 45 degree to the right. is doing.

  The distance measuring pupils 96 and 97 corresponding to the photoelectric conversion units 14 and 15 are shown in FIG. The first type pixel 314 receives the light beam coming from the distance measuring pupil 96, and the second type pixel 315 receives the light beam coming from the distance measuring pupil 97. The distance measuring pupils 94 and 95 are arranged symmetrically with respect to a straight line of 45 degrees to the right passing through the origins of the x and y axes. An array in which the pixels 314 and 315 are alternately arranged in the 45 ° upward diagonal direction (the arrangement direction of the photoelectric conversion units 14 and 15) is shifted by half a pixel every two pixels in the 45 ° upward diagonal direction. As described above, in a square pixel array in which pixels are densely arranged in the horizontal direction and the vertical direction, it is possible to detect an image shift in a 45 ° upward and diagonal direction.

  As described above, in a square pixel array in which pixels are densely arranged in the horizontal direction and the vertical direction, the horizontal direction (row direction = direction in which adjacent pixels are arranged in the horizontal direction) and the vertical direction (column direction = vertical direction). In the direction in which adjacent pixels are lined up) and in the oblique 45 ° direction (upward to the left, upward to the right = the diagonal direction of the adjacent pixels), four or more pixels of the same type are not continuous. In addition, the first type pixel 314 and the second type pixel 315 appear at the same ratio (that is, 1: 1) in the horizontal direction, the vertical direction, and the oblique 45 degree direction. Thereby, zebra pattern imaging when the line pattern is blurred can be prevented.

  In the pixel array shown in FIG. 15, two types of pixels appear in the same direction in vertical and horizontal diagonal directions. Therefore, when the luminous fluxes received by pixels arranged in the same direction are averaged, the center of gravity of the average luminous flux is It coincides with the center of gravity of the light beam obtained by adding the light beams passing through the plurality of distance measurement pupils, and becomes the center of the exit pupil 90 (intersection with the optical axis). As a result, even when pixel data is used as image data, the image in a specific direction is not formed by a light beam that passes through a biased region in the exit pupil, so image quality does not deteriorate.

  FIG. 16 shows another modification of the pixel array of the image sensor 211. The first type pixel 311 includes the microlens 10 and the photoelectric conversion unit 12. The photoelectric conversion unit 12 is rectangular, and the left long side thereof is close to a vertical bisector along the arrangement direction of the microlenses 10 (the vertical direction in the figure). The second type pixel 312 includes the microlens 10 and the photoelectric conversion unit 13. The photoelectric conversion unit 13 is rectangular, and the right long side thereof is close to the vertical bisector of the microlens 10 in parallel. The photoelectric conversion units 12 and 13 are arranged in the horizontal direction when the microlenses 10 are displayed in an overlapped manner, and have a symmetrical shape with respect to the vertical bisector of the microlens 10.

  The third type pixel 316 includes the microlens 10 and the photoelectric conversion unit 16. The photoelectric conversion unit 16 has a rectangular shape, and an upper long side thereof is close to a horizontal bisector along the arrangement direction of the microlenses 10 (the horizontal direction in the drawing). The fourth type pixel 317 includes the microlens 10 and the photoelectric conversion unit 17. The photoelectric conversion unit 17 is rectangular, and its lower long side is close to the horizontal bisector of the microlens 10 in parallel. The photoelectric conversion units 16 and 17 are arranged vertically in the vertical direction when the microlenses 10 are superimposed and displayed, and have a symmetrical shape with respect to the horizontal bisector of the microlens 10. FIG. 9 shows distance measuring pupils 94 and 95 corresponding to the photoelectric conversion units 16 and 17. The third type pixel 316 receives the light beam coming from the distance measuring pupil 94, and the fourth type pixel 317 receives the light beam coming from the distance measuring pupil 95. The distance measuring pupils 94 and 95 are arranged line-symmetrically with respect to the axis x.

  A second pixel row in which pixels 316 and 317 are alternately arranged in the horizontal direction is arranged in a row next to the first pixel row in which pixels 311 and 312 are alternately arranged in the horizontal direction. Includes a third pixel row in which the first pixel row is shifted by one pixel in the horizontal direction, and a fourth pixel row in which the second pixel row is shifted in the horizontal direction by one pixel is further arranged in the next row. Be placed. The above four rows are sequentially arranged in the vertical direction. As described above, in a square pixel array in which pixels are densely arranged in the horizontal direction and the vertical direction, image shift detection can be performed in the horizontal direction and the vertical direction.

  As described above, in a square pixel array in which pixels are densely arranged in the horizontal direction and the vertical direction, the horizontal direction (row direction = direction in which adjacent pixels are arranged in the horizontal direction) and the vertical direction (column direction = vertical direction). The pixels of the same type are not continuous in the direction in which the adjacent pixels are arranged) and in the oblique 45 ° direction (upward to the left, upward to the right = the diagonal direction of the adjacent pixels). Further, the first type pixel 311 and the second type pixel 312 or the third type pixel 316 and the fourth type pixel at the same ratio (that is, 1: 1) in the horizontal direction, the vertical direction, and the oblique 45 degree direction. 317 appears. Further, the first type pixel 311, the second type pixel 312, the third type pixel 316, and the fourth type pixel 317 have the same ratio (that is, 1: 1: 1: 1) in the vertical direction and the oblique 45 degree direction. Appears. Thereby, zebra pattern imaging when the line pattern is blurred can be prevented.

  In the pixel array shown in FIG. 16, since two types of pixels appear in the same direction in the vertical and horizontal diagonal directions, when the luminous flux received by the pixels arranged in the same direction is averaged, the average centroid of the luminous flux is plural. The center of the exit pupil 90 (intersection with the optical axis) coincides with the center of gravity of the luminous flux obtained by adding the luminous flux passing through the distance measuring pupil. As a result, even when pixel data is used as image data, the image in a specific direction is not formed by a light beam that passes through a biased region in the exit pupil, so image quality does not deteriorate.

  FIG. 17 shows another modification of the pixel array of the image sensor 211. The first type pixel 314 includes the microlens 10 and the photoelectric conversion unit 14. The photoelectric conversion unit 14 has a rectangular shape, and one long side of the photoelectric conversion unit 14 is close to a bisector of the microlens 10 along the diagonally right upward direction of 45 degrees in the drawing. The second type pixel 315 includes the microlens 10 and the photoelectric conversion unit 15. The photoelectric conversion unit 15 has a rectangular shape, and one long side of the photoelectric conversion unit 15 is close to the bisector of the microlens 10 at an angle of 45 degrees upward to the right. The photoelectric conversion units 14 and 15 are arranged in a diagonally upward 45 degree leftward direction when the microlens 10 is superimposed and displayed, and have a symmetrical shape with respect to the bisector of the microlens 10 obliquely upward 45 degree to the right. Yes. FIG. 10 shows distance measuring pupils 96 and 97 corresponding to the photoelectric conversion units 14 and 15. The first type pixel 314 receives the light beam coming from the distance measuring pupil 96, and the second type pixel 315 receives the light beam coming from the distance measuring pupil 97. The distance measuring pupils 96 and 97 are arranged symmetrically with respect to a 45 degree straight line passing through the origin.

  The third type pixel 318 includes the microlens 10 and the photoelectric conversion unit 18. The photoelectric conversion unit 18 has a rectangular shape, and one long side of the photoelectric conversion unit 18 is close to the bisector of the microlens 10 along the diagonally 45 degree upward direction in the figure. The fourth type pixel 319 includes the microlens 10 and the photoelectric conversion unit 19. The photoelectric conversion unit 19 has a rectangular shape, and one long side of the photoelectric conversion unit 19 is close to the bisector of the microlens 10 that is inclined upward 45 degrees to the left. The photoelectric conversion units 18 and 19 are arranged in a 45-degree obliquely upward right direction when the microlenses 10 are displayed in a superimposed manner, and have a symmetrical shape with respect to the bisector of the microlens 10 obliquely upward 45 degrees to the left. Yes. FIG. 11 shows distance measuring pupils 98 and 99 corresponding to the photoelectric conversion units 18 and 19. The third type pixel 318 receives the light beam coming from the distance measuring pupil 98, and the fourth type pixel 319 receives the light beam coming from the distance measuring pupil 99. The distance measuring pupils 98 and 99 are arranged symmetrically with respect to a 45 ° straight line passing through the origin.

  As shown in FIG. 17, the first type pixel, the second type pixel, the third type pixel, and the fourth type pixel are arranged in 8 pixels × 8 pixels, and the layout of the 8 pixels × 8 pixels is arranged. Spread the pattern in two dimensions. As described above, in the square pixel array in which the pixels are densely arranged in the horizontal direction and the vertical direction, it is possible to detect the image shift in the 45 ° upward and 45 ° diagonal directions.

  As described above, in a square pixel array in which pixels are densely arranged in the horizontal direction and the vertical direction, the horizontal direction (row direction = direction in which adjacent pixels are arranged in the horizontal direction) and the vertical direction (column direction = vertical direction). The pixels of the same type are not continuous in the direction in which the adjacent pixels are arranged) and in the oblique 45 ° direction (upward to the left, upward to the right = the diagonal direction of the adjacent pixels). Further, the first type pixel 314, the second type pixel 315, the third type pixel 318, and the fourth type at the same ratio (that is, 1: 1: 1: 1) in the horizontal direction, the vertical direction, and the oblique 45 degree direction. A type of pixel 319 appears. Thereby, zebra pattern imaging when the line pattern is blurred can be prevented.

  In the pixel array shown in FIG. 17, since two types of pixels appear in the same direction in the vertical and horizontal diagonal directions, when the luminous fluxes received by the pixels arranged in the same direction are averaged, the average centroid of the luminous flux is plural. The center of the exit pupil 90 (intersection with the optical axis) coincides with the center of gravity of the luminous flux obtained by adding the luminous flux passing through the distance measuring pupil. As a result, even when pixel data is used as image data, the image in a specific direction is not formed by a light beam that passes through a biased region in the exit pupil, so image quality does not deteriorate.

  FIG. 18 shows another modification of the pixel array of the image sensor 211. In the above-described imaging device, the example in which the photoelectric conversion units of the respective pixels have the same spectral sensitivity characteristic has been described, but the present invention can be applied even when each pixel has a photoelectric conversion unit of different spectral sensitivity characteristics. In the pixel array shown in FIG. 18, the photoelectric conversion units 21, 22, 23, and 24 marked G have the green (G) spectral sensitivity characteristics shown in FIG. 19, and the photoelectric conversion units 31 and 32 marked R are The photoelectric conversion units 41 and 42 having red (R) spectral sensitivity characteristics shown in FIG. 19 and B are shown in FIG. 19 have green (B) spectral sensitivity characteristics.

  The first type pixel 321 includes the microlens 10 and the photoelectric conversion unit 21. The photoelectric conversion unit 21 is rectangular, and one long side thereof is close to the vertical bisector of the microlens 10 in parallel. The second type pixel 322 includes the microlens 10 and the photoelectric conversion unit 22. The photoelectric conversion unit 22 is rectangular, and one long side thereof is close to the vertical bisector of the microlens 10 in parallel. The photoelectric conversion units 21 and 22 are arranged in the horizontal direction when the microlenses 10 are displayed in a superimposed manner, and have a symmetrical shape with respect to the vertical bisector of the microlens 10. The distance measuring pupils 92 and 93 corresponding to the photoelectric conversion units 21 and 22 are shown in FIG. The first type pixel 321 receives the light beam coming from the distance measuring pupil 92, and the second type pixel 322 receives the light beam coming from the distance measuring pupil 93. The distance measuring pupils 92 and 93 are arranged symmetrically with respect to the axis y.

  The third type pixel 323 includes the microlens 10 and the photoelectric conversion unit 23. The photoelectric conversion unit 23 has a rectangular shape, and one long side thereof is close to the horizontal bisector of the microlens 10 in parallel. The fourth type pixel 324 includes the microlens 10 and the photoelectric conversion unit 24. The photoelectric conversion unit 24 is rectangular, and one long side thereof is close to the horizontal bisector of the microlens 10 in parallel. The photoelectric conversion units 23 and 24 are arranged vertically in the vertical direction when the microlenses 10 are superimposed and displayed, and have a symmetrical shape with respect to the horizontal bisector of the microlens 10. FIG. 9 shows distance measuring pupils 94 and 95 corresponding to the photoelectric conversion units 23 and 24. The third type pixel 323 receives the light beam coming from the distance measuring pupil 94, and the fourth type pixel 324 receives the light beam coming from the distance measuring pupil 95. The distance measuring pupil 94 and the distance measuring pupil 95 are arranged symmetrically with respect to the axis x.

  The fifth type pixel 331 includes the microlens 10 and the photoelectric conversion unit 31. The photoelectric conversion unit 31 has a rectangular shape, and one long side of the photoelectric conversion unit 31 is close to the horizontal bisector of the microlens 10 in parallel with the upper side. The sixth type pixel 332 includes the microlens 10 and the photoelectric conversion unit 32. The photoelectric conversion unit 32 is rectangular, and one long side thereof is close to the horizontal bisector of the microlens 10 in parallel. The photoelectric conversion units 31 and 32 are arranged vertically in the vertical direction when the microlenses 10 are superimposed and displayed, and have a symmetrical shape with respect to the horizontal bisector of the microlens 10. FIG. 9 shows distance measuring pupils 94 and 95 corresponding to the photoelectric conversion units 31 and 32. The fifth type pixel 331 receives the light beam coming from the distance measuring pupil 94, and the sixth type pixel 332 receives the light beam coming from the distance measuring pupil 95. The distance measuring pupil 94 and the distance measuring pupil 95 are arranged symmetrically with respect to the axis x.

  The seventh type of pixel 341 includes the microlens 10 and the photoelectric conversion unit 41. The photoelectric conversion unit 41 is rectangular, and one long side thereof is close to the vertical bisector of the microlens 10 in parallel. The eighth type pixel 342 includes the microlens 10 and the photoelectric conversion unit 42. The photoelectric conversion unit 42 is rectangular, and one long side thereof is close to the vertical bisector of the microlens 10 in parallel. The photoelectric conversion units 41 and 42 are aligned in the horizontal direction when the microlenses 10 are displayed in an overlapping manner, and have a symmetrical shape with respect to the vertical bisector of the microlens 10. The distance measuring pupils 92 and 93 corresponding to the photoelectric conversion units 41 and 42 are shown in FIG. The seventh type pixel 341 receives the light beam coming from the distance measuring pupil 92, and the eighth type pixel 342 receives the light beam coming from the distance measuring pupil 93. The distance measuring pupil 92 and the distance measuring pupil 93 are arranged symmetrically with respect to the axis y.

  As shown in FIG. 18, the first type of pixels to the eighth type of pixels are arranged in 8 pixels × 8 pixels, and the layout pattern of 8 pixels × 8 pixels is spread two-dimensionally. As described above, in a square pixel array in which pixels are densely arranged in the horizontal direction and the vertical direction, it is possible to detect the image misalignment between green and blue in the horizontal direction and the image misalignment between green and red in the vertical direction. Detection is possible.

  As described above, in the square pixel array in which the pixels are densely arranged in the horizontal direction and the vertical direction, the horizontal direction (row direction = the direction in which adjacent pixels are arranged in the horizontal direction) and the vertical direction (column direction = vertical) for each color. The pixels of the same type are not continuous in the direction in which adjacent pixels are aligned in the direction) and in the oblique 45 degree direction (upward to the left, upward to the right = the diagonal direction of the adjacent pixels). In addition, each type of pixel appears at the same rate in the horizontal direction, the vertical direction, and the 45 ° oblique direction. Thereby, zebra pattern imaging when the line pattern is blurred can be prevented.

  In the pixel array shown in FIG. 18, since each type of pixel appears in the same direction in the vertical and horizontal diagonal directions, when the light beams received by the pixels arranged in the same direction are averaged, the center of gravity of the average light flux is measured by a plurality of measurement points. It coincides with the center of gravity of the luminous flux obtained by adding the luminous fluxes passing through the distance pupil, and becomes the center of the exit pupil 90 (intersection with the optical axis). As a result, even when pixel data is used as image data, the image in a specific direction is not formed by a light beam that passes through a biased region in the exit pupil, so image quality does not deteriorate.

  Since the pixel array shown in FIG. 18 is a Bayer array, normal image processing can be directly applied to the pixel data. In order to vary the spectral sensitivity characteristics of the photoelectric conversion unit, a color separation filter is disposed in the optical path from the microlens to the photoelectric conversion unit.

  Further, in the pixel array shown in FIG. 18, defocus amounts for a plurality of colors are obtained at one focus detection position. As a method for finally determining one defocus amount, there are the following methods. (1) An average of two defocus amounts. (2) Prioritize the defocus amount of one color. For example, priority is given to a green defocus amount having high specific visibility. (3) By selecting a defocus amount of a color having a high average value of data, it is possible to perform focus detection with a high SN ratio and high accuracy. (4) A defocus amount with higher reliability is selected based on the reliability determination described above. (5) A priority according to the focus detection direction is provided.

  FIG. 20 shows another modification of the pixel array of the image sensor 211. In the pixel array shown in FIG. 20, the photoelectric conversion units 25, 26, 27, and 28 marked G have the green (G) spectral sensitivity characteristics shown in FIG. 19, and the photoelectric conversion units 31 and 32 marked R are The photoelectric conversion units 41 and 42 having red (R) spectral sensitivity characteristics shown in FIG. 19 and B are shown in FIG. 19 have green (B) spectral sensitivity characteristics.

  The first type pixel 325 includes the microlens 10 and the photoelectric conversion unit 25. The photoelectric conversion unit 25 has a rectangular shape, and one long side of the photoelectric conversion unit 25 is close to the bisector of the microlens 10 at an angle of 45 degrees to the right. The second type pixel 326 includes the microlens 10 and the photoelectric conversion unit 26. The photoelectric conversion unit 26 has a rectangular shape, and one long side of the photoelectric conversion unit 26 is close to the bisector of the microlens 10 at an angle of 45 degrees to the right. The photoelectric conversion units 25 and 26 are arranged in a diagonally upward 45 degree leftward direction when the microlens 10 is superimposed and displayed, and have a symmetrical shape with respect to the bisector of the microlens 10 obliquely upward 45 degree to the right. Yes. FIG. 10 shows distance measuring pupils 96 and 97 corresponding to the photoelectric conversion units 25 and 26. The first type pixel 325 receives the light beam coming from the distance measuring pupil 96, and the second type pixel 326 receives the light beam coming from the distance measuring pupil 97. The distance measuring pupil 96 and the distance measuring pupil 97 are arranged symmetrically with respect to a 45 degree straight line passing through the origin of the x and y axes.

  The third type pixel 327 includes the microlens 10 and the photoelectric conversion unit 27. The photoelectric conversion unit 27 has a rectangular shape, and one long side of the photoelectric conversion unit 27 is close to the bisector of the microlens 10 that is inclined upward 45 degrees to the left. The fourth type pixel 328 includes the microlens 10 and the photoelectric conversion unit 28. The photoelectric conversion unit 28 has a rectangular shape, and one long side of the photoelectric conversion unit 28 is close to the bisector of the microlens 10 that is inclined upward 45 degrees to the left. The photoelectric conversion units 27 and 28 are arranged in a 45-degree obliquely upward direction when the microlens 10 is superimposed and displayed, and have a symmetrical shape with respect to the bisector of the microlens 10 obliquely upward 45 degree to the left. Yes. FIG. 11 shows distance measuring pupils 98 and 99 corresponding to the photoelectric conversion units 27 and 28. The third type pixel 327 receives the light beam coming from the distance measuring pupil 98, and the fourth type pixel 328 receives the light beam coming from the distance measuring pupil 99. The distance measuring pupil 98 and the distance measuring pupil 99 are arranged line-symmetrically with respect to a 45-degree straight line passing through the origin.

  The fifth type pixel 331 includes the microlens 10 and the photoelectric conversion unit 31. The photoelectric conversion unit 31 has a rectangular shape, and one long side thereof is close to the horizontal bisector of the microlens 10 in parallel. The sixth type pixel 332 includes the microlens 10 and the photoelectric conversion unit 32. The photoelectric conversion unit 32 has a rectangular shape, and one long side thereof has a rectangular shape in which the lower side is substantially in contact with the horizontal bisector of the microlens 10. The photoelectric conversion units 31 and 32 are arranged vertically in the vertical direction when the microlenses 10 are superimposed and displayed, and have a symmetrical shape with respect to the horizontal bisector of the microlens 10. FIG. 9 shows distance measuring pupils 94 and 95 corresponding to the photoelectric conversion units 31 and 32. The fifth type pixel 331 receives the light beam coming from the distance measuring pupil 94, and the sixth type pixel 332 receives the light beam coming from the distance measuring pupil 95. The distance measuring pupil 94 and the distance measuring pupil 95 are arranged symmetrically with respect to the axis x.

  The seventh type of pixel 341 includes the microlens 10 and the photoelectric conversion unit 41. The photoelectric conversion unit 41 is rectangular, and one long side thereof is close to the vertical bisector of the microlens 10 in parallel. The eighth type pixel 342 includes the microlens 10 and the photoelectric conversion unit 42. The photoelectric conversion unit 42 is rectangular, and one long side thereof is close to the vertical bisector of the microlens 10 in parallel. The photoelectric conversion units 41 and 42 are aligned in the horizontal direction when the microlenses 10 are displayed in an overlapping manner, and have a symmetrical shape with respect to the vertical bisector of the microlens 10. The distance measuring pupils 92 and 93 corresponding to the photoelectric conversion units 41 and 42 are shown in FIG. The seventh type pixel 341 receives the light beam coming from the distance measuring pupil 92, and the eighth type pixel 342 receives the light beam coming from the distance measuring pupil 93. The distance measuring pupil 92 and the distance measuring pupil 93 are arranged symmetrically with respect to the axis y.

  As shown in FIG. 20, the 8th type pixels from the 1st type pixels are arranged in 8 pixels × 8 pixels, and the layout pattern of 8 pixels × 8 pixels is spread two-dimensionally. As described above, in the square pixel array in which the pixels are densely arranged in the horizontal direction and the vertical direction, it is possible to detect a green image shift in an oblique direction (upward to the right and upward to the left), and a blue image in the horizontal direction. Deviation detection is possible, and red image deviation detection is possible in the vertical direction.

  As described above, in the square pixel array in which the pixels are densely arranged in the horizontal direction and the vertical direction, the horizontal direction (row direction = the direction in which adjacent pixels are arranged in the horizontal direction) and the vertical direction (column direction = vertical) for each color. The pixels of the same type are not continuous in the direction in which adjacent pixels are aligned in the direction) and in the oblique 45 degree direction (upward to the left, upward to the right = the diagonal direction of the adjacent pixels). In addition, each type of pixel appears at the same rate in the horizontal direction, the vertical direction, and the 45 ° oblique direction. Thereby, zebra pattern imaging when the line pattern is blurred can be prevented.

  In the pixel arrangement shown in FIG. 20, since each type of pixel appears in the same direction in the vertical and horizontal diagonal directions, when the luminous flux received by the pixels arranged in the same direction is averaged, the average centroid of the luminous flux is plural. The center of the exit pupil 90 (intersection with the optical axis) coincides with the center of gravity of the luminous flux obtained by adding the luminous flux passing through the distance measuring pupil. As a result, even when pixel data is used as image data, the image in a specific direction is not formed by a light beam that passes through a biased region in the exit pupil, so image quality does not deteriorate.

  Further, since the pixel array shown in FIG. 20 is a Bayer array, normal image processing can be applied to the pixel data as it is.

  FIG. 21 shows another modification of the pixel array of the image sensor 211. This imaging element is obtained by adding four types of pixels to the six types of pixels shown in FIG. The ninth type of pixel 333 includes the microlens 10 and the photoelectric conversion unit 33. The photoelectric conversion unit 33 is rectangular, and one long side thereof is close to the vertical bisector of the microlens 10 in parallel. The tenth type pixel 334 includes the microlens 10 and the photoelectric conversion unit 34. The photoelectric conversion unit 34 has a rectangular shape, and one long side thereof is close to the vertical bisector of the microlens 10 in parallel. The photoelectric conversion units 33 and 34 are arranged in the horizontal direction when the microlenses 10 are displayed in a superimposed manner, and have a symmetrical shape with respect to the vertical bisector of the microlens 10. The distance measuring pupils 92 and 93 corresponding to the photoelectric conversion units 33 and 34 are shown in FIG. The ninth type pixel 333 receives the light beam coming from the distance measuring pupil 92, and the tenth type pixel 334 receives the light beam coming from the distance measuring pupil 93. The distance measuring pupil 92 and the distance measuring pupil 93 are arranged symmetrically with respect to the axis y.

  The eleventh type of pixel 343 includes the microlens 10 and the photoelectric conversion unit 43. The photoelectric conversion unit 43 is rectangular, and one long side thereof is close to the horizontal bisector of the microlens 10 in parallel. The twelfth type of pixel 344 includes the microlens 10 and the photoelectric conversion unit 44. The photoelectric conversion unit 44 is rectangular, and one long side thereof is close to the horizontal bisector of the microlens 10 in parallel. The photoelectric conversion units 43 and 44 are arranged vertically in the vertical direction when the microlenses 10 are superimposed and displayed, and have a symmetrical shape with respect to the horizontal bisector of the microlens 10. FIG. 9 shows distance measuring pupils 94 and 95 corresponding to the photoelectric conversion units 43 and 44. The eleventh type pixel 343 receives the light beam coming from the distance measuring pupil 94, and the twelfth type pixel 344 receives the light beam coming from the distance measuring pupil 95. The distance measuring pupil 94 and the distance measuring pupil 95 are arranged symmetrically with respect to the axis x.

  As shown in FIG. 21, the 12th type pixels from the 1st type pixels are arranged in 8 pixels × 8 pixels, and the layout pattern of 8 pixels × 8 pixels is spread in two dimensions. In this way, in a square pixel array in which pixels are densely arranged in the horizontal direction and the vertical direction, it is possible to detect a green image shift in an oblique direction (upward to the right, upward to the left), and in the horizontal and vertical directions. Can detect image misalignment between blue and red.

  As described above, in the square pixel array in which the pixels are densely arranged in the horizontal direction and the vertical direction, the horizontal direction (row direction = the direction in which adjacent pixels are arranged in the horizontal direction) and the vertical direction (column direction = vertical) for each color. The pixels of the same type are not continuous in the direction in which adjacent pixels are aligned in the direction) and in the oblique 45 degree direction (upward to the left, upward to the right = the diagonal direction of the adjacent pixels). In addition, each type of pixel appears at the same rate in the horizontal direction, the vertical direction, and the 45 ° oblique direction. Thereby, zebra pattern imaging when the line pattern is blurred can be prevented.

  In the pixel array shown in FIG. 21, since each type of pixel appears in the same direction in the vertical and horizontal diagonal directions, when the luminous flux received by the pixels arranged in the same direction is averaged, the average centroid of the luminous flux is plural. The center of the exit pupil 90 (intersection with the optical axis) coincides with the center of gravity of the luminous flux obtained by adding the luminous flux passing through the distance measuring pupil. As a result, even when pixel data is used as image data, the image in a specific direction is not formed by a light beam that passes through a biased region in the exit pupil, so image quality does not deteriorate.

  Further, since the pixel arrangement shown in FIG. 21 is a Bayer arrangement, normal image processing can be applied to the pixel data as it is.

  FIG. 22 shows another modification of the pixel array of the image sensor 211. In the image sensor described above, an example in which each pixel is configured by one photoelectric conversion unit and one microlens is shown. However, the present invention can be applied even when each pixel has different spectral sensitivity characteristics from one microlens. It is. In the pixel array shown in FIG. 22, photoelectric conversion units 51, 52, 53, and 54 marked with G have the green (G) spectral sensitivity characteristics shown in FIG. 19, and photoelectric conversion units 63 and 64 marked with R are The photoelectric conversion units 71 and 72 having red (R) spectral sensitivity characteristics shown in FIG. 19 and B are shown in FIG. 19 have green (B) spectral sensitivity characteristics. The first type pixel 421 includes the microlens 10 and the photoelectric conversion units 51 and 71. The photoelectric conversion unit 51 is rectangular, and one long side thereof is close to the vertical bisector of the microlens 10 in parallel. The photoelectric conversion unit 71 is rectangular, and one long side thereof is close to the vertical bisector of the microlens 10 in parallel.

  FIG. 23 is a cross-sectional view of the pixel 421. In the pixel 421, the common microlens 10 is disposed in front of the photoelectric conversion units 51 and 71, and the photoelectric conversion units 51 and 71 are projected forward by the microlens 10. A green color filter 401 and a red color filter 402 are disposed between the microlens and the photoelectric conversion unit. The photoelectric conversion units 51 and 71 are formed on the semiconductor circuit substrate 29, and the color filters 401 and 401 and the microlens 10 are integrally and fixedly formed thereon by the manufacturing process of the semiconductor image sensor. The structures of a pixel 422, a pixel 413, and a pixel 414, which will be described later, are similar to the structure shown in FIG.

  The second type pixel 422 includes the microlens 10 and the photoelectric conversion units 52 and 72. The photoelectric conversion unit 52 is rectangular, and one long side thereof is close to the vertical bisector of the microlens 10 in parallel. The photoelectric conversion unit 72 is rectangular, and one long side thereof is close to the vertical bisector of the microlens 10 in parallel. The photoelectric conversion units 51 and 52 are arranged in the left-right horizontal direction when the microlenses 10 are superimposed and displayed, and have a symmetrical shape with respect to the vertical bisector of the microlens 10. The photoelectric conversion units 71 and 72 are aligned in the horizontal direction when the microlenses 10 are displayed in a superimposed manner, and have a symmetrical shape with respect to the vertical bisector of the microlens 10.

  FIG. 8 shows distance measuring pupils 92 and 93 corresponding to the photoelectric conversion units 51, 52, 71 and 72. The photoelectric conversion units 51 and 72 receive the light beam coming from the distance measuring pupil 92, and the photoelectric conversion units 52 and 71 receive the light beam coming from the distance measurement pupil 93. The distance measuring pupil 92 and the distance measuring pupil 93 are arranged symmetrically with respect to the axis y.

  The third type pixel 413 includes the microlens 10 and the photoelectric conversion units 53 and 63. The photoelectric conversion unit 53 is rectangular, and one long side thereof is close to the horizontal bisector of the microlens 10 in parallel. The photoelectric conversion unit 63 has a rectangular shape, and one long side thereof is close to the horizontal bisector of the microlens 10 in parallel. The fourth type pixel 414 includes the microlens 10 and the photoelectric conversion units 54 and 64. The photoelectric conversion unit 54 is rectangular, and one long side thereof is close to the horizontal bisector of the microlens 10 in parallel. The photoelectric conversion unit 64 is rectangular, and one long side thereof is close to the horizontal bisector of the microlens 10 in parallel.

  The photoelectric conversion units 53 and 54 are arranged vertically in the vertical direction when the microlenses 10 are superimposed and displayed, and have a symmetrical shape with respect to the horizontal bisector of the microlens 10. The photoelectric conversion units 63 and 64 are arranged vertically in the vertical direction when the microlenses 10 are superimposed and displayed, and have a symmetrical shape with respect to the horizontal bisector of the microlens 10. FIG. 9 shows distance measuring pupils 94 and 95 corresponding to the photoelectric conversion units 53, 54, 63 and 64. The photoelectric conversion units 53 and 64 receive the light beam coming from the distance measuring pupil 94, and the photoelectric conversion units 54 and 63 receive the light beam coming from the distance measuring pupil 95. The distance measuring pupil 94 and the distance measuring pupil 95 are arranged symmetrically with respect to the axis x.

  As shown in FIG. 22, the first type of pixels to the fourth type of pixels are arranged in 8 pixels × 8 pixels, and the layout pattern of 8 pixels × 8 pixels is spread in a two-dimensional manner. As described above, in a square pixel array in which pixels are densely arranged in the horizontal direction and the vertical direction, it is possible to detect the image misalignment between green and blue in the horizontal direction and the image misalignment between green and red in the vertical direction. Detection is possible.

  As described above, in the square pixel array in which the pixels are densely arranged in the horizontal direction and the vertical direction, the horizontal direction (row direction = the direction in which adjacent pixels are arranged in the horizontal direction) and the vertical direction (column direction = vertical) for each color. The pixels of the same type are not continuous in the direction in which adjacent pixels are aligned in the direction) and in the oblique 45 degree direction (upward to the left, upward to the right = the diagonal direction of the adjacent pixels). In addition, each type of pixel appears at the same rate in the horizontal direction, the vertical direction, and the 45 ° oblique direction. Thereby, zebra pattern imaging when the line pattern is blurred can be prevented.

  In the pixel array shown in FIG. 22, since each type of pixel appears in the same direction in the vertical and horizontal diagonal directions, when the luminous flux received by the pixels arranged in the same direction is averaged for each color, the center of gravity of the average luminous flux Coincides with the center of gravity of the light beam obtained by adding the light beams passing through the plurality of distance measurement pupils, and becomes the center of the exit pupil 90 (intersection with the optical axis). As a result, even when pixel data is used as image data, the image in a specific direction is not formed by a light beam that passes through a biased region in the exit pupil, so image quality does not deteriorate.

  The pixel array shown in FIG. 22 has a configuration with higher color detection density than the Bayer array (green is detected in all pixels, blue and red are detected in a ratio of one pixel to two pixels), and color reproducibility And the image quality is improved.

  FIG. 24 shows another modification of the pixel array of the image sensor 211. In the above-described embodiment, an example in which each pixel of the image sensor is configured by one photoelectric conversion unit and one microlens is shown, but each pixel has a spectral sensitivity characteristic different from that of one microlens. However, the present invention is applicable.

  The first type pixel 421 includes the microlens 10 and photoelectric conversion units 55 and 65. The photoelectric conversion unit 55 is rectangular, and one long side thereof is close to the vertical bisector of the microlens 10 in parallel. The photoelectric conversion unit 65 is rectangular, and one long side thereof is close to the vertical bisector of the microlens 10 in parallel. The second type pixel 416 includes the microlens 10 and the photoelectric conversion units 56 and 66. The photoelectric conversion unit 56 is rectangular, and one long side thereof is close to the vertical bisector of the microlens 10 in parallel. The photoelectric conversion unit 66 is rectangular, and one long side thereof is close to the vertical bisector of the microlens 10 in parallel.

  The photoelectric conversion units 55 and 56 are arranged in the horizontal direction when the microlenses 10 are displayed in a superimposed manner, and have a symmetrical shape with respect to the vertical bisector of the microlens 10. The photoelectric conversion units 65 and 66 are arranged in the horizontal direction when the microlenses 10 are displayed in a superimposed manner, and have a symmetrical shape with respect to the vertical bisector of the microlens 10. FIG. 8 shows distance measuring pupils 92 and 93 corresponding to the photoelectric conversion units 55, 56, 65 and 66. The photoelectric conversion units 55 and 66 receive the light beam coming from the distance measuring pupil 92, and the photoelectric conversion units 56 and 65 receive the light beam coming from the distance measurement pupil 93. The distance measuring pupil 92 and the distance measuring pupil 93 are arranged symmetrically with respect to the axis y.

  The third type pixel 413 includes the microlens 10 and the photoelectric conversion units 53 and 63. The photoelectric conversion unit 53 is rectangular, and one long side thereof is close to the horizontal bisector of the microlens 10 in parallel. The photoelectric conversion unit 63 has a rectangular shape, and one long side thereof is close to the horizontal bisector of the microlens 10 in parallel. The fourth type pixel 414 includes the microlens 10 and the photoelectric conversion units 54 and 64. The photoelectric conversion unit 54 is rectangular, and one long side thereof is close to the horizontal bisector of the microlens 10 in parallel. The photoelectric conversion unit 64 is rectangular, and one long side thereof is close to the horizontal bisector of the microlens 10 in parallel.

  The photoelectric conversion units 53 and 54 are arranged vertically in the vertical direction when the microlenses 10 are superimposed and displayed, and have a symmetrical shape with respect to the horizontal bisector of the microlens 10. The photoelectric conversion units 63 and 64 are arranged vertically in the vertical direction when the microlenses 10 are superimposed and displayed, and have a symmetrical shape with respect to the horizontal bisector of the microlens 10. FIG. 9 shows distance measuring pupils 94 and 95 corresponding to the photoelectric conversion units 53, 54, 63 and 64. The photoelectric conversion units 53 and 64 receive the light beam coming from the distance measuring pupil 94, and the photoelectric conversion units 54 and 63 receive the light beam coming from the distance measuring pupil 95. The distance measuring pupil 94 and the distance measuring pupil 95 are arranged symmetrically with respect to the axis x.

  The fifth type pixel 425 includes the microlens 10 and photoelectric conversion units 85 and 75. The photoelectric conversion unit 85 has a rectangular shape, and one long side of the photoelectric conversion unit 85 is close to the bisector of the microlens 10 at an angle of 45 degrees to the right. The photoelectric conversion unit 75 has a rectangular shape, and one long side of the photoelectric conversion unit 75 is close to the bisector of the microlens 10 at an angle of 45 degrees to the right. The sixth type pixel 426 includes the microlens 10 and the photoelectric conversion units 86 and 76. The photoelectric conversion unit 86 has a rectangular shape, and one long side of the photoelectric conversion unit 86 is close to the bisector of the microlens 10 at an angle of 45 degrees to the right. The photoelectric conversion unit 76 has a rectangular shape, and one long side of the photoelectric conversion unit 76 is close to the bisector of the microlens 10 diagonally upward 45 degrees to the right.

  The photoelectric conversion units 85 and 86 are arranged in a 45-degree obliquely left upward direction when the microlens 10 is superimposed and displayed, and have a symmetrical shape with respect to the bisector of the microlens 10 obliquely 45-degree upward right. Yes. The photoelectric conversion units 75 and 76 are arranged in a diagonally upward 45 degree leftward direction when the microlens 10 is superimposed and displayed, and have a symmetrical shape with respect to the bisector of the microlens 10 obliquely upward 45 degree to the right. Yes. FIG. 10 shows distance measuring pupils 96 and 97 corresponding to the photoelectric conversion units 85, 86, 75 and 76. The photoelectric conversion units 85 and 76 receive the light beam coming from the distance measuring pupil 96, and the photoelectric conversion units 86 and 75 receive the light beam coming from the distance measuring pupil 97. The distance measuring pupils 96 and 97 are arranged symmetrically with respect to a 45 degree straight line passing through the origin.

  The seventh type pixel 423 includes the microlens 10 and photoelectric conversion units 83 and 73. The photoelectric conversion unit 83 is rectangular, and one long side of the photoelectric conversion unit 83 is close to the bisector of the microlens 10 that is inclined upward 45 degrees to the left. The photoelectric conversion unit 73 has a rectangular shape, and one long side of the photoelectric conversion unit 73 is close to the bisector of the microlens 10 that is inclined upward 45 degrees to the left. The eighth type pixel 424 includes the microlens 10 and the photoelectric conversion units 84 and 74. The photoelectric conversion unit 84 has a rectangular shape, and one long side of the photoelectric conversion unit 84 is close to the bisector of the microlens 10 that is inclined upward 45 degrees to the left. The photoelectric conversion unit 74 has a rectangular shape, and one long side of the photoelectric conversion unit 74 is close to the bisector of the microlens 10 obliquely upward 45 degrees to the left.

  The photoelectric conversion units 83 and 84 are arranged in a 45-degree obliquely upward direction when the microlenses 10 are displayed in an overlapping manner, and have a symmetrical shape with respect to the bisector of the microlens 10 obliquely upward 45 degrees to the left. Yes. The photoelectric conversion units 73 and 74 are arranged in a 45-degree obliquely upward direction when the microlenses 10 are displayed in a superimposed manner, and have a symmetrical shape with respect to the bisector of the microlens 10 obliquely upward 45 degrees to the left. Yes. FIG. 11 shows distance measuring pupils 98 and 99 corresponding to the photoelectric conversion units 83, 84, 73 and 74. The photoelectric conversion units 83 and 74 receive the light beam coming from the distance measuring pupil 98, and the photoelectric conversion units 84 and 73 receive the light beam coming from the distance measuring pupil 99. The distance measuring pupils 98 and 99 are arranged symmetrically with respect to a 45-degree straight line that passes through the origins of the x and y axes.

  As shown in FIG. 24, the first type of pixels to the eighth type of pixels are arranged in 8 pixels × 8 pixels, and the layout pattern of 8 pixels × 8 pixels is spread two-dimensionally. In this manner, in a square pixel array in which pixels are densely arranged in the horizontal direction and the vertical direction, it is possible to detect image misalignment between green and red in the horizontal direction and the vertical direction, and in an oblique direction (upward / leftward) In the (up)), it is possible to detect the image misalignment between green and blue.

  As described above, in the square pixel array in which the pixels are densely arranged in the horizontal direction and the vertical direction, the horizontal direction (row direction = the direction in which adjacent pixels are arranged in the horizontal direction) and the vertical direction (column direction = vertical) for each color. The pixels of the same type are not continuous in the direction in which adjacent pixels are aligned in the direction) and in the oblique 45 degree direction (upward to the left, upward to the right = the diagonal direction of the adjacent pixels). In addition, each type of pixel appears at the same rate in the horizontal direction, the vertical direction, and the 45 ° oblique direction. Thereby, zebra pattern imaging when the line pattern is blurred can be prevented.

  In the pixel arrangement shown in FIG. 24, since each type of pixel appears in the same direction in the vertical and horizontal diagonal directions, when the luminous flux received by the pixels arranged in the same direction is averaged for each color, the center of gravity of the average luminous flux Coincides with the center of gravity of the light beam obtained by adding the light beams passing through the plurality of distance measurement pupils, and becomes the center of the exit pupil 90 (intersection with the optical axis). As a result, even when pixel data is used as image data, the image in a specific direction is not formed by a light beam that passes through a biased region in the exit pupil, so image quality does not deteriorate.

  In addition, the pixel array shown in FIG. 24 has a configuration in which the color detection density is higher than that of the Bayer array (green is detected in all pixels, blue and red are detected in the ratio of one pixel to two pixels). Reproducibility is improved and image quality is improved.

  In addition to the pixel arrangement described above, many variations can be configured. In short, any pixel array that does not concentrate pixels that receive a light beam coming from a specific distance measuring pupil in a specific direction of one pixel array may be used.

  In the image pickup device of the image pickup apparatus shown in FIG. 1, pixels 311 and 312 having rectangular photoelectric conversion units are two-dimensionally arranged as shown in FIG. 3, but shown in FIGS. 25 (a) and 25 (b). In this way, the photoelectric conversion portion can be replaced with a semicircular pixel. The pixel 602 includes a microlens 10 and a semicircular photoelectric conversion unit 513, and the photoelectric conversion unit 513 is disposed on the left side with respect to a straight line passing through the center of the microlens 10. The pixel 601 includes a microlens 10 and a semicircular photoelectric conversion unit 512, and the photoelectric conversion unit 512 is arranged on the right side with respect to a straight line passing through the center of the microlens 10.

  22 and 24, an example in which two photoelectric conversion units having different spectral sensitivity characteristics are provided in one pixel has been described. However, three or more photoelectric conversion units having different spectral sensitivity characteristics are provided in one pixel. You may prepare.

  In the above-described image sensor of the embodiment and its modification, an example using filters of three primary colors of red, blue, and green has been shown. However, an image sensor of only two colors or four or more colors is detected. The present invention can also be applied to an image sensor provided with a filter that performs this. In the image pickup device according to the embodiment and the modification described above, an example in which the color separation filter is a primary color filter (RGB) has been described. However, complementary color filters (green: G, yellow: Ye, magenta: Mg, cyan) : Cy) may be employed. Further, color separation can be achieved by changing the spectral sensitivity characteristics of the photodiodes constituting the photoelectric conversion unit for each photoelectric conversion unit in addition to the color filter.

  A pupil division method using a polarization filter will be described with reference to FIG. The pupil division method is not limited to the microlens method. In the figure, reference numeral 690 denotes a polarizing filter holding frame, and portions other than the polarizing filter are shielded from light. Reference numeral 692 denotes a polarizing filter (which forms a distance measuring pupil based on the position and shape of the filter). Reference numeral 693 denotes a polarization filter, and the polarization direction of the polarization filter 692 is orthogonal (the distance measurement pupil is configured by the position and shape of the filter). Reference numeral 91 denotes an optical axis of the interchangeable lens. A polarization filter 621 has a polarization direction that matches that of the polarization filter 692. Further, reference numeral 622 denotes a polarizing filter whose polarization direction matches that of the polarizing filter 693. Reference numerals 611 and 612 denote photoelectric conversion units, 631 denotes a first type pixel, 632 denotes a second type pixel, and 672, 673, 682, and 683 denote light beams.

  FIG. 26 schematically illustrates four adjacent pixels. In the pixel 631, the photoelectric conversion unit 611 receives the light beam passing through the distance measuring pupil formed by the polarization filter 692 by the action of the polarization filter 621, and corresponds to the intensity of the image formed by the light beam 672 or 682. Output a signal. In the pixel 632, the photoelectric conversion unit 612 receives the light beam that passes through the distance measuring pupil formed by the polarization filter 693 by the action of the polarization filter 622, and changes the intensity of the image formed by the light beam 673 or 683. Output the corresponding signal.

  By arranging a large number of first-type pixels and second-type pixels using such a polarizing filter in a straight line, and collecting the output of the photoelectric conversion unit of each pixel into an output group corresponding to the distance measuring pupil, Information regarding the intensity distribution of a pair of images formed on the pixel array by the focus detection light beams that pass through the distance measuring pupils can be obtained. By applying an image shift detection calculation process (correlation calculation process, phase difference detection process), which will be described later, to this information, an image shift amount of a pair of images is detected by a so-called pupil division type phase difference detection method.

  Note that the imaging device according to the embodiment and the modification described above can be formed as a CCD image sensor or a CMOS image sensor.

  In the flowchart shown in FIG. 12, the example in which the image data is stored in the memory card 213 is shown. However, the image data may be displayed on an electronic viewfinder or a back monitor screen (not shown) provided on the back of the body. .

  In the image pickup apparatus shown in FIG. 1, the example in which the image pickup device 211 of the embodiment is used for generating image data has been shown. However, as shown in FIG. 27, the image pickup device 221 dedicated to image pickup is provided, The image sensor 211 may be used for focus detection and electronic viewfinder display.

  In FIG. 27, the same components as those in FIG. The camera body 203A is provided with a half mirror 222 for separating a photographing light beam, an imaging element 221 dedicated for imaging on the transmission side, and an imaging element 211 for focus detection and electronic viewfinder display on the reflection side. Before shooting, focus detection and electronic viewfinder display are performed according to the output of the image sensor 211. At the time of release, image data corresponding to the output of the imaging element 221 dedicated to imaging is generated. The half mirror 222 may be a total reflection mirror, and may be retracted from the photographing optical path during photographing. In this way, even if the pixel size of the image sensor 211 for focus detection and electronic viewfinder display is increased, the output is only used for electronic viewfinder display with low focus detection and resolution requirements, and image data There is no loss of resolution.

  Thus, according to one embodiment and its modification, in an imaging device in which a plurality of types of pixels that receive light beams that have passed through different regions of the exit pupil of the optical system are arranged two-dimensionally, Since each type of pixel is distributed and arranged so that the same type of pixel is not concentrated in an arbitrary direction, a high-quality captured image can be obtained while achieving the focus detection function.

<< Applicable scope of one embodiment >>
The imaging apparatus according to the embodiment is not limited to a digital still camera or a film still camera including a detachable interchangeable lens and a camera body. For example, the imaging apparatus can also be applied to a lens-integrated digital still camera, a video camera, or a film camera. can do. Further, the present invention can be applied to a small camera module or a surveillance camera built in a mobile phone or the like. Alternatively, the present invention can be applied to a focus detection device other than a camera, a distance measuring device, or a stereo distance measuring device.

The figure which shows the structure of the digital still camera of one embodiment The figure which shows schematic structure of an image sensor The figure which shows the detail of the pixel arrangement | sequence of an image pick-up element Sectional view of the first type of pixel Sectional view of the second type of pixel The figure explaining the focus detection method by the pupil division type phase difference detection method using a micro lens Diagram showing the projection relationship on the exit pupil plane The figure which shows the arrangement relation of the distance measuring pupil The figure which shows the arrangement relation of the distance measuring pupil The figure which shows the arrangement relation of the distance measuring pupil The figure which shows the arrangement relation of the distance measuring pupil The flowchart which shows the operation | movement of the digital still camera of one embodiment Partial enlarged view of an image sensor according to an embodiment Diagram explaining correlation calculation method The figure which shows the modification of an image pick-up element The figure which shows the other modification of an image pick-up element The figure which shows the other modification of an image pick-up element The figure which shows the other modification of an image pick-up element Diagram showing spectral sensitivity characteristics of pixels The figure which shows the other modification of an image pick-up element The figure which shows the other modification of an image pick-up element The figure which shows the other modification of an image pick-up element Cross section of pixel The figure which shows the other modification of an image pick-up element Diagram showing pixel structure The figure explaining the focus detection method of the pupil division system using a polarization filter Sectional view of a modified digital still camera The figure explaining the image formation state in a pupil division type phase difference detection system The figure which shows the conventional image sensor

Explanation of symbols

10 Microlenses 12-19, 21-28, 31-34, 41-44, 51-56, 63-66, 71-76, 83-86, 512, 513 Photoelectric converter 90 Exit pupil 92-99 Distance pupil 202 Interchangeable lens 211 Imaging elements 311, 312, 314 to 319, 321 to 324, 325 to 328, 331 to 334, 341 to 344, 413 to 416, 421 to 426, 601 and 602 pixels

Claims (5)

A plurality of first pixels that respectively receive one of the pair of light beams passing through a pair of regions of the exit pupil of the optical system; and a plurality of second pixels that respectively receive the other light beam of the pair of light beams; An imaging device in which the plurality of first pixels and the plurality of second pixels are two-dimensionally arranged,
Reading means for reading out the output of the first pixel and the output of the second pixel from the imaging device;
Image generation means for generating a captured image signal based on the output of the first pixel and the output of the second pixel read by the reading means;
A phase difference detection type focus detection unit that detects a focus adjustment state of the optical system based on the output of the first pixel and the output of the second pixel read by the reading unit;
On the image sensor, the first pixels and the second pixels are alternately arranged in a first direction in which the pair of regions are arranged, and adjacent to each other in a second direction perpendicular to the first direction. An imaging apparatus, wherein the same number of the first pixels and the second pixels exist within a pixel range. The imaging device according to claim 1,
The first pixels are arranged so that three or more pixels are not continuously arranged in any of the first direction, the second direction, and the 45-degree oblique direction of the first direction,
The second pixels are arranged so that three or more pixels are not continuously arranged in any of the first direction, the second direction, and the 45 ° oblique direction of the first direction. An imaging apparatus characterized by the above. The imaging device according to claim 1 or 2,
The image sensor is
A first array in which the first pixels and the second pixels are alternately arranged in a first arrangement pattern in the first direction;
A second array that is adjacent to and juxtaposed with the first array in the second direction, wherein the first pixels and the second pixels are alternately arrayed in the first array pattern in the first direction;
A second array pattern that is adjacent to the second array in the second direction and in which the first pixel and the second pixel are shifted by one pixel with respect to the first array pattern in the first direction; A third array arranged alternately with
A fourth array arranged adjacent to the third array in the second direction, wherein the first pixels and the second pixels are alternately arrayed in the second array pattern in the first direction;
A fifth array that is adjacent to and juxtaposed with the fourth array in the second direction, wherein the first pixels and the second pixels are alternately arrayed in the first array pattern in the first direction;
A sixth array arranged adjacent to the fifth array in the second direction, wherein the first pixels and the second pixels are alternately arranged in the first array pattern in the first direction;
A seventh array, which is adjacent to the sixth array in the second direction and in which the first pixels and the second pixels are alternately arrayed in the second array pattern in the first direction;
An eighth array that is adjacent to the seventh array in the second direction and in which the first pixels and the second pixels are alternately arrayed in the second array pattern in the first direction;
An imaging device comprising: In the imaging device according to any one of claims 1 to 3,
An imaging apparatus, further comprising display means for displaying a captured image based on the captured image signal of the image generation means. In the imaging device according to any one of claims 1 to 4,
The first pixel includes a first microlens and a first photoelectric conversion unit that receives the one light flux that has passed through the first microlens,
The first photoelectric conversion unit is arranged to be offset in the first direction with respect to the center of the first microlens,
The second pixel includes a second microlens and a second photoelectric conversion unit that receives the other light flux that has passed through the second microlens,
The imaging apparatus, wherein the second photoelectric conversion unit is arranged to be offset in a direction opposite to the first direction with respect to a center of the second microlens.

Images