JP2010181485A - Image-pickup device and imaging element - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To ensure focus detection performance of high precision by a focus-detecting pixel, while ensuring the high pixelation of an imaging element, reduction in chip size and improved image quality by pixel-size reduction in an imaging pixel. <P>SOLUTION: The pixel size of a focus-detecting pixel 311, which is arranged in an imaging element 212, is made equal to that of four imaging elements 310, to ensure focus detection performance. The focus-detecting pixel 311 is arranged obliquely in a 45° direction on the imaging element 212 to ensure many peripheral imaging pixels 212 which are used for complementing image data at the position of the focus-detecting pixel 311. Accordingly, deterioration in the image quality which is caused, accompanying the pixel size of the focus-detecting pixel 311 being made larger than that of the imaging pixel 310 is prevented. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging device and an imaging element.

  There is known an imaging apparatus that includes an imaging element having an array of a plurality of focus detection pixels and has a focus detection function of a so-called pupil division type phase difference detection method (see, for example, Patent Document 1).

JP 2007-188931 A

  In the image pickup device of such an image pickup apparatus, the focus detection pixel and the image pickup pixel are composed of a microlens and a photoelectric conversion unit formed behind the microlens. In the imaging device of the conventional imaging device described above, when the pixel size of the imaging pixel is reduced, the diffraction effect increases rapidly as the aperture diameter of the microlens approaches the wavelength order of visible light. The pupil division action by pixels becomes incomplete. For this reason, the imaging device has a problem that the focus detection performance deteriorates or the focus detection is impossible.

An image pickup apparatus according to a first aspect of the present invention includes a plurality of types of image pickup pixels that have different spectral sensitivity characteristics and generate an output signal corresponding to an image formed by a light beam passing through the optical system, and a pair that passes through the optical system. An imaging device in which a plurality of focus detection pixels that generate a pair of output signals corresponding to a pair of images formed by the light beam and a pair of output signals are used, and the focus adjustment state of the optical system based on the pair of output signals The focus detection means for detecting the output signal, and when a plurality of types of imaging pixels are arranged at the positions of the plurality of focus detection pixels, output signals generated by the plurality of types of imaging pixels are arranged around the plurality of focus detection pixels. Interpolating means for generating an interpolation value by interpolating based on output signals of a plurality of types of imaging pixels, the plurality of types of imaging pixels are arranged according to a predetermined arrangement rule, and each of the plurality of focus detection pixels is Multiple species Greater than each of the imaging pixels, on the image pickup device, characterized in that it is arranged according to contiguous sequences in a diagonal direction.
An image pickup device according to a twelfth aspect of the present invention includes a plurality of types of image pickup pixels having different spectral sensitivity characteristics and generating an output signal corresponding to an image formed by a light beam passing through an optical system, and an optical element for focus detection. A plurality of focus detection pixels that generate a pair of output signals corresponding to a pair of images formed by a pair of light beams passing through the system, and a plurality of types of imaging pixels and a plurality of focus detection pixels are arranged two-dimensionally The plurality of types of imaging pixels are arranged according to a predetermined arrangement rule, and each of the plurality of focus detection pixels is larger than each of the plurality of types of imaging pixels and is arranged according to an array that is continuous in an oblique direction. An image sensor.

  According to the present invention, there are provided an image pickup device that enables good image generation and good focus detection in the image pickup device, and an image pickup device that includes the image pickup device and performs good image generation and good focus detection. can do.

1 is a cross-sectional view illustrating a configuration of an interchangeable lens digital still camera according to an embodiment. The figure which shows the focus detection position on the imaging | photography screen of an interchangeable lens. The figure explaining the principle of a pupil division type phase difference detection system. The front view which shows the detailed structure of an image pick-up element. The figure which showed the positional relationship of a focus detection pixel and the imaging pixel (red pixel, green pixel, blue pixel) which should be arrange | positioned according to the arrangement | positioning rule of a Bayer arrangement in the position originally. The front view of an imaging pixel and a focus detection pixel. The figure which shows the spectral characteristic of a green pixel, a red pixel, and a blue pixel. The figure which shows the spectral characteristic of a focus detection pixel. Sectional drawing of an imaging pixel and a focus detection pixel. The figure for demonstrating the principle of the pupil division type phase difference detection system by a focus detection pixel. The figure for demonstrating the mode of the imaging light beam which an imaging pixel receives. The figure which shows the basic circuit structure of the photoelectric conversion part which an imaging pixel and a focus detection pixel have. The figure which showed the circuit wiring layout on the semiconductor substrate corresponding to the area | region of the imaging pixel of 6 rows 6 columns in an imaging device. The flowchart which shows the imaging operation of a digital still camera. The figure which shows the relationship of the correlation amount C (k) with respect to the shift amount k of a pair of data. The figure which shows the case where a focus detection pixel is arrange | positioned in the area | region of an imaging pixel of 6 rows and 6 columns at 45 degree | times diagonally upward. The flowchart which shows the interpolation process of the image data of the focus detection pixel position accompanying the imaging operation of a digital still camera. The front view which shows the detailed structure of an image pick-up element. The figure which shows the case where a focus detection pixel is arrange | positioned in the area | region of an imaging pixel of 6 rows and 6 columns at 45 degree | times diagonally upward. The front view which shows the detailed structure of an image pick-up element. The front view which shows the detailed structure of an image pick-up element. Sectional drawing of a focus detection pixel.

  An imaging apparatus and an imaging element according to an embodiment of the present invention will be described. FIG. 1 is a cross-sectional view showing the configuration of an interchangeable lens digital still camera according to an embodiment. A digital still camera 201 according to an embodiment includes an interchangeable lens 202 and a camera body 203, and various interchangeable lenses 202 are attached to the camera body 203 via a mount unit 204. An interchangeable lens 202 having various optical systems can be attached to the camera body 203 via a mount unit 204.

  The interchangeable lens 202 includes a lens 209, a zooming lens 208, a focusing lens 210, an aperture 211, a lens drive control device 206, and the like. The lens drive control device 206 includes a microcomputer (not shown), a memory, a drive control circuit, and the like. The lens drive control device 206 includes drive control for adjusting the focus of the focusing lens 210 and the aperture diameter of the aperture 211, zooming lens 208, and focusing. In addition to detecting the state of the lens 210 and the aperture 211, the lens information is transmitted and the camera information is received through communication with a body drive control device 214 described later. The aperture 211 forms an aperture having a variable aperture diameter at the center of the optical axis in order to adjust the amount of light and the amount of blur.

  The camera body 203 includes an imaging element 212, a body drive control device 214, a liquid crystal display element drive circuit 215, a liquid crystal display element 216, an eyepiece lens 217, a memory card 219, and the like. In the imaging element 212, imaging pixels are two-dimensionally arranged, and focus detection pixels are incorporated in portions corresponding to focus detection positions. Details of the image sensor 212 will be described later.

  The body drive control device 214 includes a microcomputer, a memory, a drive control circuit, and the like. The adjustment is repeated, and image signal processing and recording, camera operation control, and the like are performed. The body drive control device 214 communicates with the lens drive control device 206 via the electrical contact 213 to receive lens information and send camera information (defocus amount, aperture value, etc.).

  The liquid crystal display element 216 functions as an electric view finder (EVF). The liquid crystal display element driving circuit 215 displays a through image by the imaging element 212 on the liquid crystal display element 216, and the photographer can observe the through image through the eyepiece lens 217. The memory card 219 is an image storage that stores an image captured by the image sensor 212.

  A subject image is formed on the light receiving surface of the image sensor 212 by the light beam that has passed through the interchangeable lens 202. This subject image is photoelectrically converted by the image sensor 212, and an image signal and a focus detection signal are sent to the body drive control device 214.

  The body drive control device 214 calculates the defocus amount based on the focus detection signal from the focus detection pixel of the image sensor 212 and sends the defocus amount to the lens drive control device 206. The body drive control device 214 processes the image signal from the image sensor 212 to generate image data, stores the image data in the memory card 219, and transmits the through image signal from the image sensor 212 to the liquid crystal display element drive circuit 215. The through image is displayed on the liquid crystal display element 216. Further, the body drive control device 214 sends aperture control information to the lens drive control device 206 to control the aperture of the aperture 211.

  The lens drive controller 206 updates the lens information according to the focusing state, zooming state, aperture setting state, aperture opening F value, and the like. Specifically, the positions of the zooming lens 208 and the focusing lens 210 and the aperture value of the aperture 211 are detected, and lens information is calculated according to these lens positions and aperture values, or a lookup prepared in advance. Lens information corresponding to the lens position and aperture value is selected from the table.

  The lens drive control device 206 calculates a lens drive amount based on the received defocus amount, and drives the focusing lens 210 to the in-focus position according to the lens drive amount. Further, the lens drive control device 206 drives the diaphragm 211 in accordance with the received diaphragm value.

  FIG. 2 is a diagram showing a focus detection position on the photographing screen of the interchangeable lens 202, and a region (focus detection) in which a focus detection pixel array on the image sensor 212 described later samples an image on the photographing screen when focus detection is performed. An example of an area and a focus detection position is shown. In this example, focus detection areas 101 to 103 are arranged at the center and three locations above and below the rectangular shooting screen 100. The focus detection areas 101 to 103 indicated by rectangles extend obliquely upward in the shooting screen 100, and focus detection pixels are linearly arranged in the longitudinal direction of each focus detection area.

  Before describing the detailed configuration of the image sensor 212, the principle of the pupil division type phase difference detection method disclosed in Patent Document 1 will be described with reference to FIG.

  A plurality of focus detection pixels 111 are arranged on the imaging surface 110. The focus detection pixel 111 includes a microlens 112 and a pair of photoelectric conversion units 113 and 114. The pair of photoelectric conversion units 113 and 114 is projected by the microlens 112 onto the distance measuring pupil plane 120 at a distance d ahead of the imaging surface 110, thereby forming a pair of distance measuring pupils 123 and 124. In other words, the luminous flux of the distance measuring pupil 123 out of the light flux passing through the distance measuring pupil plane 120 at a distance d ahead of the imaging surface 110 is received by the photoelectric conversion unit 113 of the focus detection pixel 111 and is measured. Of the light beams passing through the pupil surface 120, the light beam of the distance measuring pupil 124 is received by the photoelectric conversion unit 114 of the focus detection pixel 111. The relative shift amount (phase difference, image shift amount) between the image signal of the photoelectric conversion unit 113 in the array of the focus detection pixels 111 and the image signal of the photoelectric conversion unit 114 is an image on the imaging surface 110. Therefore, if the amount of deviation is calculated by processing a pair of image signals generated by the focus detection pixels, the focus adjustment state of the optical system can be detected. it can.

  By the way, the pair of distance measurement pupils 123 and 124 do not have a distribution obtained by simply projecting the pair of photoelectric conversion units 113 and 114, but the light diffraction effect according to the aperture diameter (substantially coincides with the pixel size) of the microlens 112. Due to this, the distribution is blurred and has a base. In FIG. 3, when the pair of distance measurement pupils 123 and 124 are scanned in the alignment direction using the slit in the direction perpendicular to the alignment direction of the pair of distance measurement pupils 123 and 124, a pair of distance measurement pupil distributions 133 and 134 are obtained. . Due to the diffraction effect, the pair of distance measuring pupil distributions 133 and 134 have overlapping portions 135 at adjacent portions. As the ratio of the superimposing unit 135 increases with respect to the entire distance measurement pupil distribution 133 or 134, the pair of distance measurement pupils 123 and 124 are not completely separated, and the focus detection performance decreases. In particular, when the aperture value F value of the optical system is large and the aperture diameter is small, the pair of light beams transmitted through the optical system pass through the region in the vicinity of the optical axis in the pair of distance measuring pupils 123 and 124. Then, the light enters the focus detection pixel 111. For this reason, since the ratio of the superimposing unit 135 increases and the pair of light beams used for focus detection is incompletely separated, the focus detection performance deteriorates or the focus detection becomes impossible.

Table 1 shows the relationship between the aperture F value of the optical system and the diameter of the spread of the point image distribution by diffraction, and the Airy disk formula (diameter of point image = 1.22 × 2 × wavelength × F value). And the wavelength = 500 nm). For a bright optical system (small aperture F value), the diameter of the point image is on the order of μm, so that the resolution can be expected to be improved by setting the pixel size of the imaging pixel to be equal to or smaller than the diameter of the point image.

  On the other hand, as described above, when the pixel size is reduced, the influence of diffraction increases and the separation of the distance measuring pupil becomes incomplete, so that the focus detection performance is degraded.

Table 2 shows the relationship between the pixel size (opening diameter D of the microlens) and the F value corresponding to the overlapping portion 135 obtained by dividing the distance d in FIG. 3 by the dimension x of the overlapping portion 135 of the pair of distance measurement pupil distributions. And obtained from the Airy disk equation (when F = D / (1.22 × 2 × wavelength) and wavelength = 500 nm). When the pixel size is 7 μm or less, the F value of the overlapping portion 135 is 5.7 or less.

  Since the open F value of many interchangeable lenses for cameras is set to F5.6, when these interchangeable lenses are used, if the pixel size of the focus detection pixel is 7 μm or less, the aperture of F5.6 is set. Since the pair of focus detection light fluxes that pass through overlap each other, a drop in focus detection performance becomes obvious. Further, when the pixel size of the focus detection pixel is 4 μm or less, a pair of focus detection light beams passing through the aperture of approximately F2.8 are superimposed over the entire area, so that the focus detection performance is significantly deteriorated.

  As described above, regarding the pixel size, there are conflicting requirements for the imaging pixel and the focus detection pixel, and it is difficult to solve this problem if the pixel sizes of the imaging pixel and the focus detection pixel are kept the same. .

  Therefore, in the image pickup device of the image pickup apparatus of the present embodiment, the focus detection performance is maintained by reducing the pixel size of the image pickup pixel and setting the pixel size of the focus detection pixel to the pixel size of four image pickup pixels. To do.

  FIG. 4 is a front view showing a detailed configuration of the image sensor 212, and shows an enlarged vicinity of the focus detection area 101 on the image sensor 212. Imaging pixels 310 are densely arranged on the imaging element 212 in a two-dimensional square lattice pattern. The imaging pixel 310 includes a red pixel (R), a green pixel (G), and a blue pixel (B), and is arranged according to a Bayer arrangement rule. At a position corresponding to the focus detection area 101, a focus detection focus detection pixel 311 having a pixel size corresponding to four of the imaging pixels 310 is arranged so that a red pixel 45 and a blue pixel should be continuously arranged. It is arranged continuously on a straight line in the degree direction.

  FIG. 5 is a diagram showing the positional relationship between the focus detection pixels 311 and the imaging pixels (red pixels, green pixels, and blue pixels) that should be originally arranged at the positions according to the arrangement rule of the Bayer array. The reason why the focus detection pixels 311 are arranged in the oblique direction is that the number of imaging pixels 310 surrounding the focus detection pixels 311 is larger than that in the case where the focus detection pixels 311 are continuously arranged in the horizontal or vertical direction. This is because it increases. Accordingly, the imaging pixels 310 are arranged around the focus detection pixels 311 as compared to the case where the focus detection pixels 311 are continuously arranged in the horizontal or vertical direction. Since pixel interpolation using these imaging pixels 310 is possible, the performance of pixel interpolation is improved, and a high-quality image is obtained.

  The pixel size of the imaging pixel 310 can take a value in the wavelength order of visible light as a theoretical minimum value, but here, the pixel size of the focus detection pixel 311 is 4 μm, which is 2 μm, which is the minimum limit in terms of manufacturing technology. To do. Although not shown, the configuration in the vicinity of the focus detection areas 102 and 103 is the same as the configuration shown in FIG.

  As shown in FIG. 6A, the imaging pixel 310 includes the microlens 19, the photoelectric conversion unit 11, and a color filter (not shown). There are three types of color filters, red (R), green (G), and blue (B), and the respective spectral sensitivities have the characteristics shown in FIG. In the imaging device 212, imaging pixels 310 having respective color filters are arranged according to the arrangement rule of the Bayer array.

  The focus detection pixel 311 includes the microlens 10 and a pair of photoelectric conversion units 13 and 14 as illustrated in FIG. The focus detection pixel 311 is provided with an ND filter (neutral density filter), which will be described later, as a light amount reduction filter. The spectral characteristics of the focus detection pixel 311 include the spectral characteristics of a photodiode that performs photoelectric conversion and the spectral characteristics of an infrared cut filter (not shown). (See FIG. 8). That is, the spectral characteristics are obtained by adding the spectral characteristics of the green pixel, the red pixel, and the blue pixel shown in FIG. 7, and in the light wavelength region that exhibits high spectral sensitivity, the green pixel, red pixel, and blue pixel exhibit high spectral sensitivity. It covers the light wavelength region and is equal to almost the entire wavelength region of visible light.

  The photoelectric conversion unit 11 of the imaging pixel 310 is designed in such a shape that the microlens 19 receives all light beams that pass through the exit pupil diameter (for example, F1.0) of the brightest interchangeable lens. In addition, the photoelectric conversion units 13 and 14 of the focus detection pixel 311 are configured so that the microlens 10 receives all the light beams that pass through a predetermined region of the exit pupil (exit pupil diameter is F1.0, for example) of the interchangeable lens. Designed to.

  FIGS. 9A and 9B are cross-sectional views of the imaging pixel 310 and the focus detection pixel 311 (a cross section in a 45-degree direction obliquely upward to the right). In the imaging pixel 310 of FIG. 9A, the microlens 19 is disposed above the photoelectric conversion unit 11 for imaging, and the shape of the photoelectric conversion unit 11 is projected upward by the microlens 19. The photoelectric conversion unit 11 is formed on the semiconductor circuit substrate 29. Flattening layers 30 and 31 are formed on the photoelectric conversion unit 11, a color filter 33 is formed on the flattening layer 31, a flattening layer 32 is formed thereon, and the microlens 19 is formed thereon. It is formed.

  In the focus detection pixel 311 in FIG. 9B, the microlens 10 including the inner lens 35 and the upper lens 36 is disposed above the photoelectric conversion units 13 and 14, and the shape of the photoelectric conversion units 13 and 14 is changed by the microlens 10. Projected upward. The photoelectric conversion units 13 and 14 are formed on the semiconductor circuit substrate 29. A planarization layer 30 is formed on the photoelectric conversion units 13 and 14, an inner lens 35 is formed on the planarization layer 30, a planarization layer 31 is formed on the inner lens 35, and the planarization layer An ND filter 34 is formed on 31, a planarizing layer 32 is formed thereon, and an upper lens 36 is formed thereon.

  The reason why the micro lens of the focus detection pixel 311 is divided into the upper layer lens 36 and the inner lens 35 is that the light incident surface of the imaging pixel 310 and the focus detection pixel 311 (the micro lens 19 and the upper layer lens 36 are formed). However, it is preferable in manufacturing that the distance between the upper surface of the planarization layer 32 and the photoelectric conversion surface (the upper surface of the semiconductor circuit substrate 29 on which the photoelectric conversion units 11, 13, and 14 are formed) is the same. This is because the microlens 19 of the imaging pixel 310 cannot be applied to the focus detection pixel 311 with the pixel size doubled as it is.

  The distance between the light incident surface and the photoelectric conversion surface is set to be as short as possible in order to prevent oblique incident stray light from adjacent pixels. Accordingly, the radius of curvature of the microlens 19 of the imaging pixel 311 is made small so that the focal length is substantially equal to the distance between the light incident surface and the photoelectric conversion surface. The shape has an inclination. Therefore, even if a lens having the same radius of curvature as the microlens 19 is applied to the microlens 10 so that the microlens 10 of the focus detection pixel 311 has the same focal length as the microlens 19 of the imaging pixel 310, The entire area of the light incident surface of the focus detection pixel 311 having twice the pixel size cannot be covered.

  Therefore, the microlens 10 is separated into the upper layer lens 36 and the inner lens 35, the radius of curvature of the upper layer lens 36 is increased, and the combined focal length of the upper layer lens 36 and the inner lens 35 is approximately the light incident surface and the photoelectric conversion. Keep the distance between the faces. Thus, the entire area of the light incident surface of the focus detection pixel 311 having a pixel size twice that of the imaging pixel 310 can be covered.

  The reason why the ND filter 34 is installed in the focus detection pixel 311 is that, when a live view display operation as described later is performed, whether the output of the focus detection pixel is substantially equal to the output of the imaging pixel under the same exposure amount condition, It is for adjusting so that it may become less. Thereby, the image data for live view display and the data for focus detection are satisfactorily obtained in one exposure operation.

  When there is no ND filter, the output of the focus detection pixel 311 with respect to white incident light is approximately twice the output of the image pickup pixel 310 depending on the presence or absence of the color filter (the most output among the colors constituting the white light). The output of the image pickup pixel 310 without the color filter is about twice that of the output of the image pickup pixel 310 having a high green filter), and is about 4 times due to the difference in pixel size. Due to the difference in size, it is about ½ (based on the point that the photoelectric conversion unit of the focus detection pixel 311 is divided into two), which is about four times in total. In this case, there is a high possibility that the output of the focus detection pixel is saturated under an exposure condition (charge accumulation time) that is an appropriate level as the output level of the imaging pixel.

  Therefore, an ND filter that reduces the amount of light to ¼ to ８ is installed in the focus detection pixel 311 so that the output of the focus detection pixel 311 does not saturate even under the same exposure conditions as the imaging pixel 310. Here, depending on the state of vignetting of the optical system, the incident light to the focus detection pixel 311 may be concentrated on one of the two photoelectric conversion units 13 and 14, and thus the attenuation rate of the ND filter is 1 / It is preferable that it is 4 to 1/8.

  FIG. 10 is a diagram for explaining the principle of the pupil division type phase difference detection method using the focus detection pixels 311 of the image sensor 212 shown in FIG. The exit pupil 90 of the optical system is set at a position of a distance d in front of the focus detection pixels 311a and 311b arranged on the planned imaging plane of the interchangeable lens. Here, the distance d is a distance determined in accordance with the radius of curvature and the refractive index of the microlenses 50 and 60, the distance between the microlenses 50 and 60 and the photoelectric conversion unit, and is hereinafter referred to as a distance measurement pupil distance. Call. In addition to FIG. 10, photoelectric conversion units (53, 54), (63, 64) that form pairs in the optical axis 91 of the interchangeable lens, focus detection pixels 311 a, 311 b, focus detection light beams 73, 74, 83, 84. It is shown.

  Further, the distance measuring pupil 93 is formed by projecting the shapes of the photoelectric conversion units 53 and 63 by the microlenses 50 and 60. Similarly, the distance measuring pupil 94 is formed by projecting the shapes of the photoelectric conversion units 54 and 64 by the microlenses 50 and 60. In FIG. 17, a focus detection pixel 311a (having a microlens 50 and a pair of photoelectric conversion units 53 and 54) on the optical axis 91 and an adjacent focus detection pixel 311b (a microlens 60 and a pair of photoelectric conversions) are provided. (Converters 63 and 64) are schematically illustrated, but in each of the other focus detection pixels 311 arranged on the periphery of the imaging surface, the pair of photoelectric conversion units is a pair of distance measuring units. A light beam coming from the pupils 93 and 94 to the microlens is received. The arrangement direction of the focus detection pixels 311 is made to coincide with the arrangement direction of the pair of distance measuring pupils 93 and 94.

  The microlenses 50 and 60 are disposed in the vicinity of the planned imaging plane of the optical system, and the shape of the pair of photoelectric conversion units 53 and 54 disposed behind the microlens 50 disposed on the optical axis 91 is formed. Projection is performed on the exit pupil 90 separated from the microlens 50 by the distance measurement pupil distance d, and the projection shape forms distance measurement pupils 93 and 94. Further, the shape of the pair of photoelectric conversion units 63 and 64 disposed behind the microlens 60 is projected onto the exit pupil 90 separated from the microlens 60 by the distance measurement pupil distance d, and the projection shape is the distance measurement pupil. 93, 94 are formed. In other words, the microlens and photoelectric sensor in each focus detection pixel 311 are arranged so that the projection shapes (ranging pupils 93 and 94) of the photoelectric conversion units of the focus detection pixels 311 coincide on the exit pupil 90 at the distance measurement pupil distance d. The relative positional relationship of the conversion unit is determined.

  The photoelectric conversion unit 53 outputs a signal corresponding to the intensity of the image formed on the microlens 50 by the focus detection light beam 73 passing through the distance measuring pupil 93 and traveling toward the microlens 50. In addition, the photoelectric conversion unit 54 outputs a signal corresponding to the intensity of the image formed on the microlens 50 by the focus detection light beam 74 that passes through the distance measuring pupil 94 and travels toward the microlens 50. Similarly, the photoelectric conversion unit 63 outputs a signal corresponding to the intensity of the image formed on the microlens 60 by the focus detection light beam 83 that passes through the distance measuring pupil 93 and travels toward the microlens 60. In addition, the photoelectric conversion unit 64 outputs a signal corresponding to the intensity of the image formed on the microlens 60 by the focus detection light beam 84 that passes through the distance measuring pupil 94 and travels toward the microlens 60.

  A large number of such focus detection pixels 311 are arranged in a straight line, and the output of a pair of photoelectric conversion units of each focus detection pixel 311 is collected into an output group corresponding to the distance measurement pupil 93 and the distance measurement pupil 94, thereby ranging. Information on the intensity distribution of a pair of images formed on the focus detection pixel array by the focus detection light fluxes that respectively pass through the pupil 93 and the distance measurement pupil 94 is obtained. By applying an image shift detection calculation process (correlation calculation process, phase difference detection process), which will be described later, to this information, an image shift amount of a pair of images is detected by a so-called pupil division type phase difference detection method. Further, by applying a predetermined conversion process to the image shift amount, the deviation of the current imaging plane (imaging plane at the focus detection position corresponding to the position of the microlens array on the planned imaging plane) with respect to the planned imaging plane (Defocus amount) is calculated.

  FIG. 11 is a diagram for explaining the state of the imaging light beam received by the imaging pixel 311 of the imaging element 212 shown in FIG. 4 in comparison with FIG. 10, and a description of portions overlapping those in FIG. 10 is omitted.

  The imaging pixel 310 includes a microlens 59 and a photoelectric conversion unit 51 disposed behind the microlens 59, and the shape of the photoelectric conversion unit 51 is projected onto the exit pupil 90 that is separated from the microlens 59 by the distance measurement pupil distance d. The projection shape forms a region 92 that is substantially circumscribed by the distance measuring pupils 93 and 94.

  The photoelectric conversion unit 51 outputs a signal corresponding to the intensity of the image formed on the microlens 51 by the imaging light beam 71 that passes through the region 92 and travels toward the microlens 59.

  In the present embodiment, the image sensor 212 is configured as a CMOS image sensor. FIG. 12 is a diagram illustrating a basic circuit configuration of one photoelectric conversion unit included in the imaging pixel 310 and the focus detection pixel 311.

  The photoelectric conversion unit includes a photodiode PD. The charges accumulated in the PD are accumulated in the floating diffusion layer FD (floating diffusion). The FD is connected to the gate of the amplification MOS transistor AMP, and the AMP generates a signal corresponding to the amount of charge accumulated in the FD. The FD is connected to the power supply voltage Vdd via the reset MOS transistor SW1. When the reset MOS transistor SW1 is turned on by the control signal ΦRn, the charges accumulated in the FD and PD are cleared and the reset state is set. The output terminal of the AMP is connected to the vertical output line Vout via the row selection MOS transistor SW2. When the row selection MOS transistor SW2 is turned on by the control signal ΦSn, the output of the AMP is output to the vertical output line Vout. . Although omitted here, a transfer transistor is arranged between PD and FD.

  FIG. 13 is a diagram showing a circuit wiring layout on the semiconductor substrate corresponding to the region of the imaging pixels 310 of 6 rows and 6 columns in the imaging device 212 shown in FIG. A focus detection pixel 311 is arranged in a 45 ° upward and diagonal direction, PD is a photoelectric conversion unit of the imaging pixel 310, PD1 and PD2 are a pair of photoelectric conversion units of the focus detection pixel 311, and CM is a pixel circuit of the imaging pixel 310 (FIG. CM1 and CM2 are a pair of pixel circuits of the focus detection pixel 311, Vout1 to Vout6 are vertical output lines corresponding to pixels in the first to sixth columns, and H1 to H6. Indicates horizontal control signal lines (corresponding to the control signal lines in the broken line area indicated by H in FIG. 12) corresponding to the pixels in the first to sixth rows.

  The pair of photoelectric conversion units PD1 and PD2 of the focus detection pixel 311 needs to be adjacent to each other at an extremely narrow interval in order to perform focus detection corresponding to an interchangeable lens having a low F value. If the vertical output line Vout3 and the horizontal control signal line H3 are laid out in the same manner as the other vertical control signal lines and horizontal control signal lines, there is a risk of interference with the photoelectric conversion units PD1 and PD2 of the focus detection pixel 311. In addition, wiring around the photoelectric conversion units PD1 and PD2 of the focus detection pixel 311 is bypassed. Thereby, the signal of the focus detection pixel 311 can be read with high accuracy by the same pixel signal read-out operation as that of the image sensor 212 having only the image pickup pixel 310 without the focus detection pixel 311.

  FIG. 14 is a flowchart illustrating an imaging operation of the digital still camera 201 according to the present embodiment. When the power of the digital still camera 201 is turned on in step S100, the body drive control device 214 starts an imaging operation after step S110. In step S110, the data of the imaging pixel 310 is read out and displayed on the electronic viewfinder. In subsequent step S120, a pair of image data corresponding to the pair of images is read from the focus detection pixel array. The focus detection area is assumed to be selected in advance by the photographer using one of the focus detection areas 101 to 103 using a focus detection area selection member (not shown).

  In step S130, an image shift detection calculation process (correlation calculation process, phase difference detection process) described later is performed based on the read pair of image data, and the image shift amount is calculated and converted into a defocus amount. In step S140, it is checked whether or not the focus is close, that is, whether or not the calculated absolute value of the defocus amount is within a predetermined value. If it is determined that the lens is not in focus, the process proceeds to step S150, the defocus amount is transmitted to the lens drive controller 206, and the focusing lens 210 of the interchangeable lens 202 is driven to the focus position. Then, it returns to step S110 and repeats the operation | movement mentioned above.

  Even when focus detection is impossible, the process branches to this step, a scan drive command is transmitted to the lens drive control device 206, and the focusing lens 210 of the interchangeable lens 202 is driven to scan from infinity to the nearest. Then, it returns to step S110 and repeats the operation | movement mentioned above.

  If it is determined in step S140 that the focus is close, the process proceeds to step S160, and it is determined whether or not a shutter release has been performed by operating a shutter button (not shown). If it is determined that the shutter release has not been performed, the process returns to step S110 and the above-described operation is repeated. On the other hand, if it is determined that the shutter release has been performed, the process proceeds to step S170, where an aperture adjustment command is transmitted to the lens drive control unit 206, and the aperture value of the interchangeable lens 202 is set to the control F value (F set by the photographer or automatically). Value). When the aperture control is finished, the image sensor 212 is caused to perform an imaging operation, and image data is read from the image pickup pixel 310 and all the focus detection pixels 313 and 314 of the image pickup element 212.

  In step S <b> 180, pixel interpolation is performed on the pixel data at each pixel position in the focus detection pixel row based on the data of the imaging pixels 310 around the focus detection pixel 311. In subsequent step S190, image data composed of the data of the imaging pixel 310 and the interpolated data is stored in the memory card 219, and the process returns to step S110 to repeat the above-described operation.

  Details of the image shift detection calculation process (correlation calculation process, phase difference detection process) in step S130 of FIG. 14 will be described below.

The pair of images detected by the focus detection pixel 311 may maintain the image shift detection accuracy with respect to the light quantity balance because the distance measurement pupils 93 and 94 may be displaced by the aperture of the lens and the light quantity balance may be lost. Perform possible types of correlation operations. Based on the application of the present applicant for a pair of data strings (A1 ₁ ,..., A1 _M , A2 ₁ ,..., A2 _M : M is the number of data) read out from the focus detection pixel string. The correlation amount C (k) is calculated by the equation (1) disclosed in 2007-333720.
C (k) = Σ | A1 _n · A2 _{n + 1 + k−} A2 _{n + k} · A1 _{n + 1} | (1)

In equation (1), Σ operations are accumulated for n. The range taken by n is limited to a range in which data of A1 _n , A1 _{n + 1} , A2 _{n + k} , A2 _{n + 1 + k} exists according to the image shift amount k. The image shift amount k is an integer and is a relative shift amount with the data interval of the data string as a unit.

As shown in FIG. 15A, the calculation result of the expression (1) indicates that the correlation amount C (k) is obtained when the pair of data has a high correlation amount (k = k _j = 2 in FIG. 15A). Minimal (the smaller the value, the higher the degree of correlation). The shift amount x that gives the minimum value C (x) when the correlation amount is regarded as continuous is obtained using the three-point interpolation method according to the equations (2) to (5).
x = k _j + D / SLOP (2)
C (x) = C (k _j ) − | D | (3)
D = {C (k _j −1) −C (k _j +1)} / 2 (4)
SLOP = MAX {C (k _j +1) −C (k _j ), C (k _j −1) −C (k _j )} (5)

  Whether the shift amount x calculated by the equation (2) is reliable is determined as follows. As shown in FIG. 15B, when the degree of correlation between a pair of data is low, the value of the interpolated correlation minimum value C (x) is large. Therefore, when C (x) is equal to or greater than a predetermined threshold, it is determined that the reliability of the calculated shift amount x is low, and the calculated shift amount x is canceled. Alternatively, in order to normalize C (x) with the contrast of data, when the value obtained by dividing C (x) by SLOP that is proportional to the contrast is equal to or greater than a predetermined value, the reliability of the calculated shift amount x The calculated shift amount x is cancelled.

Alternatively, if SLOP that is proportional to the contrast is equal to or smaller than a predetermined value, it is determined that the subject has low contrast and the reliability of the calculated shift amount x is low, and the calculated shift amount x is canceled. . As shown in FIG. 15C, when the correlation between the pair of data is low and there is no drop in the correlation amount C (k) between the shift range _kmin and _kmax , the minimum value C (x) is obtained. In such a case, it is determined that the focus cannot be detected. When it is determined that the calculated shift amount x is reliable, it is converted into the image shift amount shft by the equation (6).
shft = PY · x (6)

In Expression (6), PY is the pixel pitch (detection pitch) in the 45-degree direction of the focus detection pixel 311. The image shift amount calculated by Expression (6) is multiplied by a predetermined conversion coefficient Kd to convert it to a defocus amount def.
def = Kd · shft (7)

  Details of the calculation of the image data at the focus detection pixel position in step S180 in FIG. 14 will be described below with reference to FIGS. In the following description, processing focusing on a certain focus detection pixel 311 will be described, but the same processing is applied to calculation of image data at all focus detection pixel positions.

  FIG. 16 illustrates the data (G11 to G66) of the imaging pixel 310 and the central focus detection pixel 311 in the case where the focus detection pixel 311 is arranged in a 45-degree obliquely upward right direction in the region of the imaging pixel 310 of 6 rows and 6 columns. A pair of data (A1, A2) is shown corresponding to the pixel layout. G33a, R34a, B43a, and G44a are data of each imaging pixel (imaging pixel data to be interpolated by processing to be described later) when four imaging pixels are arranged at the central focus detection pixel position.

The process proceeds from step S180 of FIG. 14 to step 181 of FIG. 17, and determination of the direction in which the continuity of the image is high for each color is performed as follows based on the imaging pixel data around the focus detection pixel 311. The vertical evaluation value Sgv for green is a value obtained by dividing the absolute value of the difference between the green pixel data separated in the vertical direction in FIG. 16 by the predetermined normalization constant kgv. The distance between the pixels for which the difference is taken is integrated for all cases (in the case of a green imaging pixel, every other pixel in the vertical direction is possible every third pixel), and the difference between the imaging pixels sandwiching the focus detection pixel Is also included.
Sgv = (| G11−G31 | + | G22−G42 | + | G13−G53 | + |
G24-G64 | + | G35-G55 | + | G46-G66 |) / kgv (8)

  The normalization constant kgv (and the normalization constant described later) is a value determined according to the integrated number, the distance between pixel data for which the difference is taken, and the color, and the value of the integrated value is between the integrated number and the difference pixels. It is a constant for normalization so as not to vary according to distance and color.

Similarly, a horizontal direction evaluation value Sgh, a right upward slanting 45 degree direction evaluation value Sgc, and a left upward slanting 45 degree direction evaluation value Sgd for green are obtained.
Sgh = (| G11−G13 | + | G22−G24 | + | G31−G35 | + |
G42-G46 | + | G53-G55 | + | G64-G66 |) / kgh (9)
Sgc = (| G13-G22 | + | G22-G31 | + | G24-G42 | +
| G35-G53 | + | G46-G55 | + | G55-G64 |) / kgc (10)
Sgd = (| G11−G22 | + | G13−G24 | + | G31−G42 | +
| G35-G46 | + | G53-G64 | + | G55-G66 |) / kgd (11)

The vertical direction evaluation value Srv for red is a value obtained by dividing the absolute value of the difference between red imaging pixel data separated in the vertical direction in FIG. 16 within the range of FIG. 16 by a predetermined normalization constant krv.
Srv = (| R12-R32 | + | R14-R54 | + | R36-R66 |) / krv (12)

Similarly, the horizontal direction evaluation value Srh for red, the upward 45-degree oblique direction evaluation value Src, and the upward-left oblique 45 degree direction evaluation value Srd are obtained.
Srh = (| R12-R14 | + | R32-R36 | + | R54-R56 |) / krh (13)
Src = (| R14-R32 | + | R36-R54 |) / krc (14)
Srd = (| R14-R36 | + | R32-R54 |) / krd (15)

Similarly to the red color, the vertical direction evaluation value Sbv, the horizontal direction evaluation value Sbh, the right upward slanting 45 degree direction evaluation value Sbc, and the left upward slanting 45 degree direction evaluation value Sbd are obtained.
Sbv = (| B21−B41 | + | B23−B63 | + | B45−B65 |) / kbv (16)
Sbh = (| B21−B23 | + | B41−B45 | + | B63−B65 |) / kbh (17)
Sbc = (| B41−B23 | + | B63−B45 |) / kbc (18)
Sbd = (| B41−B63 | + | B23−R45 |) / kbd (19)

  The determination of the direction in which the continuity of each color is high is performed as follows. In the case of green, the direction corresponding to the evaluation value having the smallest value among the evaluation values Sgv, Sgh, Sgc, and Sgd is determined as the direction having high continuity. However, when the smallest evaluation value is equal to or greater than a predetermined threshold, it is determined that there is no directionality. Similarly, based on the evaluation values Srv, Srh, Src, Srd, and the evaluation values Sbv, Sbh, Sbc, Sbd, a direction in which red continuity is high and a direction in which blue continuity is high are determined.

In step S182, the first interpolation pixel data corresponding to B33a, G34a, G43a, and R44a is obtained as follows based on the directionality determination result of each color.
G33a = (G13 + G53) / 2 Continuous in the vertical direction G33a = (G31 + G35) / 2 Continuous in the horizontal direction G33a = (G24 + G42) / 2 Continuous in the 45 ° upward direction G33a = (G22 + G55) / 2 Left Continuity in 45 degree direction G33a = (G24 + G42 + G22 + G55) / 4 No continuity G44a = (G24 + G64) / 2 Continuity in vertical direction G44a = (G42 + G46) / 2 Continuity in horizontal direction G44a = (G35 + G53) / 2 There is continuity in the 45 ° upward direction G44a = (G22 + G55) / 2 There is continuity in the 45 ° upward direction G44a = (G35 + G53 + G22 + G55) / 4 No continuity R34a = (R14 + R54) / 2 There is continuity in the vertical direction R34a = (R32 + R36) / 2 Water There is continuity in the direction R34a = (R14 + R54 + R32 + R36) / 4 There is continuity in the 45 ° upward direction R34a = (R12 + R56) / 2 There is continuity in the 45 ° upward direction R34a = (R14 + R54 + R32 + R36 + R12 + R56) / 6 No continuity B43a = ( B23 + B63) / 2 Continuous in the vertical direction B43a = (B41 + B45) / 2 Continuous in the horizontal direction B43a = (B23 + B63 + B41 + B45) / 4 Continuous in the 45 ° upward direction B43a = (B21 + B65) / 2 Up 45 ° in the left direction There is continuity in the direction B43a = (B23 + B63 + B41 + B45 + B21 + B65) / 6 No continuity

  As described above, in step S182, it is possible to interpolate the first interpolation pixel data from the data of the imaging pixel 310 sandwiching the focus detection pixel 311 with respect to the four directions on the premise of the continuity of the image. An interpolation result is obtained.

  In step S183, it is determined whether or not a point image exists at the focus detection pixel position. This is because the first interpolation pixel data obtained in step S182 is data obtained on the assumption of image continuity. Therefore, when a point image is present at the focus detection pixel position in isolation, accurate interpolation pixel data is obtained. It is because it cannot be obtained.

In the point image determination, the sum of the output signal data A1 and A2 of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 represents the luminance at the focus detection pixel position, and R, G, B obtained in step S182 is obtained. Since the value representing the luminance can be similarly calculated from the linear sum of the first interpolation data, the two luminance values are compared, and if the difference is large, it is determined that a point image exists. For example, the point image determination evaluation value P is determined by the following equation.
P = P1-P2 (20)
P1 = (A1 + A2) · ka (21)
P2 = G33a · kg + G44a · kg + R34a · kr + B43a · kb (22)

  In the above equation, the coefficients ka, kg, kr, and kb are the sum of the output signal data A1 and A2 of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 in advance in the uniform illumination state of the light sources of white and other colors. And the first summation pixel data (G33a, G44a, R34a, B43a) are obtained so that the linear sums are substantially equal. When the absolute value of the point image determination evaluation value P is larger than a predetermined threshold, it is determined that there is a point image, and when it is smaller, it is determined that there is no point image.

In step S184, the processing flow branches depending on the presence or absence of a point image. If there is no point image, it is assumed in step S185 that the first interpolation pixel data is finally arranged with four imaging pixels 310 at the focus detection pixel position. Interpolated pixel data to be assigned as each imaging pixel data in this case, and the process proceeds to step S190 in FIG. If there is a point image, the process proceeds to step S186, where the first interpolation pixel data is corrected as follows to obtain final interpolation pixel data, and the process proceeds to step 190 in FIG.
G33a = G33a · P1 / P2 (23)
G44a = G44a · P1 / P2 (24)
R34a = R34a · P1 / P2 (25)
B43a = B43a · P1 / P2 (26)

  By performing the correction as described above, the luminance obtained from the linear sum of the first interpolation pixel data (G33a, G44a, R34a, B43a) becomes the output signal data of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311. Since it matches the luminance obtained from the sum of A1 and A2, a good interpolation result can be obtained even when a point image exists.

  When the focus detection pixels 311 are continuously arranged in the horizontal or vertical direction, the number of imaging pixels 310 closest to the one focus detection pixel 311 in the horizontal direction, the vertical direction, and the diagonal direction is eight. When the detection pixels 311 are continuously arranged in the oblique direction, the number of imaging pixels 310 that are closest to the focus detection pixel 311 is 10, so that the pixel interpolation accuracy is improved accordingly.

  Further, when the focus detection pixels 311 are continuously arranged in the oblique direction, assuming that the imaging pixel 310 is virtually arranged at the position of the focus detection pixel 311, it is close to the virtual imaging pixel to be interpolated, In addition, since an imaging pixel having the same color as the virtual imaging pixel always exists at a position sandwiching the virtual imaging pixel in the horizontal direction and the vertical direction, the pixel interpolation performance in the horizontal direction and the vertical direction makes the focus detection pixel 311 in the horizontal or vertical direction. It is superior to the case where it is continuously arranged. Since many pixel interpolation patterns in a general image are in the horizontal and vertical directions, an improvement in pixel interpolation performance can be expected.

In the above-described embodiment, the pupil division performance is improved by making the pixel size of the focus detection pixel 311 larger than that of the imaging pixel 310, and the imaging element 212 in which the imaging pixels 310 are arranged in a square lattice shape. In FIG. 4, the image quality is prevented from being deteriorated due to pixel interpolation by arranging in an oblique direction.
<< Other Embodiments of the Invention >>

  In the interpolation processing flow shown in FIG. 17, when it is determined that there is a point image, the first interpolation pixel data obtained by the data of the imaging pixels 310 around the focus detection pixel 311 is corrected to obtain final interpolation pixel data. However, if it is determined that there is a point image, the first interpolation pixel data obtained as having no continuity is corrected to obtain final interpolation pixel data regardless of the determination of the direction of continuity. You may do it.

  In the interpolation processing flow shown in FIG. 17, first interpolation pixel data is first obtained from data of the imaging pixels 310 around the focus detection pixel 311, and the first interpolation pixel data and the data of the focus detection pixel 311 are compared. Although the presence / absence determination of the point image is performed, the presence / absence determination of the point image may be performed as follows by directly comparing the imaging pixel data around the focus detection pixel 311 and the focus detection pixel data. good.

The sum of the output signal data A1 and A2 of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 represents the luminance at the focus detection pixel position, and the linear sum of the imaging pixel data of the three colors around the focus detection pixel 311. Represents the luminance around the focus detection pixel 311, and by comparing the two luminance values, it can be determined that a point image exists if the difference is large. For example, the point image determination evaluation value P is determined by the following equation.
P = P1-P2 (27)
P1 = (A1 + A2) · ka (28)
P2 = Gw · kg + Rw · kr + Bw · kb (29)

  In the above equation, Gw is obtained by dividing the sum of the data of 12 green imaging pixels 310 included in the range shown in FIG. 16 by 6, and Rw is imaging of 6 reds included in the range shown in FIG. The sum of the data of the pixels 310 is divided by 6, and Bw is the sum of the data of the six blue imaging pixels 310 included in the range shown in FIG.

  In the above equation, the coefficients ka, kg, kr, and kb are output signal data A1 and A2 of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 in a uniform illumination state of light sources of white and other colors in advance. And the linear sum of the imaging pixel data around the focus detection pixel 311 are determined to be substantially equal. When the absolute value of the point image determination evaluation value P is larger than a predetermined threshold value, it is determined that there is a point image having a luminance difference from its surroundings at the focus detection pixel position, and when it is smaller, it is determined that there is no point image.

  In the above determination, since only the difference between the luminance at the focus detection pixel position and the average luminance around the focus detection pixel 311 is focused, an erroneous determination is made when the luminance around the focus detection pixel 311 has fluctuated. There is a possibility to do. Therefore, the presence / absence of luminance fluctuation around the focus detection pixel 311 is determined. When there is luminance fluctuation, it is determined that there is no point image without performing point image existence determination, and only when there is no luminance fluctuation. You may make it perform determination.

  In the interpolation processing flow shown in FIG. 17, the first interpolation pixel data once obtained from the image pickup pixel data around the focus detection pixel 311 is corrected according to the presence or absence of the point image, and finally corrected. However, before calculating the first interpolation pixel data, the point image presence / absence determination as described above is performed. If there is no point image, the first interpolation pixel data is calculated as it is. As final interpolation pixel data, if there is a point image, the interpolation pixel data may be calculated separately as follows.

  That is, the color tone closest to the focus detection pixel 311 is obtained from the data of the ten imaging pixels 310 adjacent to the focus detection pixel 311, and the color tone is adjusted to the luminance value obtained from the focus detection pixel data, so that the final value is obtained. Interpolated pixel data is obtained.

In the following equation, the coefficients ka, kg, kr, and kb are the values of the output signal data A1 and A2 of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel in the uniform illumination state of the light sources of white and other colors in advance. The sum is obtained so that the linear sum of the image pickup pixel data of the three colors is substantially equal.
Gs = (G22 + G24 + G32 + G35 + G53 + G55) / 6 (30)
Rs = (R32 + R54) / 2 (31)
Bs = (B23 + B45) / 2 (32)
G33a = G44a
= Gs · ka · (A1 + A2) / (kg · Gs + kr · Rs + kb · Bs) (33)
R34a = Rs · ka · (A1 + A2) / (kg · Gs + kr · Rs + kb · Bs)
(34)
B43a = Bs · ka · (A1 + A2) / (kg · Gs + kr · Rs + kb · Bs)
(35)

  In the structure of the focus detection pixel 311 shown in FIG. 9B, the output of the photoelectric conversion units 13 and 14 of the focus detection pixel 311 and the output of the photoelectric conversion unit 11 of the imaging pixel 310 are substantially equal under the same exposure conditions. Although the ND filter 34 is provided, instead of providing the ND filter, the amplification MOS transistor AMP of the focus detection pixel 311 and the amplification MOS transistor AMP of the image pickup pixel 310 in the basic circuit configuration of the pixel shown in FIG. The output of the photoelectric conversion units 13 and 14 of the focus detection pixel 311 and the output of the photoelectric conversion unit 11 of the imaging pixel 310 may be substantially equal under the same exposure condition.

  The arrangement of the focus detection areas 101 to 103 in the image sensor 310 is not limited to that shown in FIG. 2, and the arrangement direction of the focus detection pixels 311 may be a 45 ° upward diagonal direction.

  In the arrangement of the focus detection pixels 311 shown in FIGS. 4 and 16, the focus detection pixels 311 replace the virtual imaging pixels on a straight line in which the blue and red pixels, which are the above-described virtual imaging pixels, continue in a 45-degree oblique direction. However, as shown in FIGS. 18 and 19, the focus detection pixels 311 are arranged in place of the virtual imaging pixels on a straight line in which green pixels that are virtual imaging pixels are continuous in an oblique 45 degree direction. May be.

  FIG. 19 corresponds to the enlarged view of FIG. 18, and image pickup pixel data (R11 to B66) in the case where the focus detection pixels 311 are arranged in the area of the 6 × 6 image pickup pixels 310 in a 45 ° upward and diagonal direction. ) And output signal data (A1, A2) of the pair of photoelectric conversion units 13 and 14 of the central focus detection pixel 311 are shown corresponding to the pixel layout. R33a, G34a, G43a, and B44a are data of each imaging pixel 310 (imaging pixel data to be interpolated by processing to be described later) when four imaging pixels 310 are arranged at the central focus detection pixel position 311. is there.

The operation of pixel interpolation processing in such a focus detection pixel arrangement will be described with reference to the flowchart of FIG. First, the process proceeds to step S181, and the determination of the direction in which the continuity of the image is high is performed for each color based on the imaging pixel data around the focus detection pixel 311 as follows. The vertical direction evaluation value Sgv for green is a value obtained by dividing the absolute value of the difference between the green pixel data separated in the vertical direction in FIG. 19 within the range of FIG. 19 by a predetermined standardization constant kgv. The distance between the pixels that take the difference is integrated for all cases (if the green image pickup pixel 310 is vertical, every other pixel can have a difference of every third pixel). Differences are also integrated.
Sgv = (| G21−G41 | + | G12−G32 | + | G23−G63 | +
| G14-G54 | + | G45-G65 | + | G36-G56 |) / kgv (36)

  The normalization constant kgv (and the normalization constant described later) is a value determined according to the integrated number, the distance between pixel data for which the difference is taken, and the color, and the value of the integrated value is between the integrated number and the difference pixels. It is a constant for normalization so as not to vary according to distance and color.

Similarly, a horizontal direction evaluation value Sgh, a right upward slanting 45 degree direction evaluation value Sgc, and a left upward slanting 45 degree direction evaluation value Sgd for green are obtained.
Sgh = (| G12−G14 | + | G21−G23 | + | G32−G36 |
+ | G41-G45 | + | G54-G56 | + | G63-G65 |) / kgh (37)
Sgc = (| G12−G21 | + | G41−G32 | + | G32−G23 |
+ | G23-G14 | + | G63-G54 | + | G54-G45 |
+ | G45-G36 | + | G65-G56 |) / kgc (38)
Sgd = (| G14−G36 | + | G12−G23 | + | G23−G45 |
+ | G45-G56 | + | G21-G32 | + | G32-G54 |
+ | G54-G65 | + | G41-G63 |) / kgd (39)

The vertical direction evaluation value Srv for red is a value obtained by dividing the absolute value of the difference between the red pixel data separated in the vertical direction in FIG. 19 by the predetermined normalization constant krv. .
Srv = (| R11-R31 | + | R13-R53 | + | R35-R55 |) / krv
(40)

Similarly, the horizontal direction evaluation value Srh for red, the upward 45-degree oblique direction evaluation value Src, and the upward-left oblique 45 degree direction evaluation value Srd are obtained.
Srh = (| R11-R13 | + | R31-R35 | + | R53-R55 |) / krh
(41)
Src = (| R31-R13 | + | R53-R35 |) / krc (42)
Srd = (| R13-R35 | + | R11-R55 | + | R31-R53 |) / krd
(43)

Similarly to the red color, the vertical direction evaluation value Sbv, the horizontal direction evaluation value Sbh, the right upward slanting 45 degree direction evaluation value Sbc, and the left upward slanting 45 degree direction evaluation value Sbd are obtained.
Sbv = (| B22-B42 | + | B24-B64 | + | B46-B66 |) / kbv
(44)
Sbh = (| B22−B24 | + | B42−B46 | + | B64−B66 |) / kbh
(45)
Sbc = (| B42−B24 | + | B64−B46 |) / kbc (46)
Sbd = (| B24−B46 | + | B22−R66 | + | B42−R64 |) / kbd
(47)

  The determination of the direction in which the continuity of each color is high is performed as follows. In the case of green, the direction corresponding to the evaluation value having the smallest value among the evaluation values Sgv, Sgh, Sgc, and Sgd is determined as the direction having high continuity. However, when the smallest evaluation value is equal to or greater than a predetermined threshold, it is determined that there is no directionality. Similarly, based on the evaluation values Srv, Srh, Src, Srd, and the evaluation values Sbv, Sbh, Sbc, Sbd, a direction in which red continuity is high and a direction in which blue continuity is high are determined.

In step S182, the first interpolation pixel data corresponding to B33a, G34a, G43a, and R44a is obtained as follows based on the directionality determination result of each color.
G34a = (G14 + G54) / 2 Continuous in the vertical direction G34a = (G32 + G36) / 2 Continuous in the horizontal direction G34a = (G14 + G54 + G32 + G36 + G23 + G45) / 6 Continuous in the 45 ° upward direction G34a = (G23 + G45) / 2 Left Continuity in the upward 45 degree direction G34a = (G14 + G54 + G32 + G36) / 4 No continuity G43a = (G23 + G63) / 2 Continuity in the vertical direction G43a = (G41 + G45) / 2 Continuity in the horizontal direction G43a = (G23 + G63 + G41 + G45 + G32 + G54) / 6 Continuity in 45 ° upward direction G43a = (G32 + G54) / 2 Continuity in 45 ° upward left direction G43a = (G23 + G63 + G41 + G45) / 4 No continuity R33a = (R13 + R5 3) / 2 Continuous in the vertical direction R33a = (R31 + R35) / 2 Continuous in the horizontal direction R33a = (R13 + R53 + R31 + R35) / 4 Continuous in the 45 ° upward direction R33a = (R11 + R55) / 2 45 ° upward There is continuity in the direction R33a = (R13 + R53 + R31 + R35 + R11 + R55) / 6 No continuity B44a = (B24 + B64) / 2 There is continuity in the vertical direction B44a = (B42 + B46) / 2 There is continuity in the horizontal direction B44a = (B24 + B64 + B42 + B46) / 4 There is continuity in the 45 degree direction B44a = (B22 + B66) / 2 There is continuity in the 45 degree upward direction B44a = (B24 + B64 + B42 + B46 + B22 + B66) / 6 No continuity

  As described above, in step S182, it is possible to interpolate the first interpolation pixel data from the data of the imaging pixel 310 sandwiching the focus detection pixel 311 with respect to the four directions on the premise of the continuity of the image. Results are obtained.

  In step S183, it is determined whether or not a point image exists at the focus detection pixel position. This is because the first interpolation pixel data obtained in step S182 is data obtained on the assumption of image continuity. Therefore, when a point image is present at the focus detection pixel position in isolation, accurate interpolation pixel data is obtained. It is because it cannot be obtained.

In the point image determination, the sum of the output signal data A1 and A2 of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 represents the luminance at the focus detection pixel position, and R, G, and B obtained in step S182 are obtained. Since a value representing luminance can be similarly calculated from the linear sum of the first interpolation data, the two luminance values are compared, and if the difference is large, it is determined that a point image exists. For example, the point image determination evaluation value P is determined by the following equation.
P = P1-P2 (48)
P1 = (A1 + A2) · ka (49)
P2 = G34a · kg + G43a · kg + R33a · kr + B44a · kb (50)

  In the above equation, the coefficients ka, kg, kr, and kb are output signal data A1 and A2 of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311 in a uniform illumination state of light sources of white and other colors in advance. And the linear sum of the first interpolation pixel data (G34a, G43a, R33a, B44a) are obtained so as to be substantially equal. When the absolute value of the point image determination evaluation value P is larger than a predetermined threshold, it is determined that there is a point image, and when it is smaller, it is determined that there is no point image.

If there is no point image in step S184, the process branches depending on the presence or absence of the point image. In step S185, each of the first interpolation pixel data is finally arranged at the focus detection pixel position in which four imaging pixels 310 are arranged. The interpolated pixel data assigned as the imaging pixel data is set, and the process proceeds to Step 190 in FIG. If there is a point image, the process proceeds to step S186, where the first interpolation pixel data is corrected as follows to obtain final interpolation pixel data, and the process proceeds to step 190 in FIG.
G34a = G34a · P1 / P2 (51)
G43a = G43a · P1 / P2 (52)
R33a = R33a · P1 / P2 (53)
B44a = B44a · P1 / P2 (54)

  By performing the correction as described above, the luminance obtained from the linear sum of the first interpolation pixel data (G34a, G43a, R33a, B44a) is output signal data of the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311. Since it matches the luminance obtained from the sum of A1 and A2, a good interpolation result can be obtained even when a point image exists.

  Regarding the pixel interpolation, the arrangement of the focus detection pixels 311 in FIGS. 4 and 16 is compared with the arrangement of the focus detection pixels 311 in FIGS. 18 and 19. The number of surrounding red pixels closest to the red pixel that is the virtual imaging pixel is 6 (horizontal direction 2, vertical direction 2, left diagonal direction 2) in either arrangement, and the same applies to the blue pixel. . On the other hand, the number of surrounding green pixels closest to the green pixel that is the virtual imaging pixel is 7 in the former arrangement (horizontal direction 2, vertical direction 2, right diagonal direction 2, left diagonal direction 1), and the latter Since the arrangement is 6 (horizontal direction 2, vertical direction 2, left diagonal direction 2), the former arrangement is preferable in terms of image quality after pixel interpolation.

  In the imaging device 212 illustrated in FIG. 4, the example in which the focus detection pixel 311 includes the pair of photoelectric conversion units 13 and 14 is illustrated. However, one focus detection pixel 311 may include one photoelectric conversion unit. . FIG. 20 is a partially enlarged view of such an image sensor 212. The focus detection pixel 313 and the focus detection pixel 314 are photoelectric conversion units corresponding to the pair of photoelectric conversion units 13 and 14 of the focus detection pixel 311, respectively. Prepare. A pair of the focus detection pixel 313 and the focus detection pixel 314 illustrated in FIG. 20 performs a function corresponding to the focus detection pixel 311 illustrated in FIG. Similarly, FIG. 21 is a diagram in which the focus detection pixel 311 of the image sensor 212 shown in FIG. 18 is replaced with a focus detection pixel 313 and a focus detection pixel 314. In the focus detection pixels 313 and 314 shown in FIGS. 20 and 21, the shape of the photoelectric conversion units 13 and 14 is projected forward by the microlens 10 in the same manner as in FIG. It is formed.

  Further, the focus detection pixels 313 and 314 can be configured as shown in FIG. In FIG. 22, instead of projecting the shape of the photoelectric conversion unit 15 forward by the micro lens 36, the openings 43 and 44 of the light shielding mask 40 disposed immediately before the photoelectric conversion unit 15 are projected forward by the micro lens 36. Thus, a pair of distance measuring pupils is formed. In this case, the photoelectric conversion units 15 of the focus detection pixels 313 and 314 receive the light beams that have passed through the openings 43 and 44, respectively.

  In the image sensor 212 shown in FIGS. 20 and 21, the image pixel data at the focus detection pixel position can be obtained by interpolation processing based on the concept described in FIG. 17. However, in FIGS. 20 and 21, only one of the photoelectric conversion units to be paired is provided in each focus detection pixel, and therefore one of A1 and A2 in the calculation of P1 = A1 + A2 when obtaining the point image determination evaluation value P Uses the data of the photoelectric conversion unit of the focus detection pixel, and the other uses the average of the data of the photoelectric conversion unit of the two focus detection pixels sandwiching the focus detection pixel.

  In the above-described embodiment, no optical element is disposed between the image sensor 212 and the optical system, but a necessary optical element can be appropriately inserted. For example, an infrared cut filter, an optical low-pass filter, a half mirror, or the like may be installed.

  In the case of the configuration of the image sensor 212 as shown in FIG. 4, an optical low-pass filter is used so that the high-frequency cut effect of the optical low-pass filter is more effective in the alignment direction of the focus detection pixels than in the direction perpendicular to the alignment direction of the focus detection pixels 311. By setting the characteristics of the filter, it is possible to mitigate adverse effects on the focus detection accuracy when an image having a high frequency component enters between the focus detection pixels 311.

  In the imaging device 212 in the above-described embodiment, the imaging pixel 310 includes the Bayer color filter, but the configuration and arrangement of the color filter are not limited to this, and the complementary color filter (green: G, The present invention can also be applied to arrays other than yellow: Ye, magenta: Mg, cyan: Cy) and Bayer arrays. In the image sensor 212 in the above-described embodiment, the color filter is not provided in the focus detection pixels 311, 313, and 314. However, one of the color filters of the same color as the image pickup pixel 310 (for example, a green filter) is used. The present invention can be applied even when it is provided.

  Further, in the focus detection pixels 311, 313, and 314 in the above-described embodiment, an example in which the shape of the photoelectric conversion unit is rectangular is shown, but the shape of the photoelectric conversion unit of the focus detection pixels 311, 313, and 314 is limited to these. However, other shapes may be used, for example, a semicircle, an ellipse, or a polygon.

  In the focus detection pixels 211, 313, and 314 in the above-described embodiment, the size of the focus detection pixels 311, 313, and 314 is the size of four imaging pixels 310 (2 pixels in the horizontal direction and 2 pixels in the vertical direction). However, the size may be slightly larger or smaller than the size as long as the size fits in the region of the size of four of the imaging pixels 310.

  Note that the imaging apparatus according to the present invention is not limited to a digital still camera or a film still camera in which an interchangeable lens is mounted on the camera body as described above. For example, the present invention can be applied to a lens-integrated digital still camera, film still camera, or video camera. Furthermore, the present invention can be applied to a small camera module built in a mobile phone, a surveillance camera, a visual recognition device for a robot, an in-vehicle camera, and the like.

10, 19 Microlens, 11, 13, 14, 15 Photoelectric conversion unit, 29 Semiconductor circuit board, 30, 31, 32 Planarization layer, 33 color filter, 34 ND filter, 35 Inner lens, 36 Upper lens, 40 Light shielding mask , 43, 44 Aperture, 50, 59, 60 Microlens, 51, 53, 54, 63, 64 Photoelectric converter, 71 Imaging light beam, 73, 74, 83, 84 Focus detection light beam, 90 Exit pupil, 91 Exchange Optical axis of the lens, 92 Area that is substantially circumscribed by the distance measuring pupil, 93, 94 Distance pupil, 100 Shooting screen, 101 to 103, Focus detection area, 110 Imaging surface, 111 Focus detection pixel, 112 Micro lens, 113, 114 Photoelectric conversion unit, 120 Distance pupil plane, 123, 124 Distance pupil, 133, 134 Distance pupil distribution, 135 Superimposition unit, 20 Digital Still Camera, 202 Interchangeable Lens, 203 Camera Body, 204 Mount, 206 Lens Drive Control Device, 208 Zooming Lens, 209 Lens, 210 Focusing Lens, 211 Aperture, 212 Image Sensor, 213 Electrical Contact, 214 Body Drive Control Device, 215 liquid crystal display element driving circuit, 216 liquid crystal display element, 217 eyepiece, 219 memory card, 310 imaging pixel, 311 311a, 311b, 313, 314 focus detection pixel

Claims (12)

A plurality of types of imaging pixels having different spectral sensitivity characteristics and generating an output signal corresponding to an image formed by a light beam passing through an optical system, and a pair of images formed by a pair of light beams passing through the optical system An imaging device in which a plurality of focus detection pixels that generate a corresponding pair of output signals are two-dimensionally arranged;
Focus detection means for detecting a focus adjustment state of the optical system based on the pair of output signals;
When the plurality of types of imaging pixels are arranged at the positions of the plurality of focus detection pixels, an output signal generated by the plurality of types of imaging pixels is output to the plurality of types of imaging arranged around the plurality of focus detection pixels. Interpolating means for generating an interpolation value by interpolating based on the output signal of the pixel for use,
The plurality of types of imaging pixels are arranged according to a predetermined arrangement rule,
Each of the plurality of focus detection pixels is larger than each of the plurality of types of imaging pixels and is arranged on the imaging element according to an array that is continuous in an oblique direction. The imaging device according to claim 1,
The plurality of types of imaging pixels are arranged to form a square lattice,
Each of the plurality of focus detection pixels is substantially equal to the size of the entire four pixels formed by two adjacent vertical and horizontal pixels of the plurality of types of imaging pixels,
2. The imaging apparatus according to claim 1, wherein the oblique direction is a diagonal direction of the square lattice. The imaging device according to claim 1 or 2,
The plurality of types of imaging pixels are red pixels, green pixels, blue pixels,
The image pickup apparatus, wherein the predetermined arrangement rule is an arrangement rule based on a Bayer arrangement. The imaging device according to claim 3.
The oblique direction is an oblique direction in which the red pixel and the blue pixel are continuously arranged when the plural types of imaging pixels are arranged at the positions of the plurality of focus detection pixels. apparatus. In the imaging device according to any one of claims 1 to 4,
A point image exists at each position of the plurality of focus detection pixels based on output signals of the plurality of types of imaging pixels arranged around the plurality of focus detection pixels and output signals of the plurality of focus detection pixels. Further comprising point image determination means for determining whether or not
When the plurality of types of imaging pixels are arranged at the positions of the plurality of focus detection pixels in accordance with the predetermined arrangement rule, the interpolation unit calculates the interpolation value of each of the plurality of types of imaging pixels as the point image determination unit. An image pickup apparatus that performs calculation based on output signals of the plurality of types of image pickup pixels arranged around the plurality of focus detection pixels in accordance with a determination result obtained by the method. The imaging apparatus according to claim 5,
Direction determining means is further provided for determining a direction in which the image is highly continuous in the vicinity of each of the plurality of focus detection pixels based on output signals of the plurality of types of imaging pixels arranged around the plurality of focus detection pixels. ,
The interpolation means outputs the interpolation values of the plurality of types of imaging pixels arranged around the plurality of focus detection pixels according to the determination result by the point image determination means and the determination result by the direction determination means. An image pickup apparatus that calculates based on a signal and an output signal of the focus detection pixel. The imaging device according to claim 1,
An imaging apparatus, wherein a wavelength region indicating spectral sensitivity characteristics of each of the plurality of focus detection pixels is substantially equal to the entire wavelength region of visible light. The imaging apparatus according to claim 7,
Each of the plurality of focus detection pixels includes a microlens, a photoelectric conversion unit, and a dimming unit that attenuates the amount of incident light incident on the microlens until the incident light reaches the photoelectric conversion unit. An imaging apparatus comprising: The imaging device according to claim 1,
Each of the plurality of focus detection pixels includes a first microlens, a first photoelectric conversion unit, and an inner lens disposed between the first microlens and the first photoelectric conversion unit. ,
Each of the plurality of types of imaging pixels is composed of a second microlens and a second photoelectric conversion unit,
The first photoelectric conversion unit and the second photoelectric conversion unit are disposed on the same plane,
The image pickup apparatus, wherein the first microlens and the second microlens are arranged on substantially the same plane. The imaging device according to claim 1,
Each of the plurality of types of imaging pixels includes one photoelectric conversion unit,
Each of the plurality of focus detection pixels includes a pair of photoelectric conversion units,
Signal lines regularly wired to a plurality of photoelectric conversion units included in the plurality of types of imaging pixels and the focus detection pixels arranged two-dimensionally are around the pair of photoelectric conversion units. An image pickup apparatus that is wired so as to detour. The imaging device according to claim 1,
An imaging device, wherein the maximum length of the imaging pixel is 7 μm. A plurality of types of imaging pixels having different spectral sensitivity characteristics and generating an output signal corresponding to an image formed by a light beam passing through an optical system;
A plurality of focus detection pixels that generate a pair of output signals corresponding to a pair of images formed by a pair of light beams passing through the optical system for focus detection;
The plurality of types of imaging pixels and the plurality of focus detection pixels are two-dimensionally arranged,
The plurality of types of imaging pixels are arranged according to a predetermined arrangement rule,
Each of the plurality of focus detection pixels is larger than each of the plurality of types of imaging pixels, and is arranged according to an array that is continuous in an oblique direction.

Images