JP5228777B2 - Focus detection apparatus and imaging apparatus - Google Patents

Description

  According to the present invention, a pair of image signals corresponding to a pair of images formed by a pair of light beams passing through an optical system is generated by an array of a plurality of focus detection pixels, and an optical signal is generated based on a shift amount of the pair of image signals. The present invention relates to a focus detection device and an imaging device that detect a focus adjustment state of a system.

An image sensor in which an array of a plurality of focus detection pixels that generate a pair of image signals corresponding to a pair of images formed by a pair of light beams passing through an optical system is mixed in the imaging pixel, 2. Description of the Related Art An image pickup apparatus that generates an image signal and has a focus detection function that detects a focus adjustment state of an optical system based on a shift amount between a pair of image signals generated by a focus detection pixel is known (for example, a patent) Reference 1).
JP 2007-188931 A

  However, in the above-described conventional imaging device, when image shift detection is performed while including a defective focus detection pixel whose output is abnormal in some of the plurality of focus detection pixels, the focus detection accuracy deteriorates and the wrong focus is detected. There is a problem that detection results are output and focus detection becomes impossible. In order to prevent such a problem, if a measure for selecting an image sensor that does not include a defective focus detection pixel by individual inspection and incorporating it into the imaging apparatus is taken, the yield of the image sensor is reduced and the cost of the imaging apparatus is increased. The problem arises.

A focus detection device according to a first aspect of the present invention is an image pickup device in which an image pickup pixel and a focus detection pixel are two-dimensionally arranged, and a pair of images passing through an imaging optical system by the arrangement of the plurality of focus detection pixels. When there is an image sensor that generates a pair of image signals corresponding to a pair of images formed by a light beam and a defective focus detection pixel among the plurality of focus detection pixels, outputs of pixels around the defective focus detection pixel A pair generated by an interpolation means for calculating an output signal of the defective focus detection pixel based on a signal by interpolation, an output signal of the focus detection pixel and an output signal of the defective focus detection pixel calculated by the interpolation means. Detection means for detecting a relative shift amount of the pair of images based on the image signal of the image, and focus adjustment of the imaging optical system based on the shift amount of the pair of images detected by the detection means State Calculating means, and the interpolation means calculates an output signal of the defective focus detection pixel based on an output signal of the imaging pixel around the defective focus detection pixel by interpolation, and the imaging pixel A plurality of types of color filters are provided, and the interpolation means outputs an output signal of the defective focus detection pixel not provided with the color filter to an output signal of the imaging pixel provided with each of the color filters. It is characterized by calculating as a linear sum.
A focus detection device according to a second aspect of the present invention is an image pickup device in which an image pickup pixel and a focus detection pixel are arranged in a two-dimensional shape, and a pair of images passing through an imaging optical system by the arrangement of the plurality of focus detection pixels. When there is an image sensor that generates a pair of image signals corresponding to a pair of images formed by a light beam and a defective focus detection pixel among the plurality of focus detection pixels, outputs of pixels around the defective focus detection pixel A pair generated by an interpolation means for calculating an output signal of the defective focus detection pixel based on a signal by interpolation, an output signal of the focus detection pixel and an output signal of the defective focus detection pixel calculated by the interpolation means. Detection means for detecting a relative shift amount of the pair of images based on the image signal of the image, and focus adjustment of the imaging optical system based on the shift amount of the pair of images detected by the detection means State Calculating means for calculating, the interpolation means calculates the output signal of the defective focus detection pixel by interpolation based on the output signal of the imaging pixel and the focus detection pixel around the defective focus detection pixel, The image pickup pixel is provided with a plurality of types of color filters, and the interpolation means outputs the output signal of the defective focus detection pixel not provided with the color filter to the image pickup provided with each of the color filters. It is calculated as a linear sum of output signals of pixels.
A focus detection device according to a third aspect of the present invention is an image pickup device in which an image pickup pixel and a focus detection pixel are arranged two-dimensionally, and a pair of images passing through an imaging optical system by the arrangement of the plurality of focus detection pixels. The imaging element that generates a pair of image signals corresponding to a pair of images formed by a light beam, and the defective focus detection pixel in the plurality of focus detection pixels, the position located around the defective focus detection pixel Interpolation means for calculating the output signal of the defective focus detection pixel by interpolation based on the output signals of the imaging pixel and the focus detection pixel, and the defective focus detection pixel calculated by the output signal of the focus detection pixel and the interpolation means Based on a pair of image signals generated by the output signals of the two, based on a detection unit that detects a relative shift amount of the pair of images and a shift amount of the pair of images detected by the detection unit. Te, characterized in that it comprises a calculating means for calculating a focusing state of the imaging optical system.
A focus detection apparatus according to a fourth aspect of the present invention is an image pickup device in which an image pickup pixel and a focus detection pixel are two-dimensionally arranged, and the focus detection pixel is a pair of focus detection light beams that pass through an imaging optical system. The first and second focus detection pixels that receive one and the other respectively, and three or more of the first and second focus detection pixels are alternately arranged adjacent to each other in one direction. When an image sensor that outputs a pair of image signals corresponding to a pair of images by a focus detection light beam and the first focus detection pixel is a defect focus detection pixel, they are positioned at least on both sides of the defect focus detection pixel, respectively. Interpolating means for calculating the output signal of the defective focus detection pixel by interpolation based on the output signal of the first focus detection pixel adjacent to the second focus detection pixel, and the output signal of the focus detection pixel and the complement Detection means for detecting a relative shift amount of the pair of images based on a pair of image signals generated by an output signal of the defective focus detection pixel calculated by the means; and detected by the detection means Computing means for computing a focus adjustment state of the imaging optical system based on a deviation amount of the pair of images , wherein the interpolation means is a second focus located respectively on both sides of the defect focus detection pixel. The output signal of the defective focus detection pixel is calculated by interpolation based on the output signal of the first focus detection pixel adjacent to the detection pixel and the output signal of the imaging pixel around the defective focus detection pixel. Features.
A focus detection apparatus according to a fifth aspect of the present invention is an image pickup device in which image pickup pixels and focus detection pixels are two-dimensionally arranged, and a plurality of the focus detection pixels are arranged in one direction, and an imaging optical system Corresponding to a pair of images formed by a pair of focus detection light beams that pass through the imaging optical system. In the case where there is a defect in the output signal from the first photoelectric conversion unit of the focus detection pixel and the imaging device that outputs the pair of image signals, the first of the focus detection pixels located on both sides of the defective focus detection pixel, respectively. Interpolation means for calculating the output signal of the first photoelectric conversion section of the defective focus detection pixel based on the output signal of one photoelectric conversion section by interpolation, the output signal of the focus detection pixel and the interpolation means The defect focus detection image Based on a pair of image signals generated by the output signals of the first and second detection means, and detecting means for detecting a relative shift amount of the pair of images, and based on a shift amount of the pair of images detected by the detection means. Calculating means for calculating a focus adjustment state of the imaging optical system, and the interpolation means includes an output signal of a first photoelectric conversion unit of a focus detection pixel located on both sides of the defective focus detection pixel, respectively. The output signal of the first photoelectric conversion unit of the defective focus detection pixel is calculated by interpolation based on the output signal of the imaging pixel around the defective focus detection pixel .
A focus detection apparatus according to a sixth aspect of the present invention is an image pickup device in which an image pickup pixel and a focus detection pixel that receives a pair of focus detection light beams that pass through an imaging optical system are two-dimensionally arranged, and the focus detection An image sensor that outputs a pair of image signals corresponding to a pair of images formed by the pair of focus detection light beams, the pixels being arranged on a straight line adjacent to each other, and a defect in the focus detection pixels Interpolation that calculates the output signal of the defective focus detection pixel by interpolation based on the output signal of at least one of the surrounding imaging pixel and the focus detection pixel adjacent to the defective focus detection pixel when there is a focus detection pixel Based on a pair of image signals generated by the output signal of the focus detection pixel and the output signal of the defective focus detection pixel calculated by the interpolation means. Detecting means for detecting an amount, based on the shift amount of the pair of images detected by the detection means, and a calculation means for calculating a focusing state of the imaging optical system, the interpolation means, The output signal of the defective focus detection pixel is calculated by interpolation based on the output signals of both the imaging pixel and the focus detection pixel .

  According to the present invention, it is possible to normally perform focus detection even when an image sensor including defective focus detection pixels is used, and it is possible to prevent a decrease in yield of the image sensor.

  A lens interchangeable digital still camera will be described as an example of a focus detection device and an imaging device according to an embodiment. FIG. 1 is a cross-sectional view of a camera showing the configuration of the camera of one embodiment. A digital still camera 201 according to an embodiment includes an interchangeable lens 202 and a camera body 203, and the interchangeable lens 202 is attached to the camera body 203 via a mount unit 204. An interchangeable lens 202 having various photographing optical systems can be attached to the camera body 203 via a mount unit 204.

  The interchangeable lens 202 includes a lens 209, a zooming lens 208, a focusing lens 210, an aperture 211, a lens drive control device 206, and the like. The lens drive control device 206 includes a microcomputer (not shown), a memory, a drive control circuit, and the like. The lens drive control device 206 includes drive control for adjusting the focus of the focusing lens 210 and the aperture diameter of the aperture 211, zooming lens 208, and focusing. In addition to detecting the state of the lens 210 and the aperture 211, the lens information is transmitted and the camera information is received through communication with a body drive control device 214 described later. The aperture 211 forms an aperture having a variable aperture diameter at the center of the optical axis in order to adjust the amount of light and the amount of blur.

  The camera body 203 includes an imaging element 212, a body drive control device 214, a liquid crystal display element drive circuit 215, a liquid crystal display element 216, an eyepiece lens 217, a memory card 219, and the like. In the imaging element 212, imaging pixels are two-dimensionally arranged, and focus detection pixels are incorporated in portions corresponding to focus detection positions. The image sensor 212 has a non-volatile memory (details will be described later), and an individual inspection is performed before the image sensor 212 is incorporated into the camera body 203, and imaging such as presence / absence of defective pixels and position information of defective pixels is performed. Information specific to each element is written and stored in the memory. Details of the image sensor 212 will be described later.

  The body drive control device 214 includes a microcomputer, a memory, a drive control circuit, and the like, and controls the drive of the image sensor 212, reads out the image signal and the focus detection signal, performs the focus detection calculation based on the focus detection signal, and the focus of the interchangeable lens 202. The adjustment is repeated, and image signal processing and recording, camera operation control, and the like are performed. The body drive control device 214 communicates with the lens drive control device 206 via the electrical contact 213 to receive lens information and send camera information (defocus amount, aperture value, etc.).

  The liquid crystal display element 216 functions as an electric view finder (EVF). The liquid crystal display element driving circuit 215 displays a through image by the imaging element 212 on the liquid crystal display element 216, and the photographer can observe the through image through the eyepiece lens 217. The memory card 219 is an image storage that stores an image captured by the image sensor 212.

  A subject image is formed on the light receiving surface of the image sensor 212 by the light beam that has passed through the interchangeable lens 202. This subject image is photoelectrically converted by the image sensor 212, and an image signal and a focus detection signal are sent to the body drive control device 214.

  The body drive control device 214 calculates the defocus amount based on the focus detection signal from the focus detection pixel of the image sensor 212 and sends the defocus amount to the lens drive control device 206. The body drive control device 214 processes the image signal from the image sensor 212 to generate an image, stores the image in the memory card 219, and sends the through image signal from the image sensor 212 to the liquid crystal display element drive circuit 215. The through image is displayed on the liquid crystal display element 216. Further, the body drive control device 214 sends aperture control information to the lens drive control device 206 to control the aperture of the aperture 211.

  The lens drive controller 206 updates the lens information according to the focusing state, zooming state, aperture setting state, aperture opening F value, and the like. Specifically, the positions of the zooming lens 208 and the focusing lens 210 and the aperture value of the aperture 211 are detected, and lens information is calculated according to these lens positions and aperture values, or a lookup prepared in advance. Lens information corresponding to the lens position and aperture value is selected from the table.

  The lens drive control device 206 calculates a lens drive amount based on the received defocus amount, and drives the focusing lens 210 to the in-focus position according to the lens drive amount. Further, the lens drive control device 206 drives the diaphragm 211 in accordance with the received diaphragm value.

  FIG. 2 is a diagram showing a focus detection position on the imaging screen of the interchangeable lens 202, and an area in which an image is sampled on the imaging screen when a focus detection pixel row on the image sensor 212 described later performs focus detection (focus detection An example of an area and a focus detection position is shown. In this example, focus detection areas 101 to 103 are arranged at the center and three locations above and below the rectangular shooting screen 100. Focus detection pixels are linearly arranged in the longitudinal direction of the focus detection area indicated by a rectangle.

  FIG. 3 is a front view showing a detailed configuration of the image sensor 212, and shows an enlarged vicinity of the focus detection area 101 on the image sensor 212. Imaging pixels 310 are densely arranged in a two-dimensional square lattice pattern on the imaging element 212, and focus detection pixels 313 and 314 for focus detection are adjacent to each other on a vertical straight line at a position corresponding to the focus detection area 101. Are alternately arranged. Although not shown, the configuration in the vicinity of the focus detection areas 102 and 103 is the same as the configuration shown in FIG.

  As illustrated in FIG. 4, the imaging pixel 310 includes a microlens 10, a photoelectric conversion unit 11, and a color filter (not shown). There are three types of color filters, red (R), green (G), and blue (B), and the respective spectral sensitivities have the characteristics shown in FIG. In the image pickup device 212, image pickup pixels 310 having respective color filters are arranged in a Bayer array.

  As shown in FIG. 5A, the focus detection pixel 313 includes the microlens 10 and the photoelectric conversion unit 13, and the photoelectric conversion unit 13 has a semicircular shape. In addition, the focus detection pixel 314 includes the microlens 10 and the photoelectric conversion unit 14 as illustrated in FIG. 5B, and the photoelectric conversion unit 14 has a semicircular shape. The focus detection pixels 313 and the focus detection pixels 314 are alternately arranged in the vertical direction in the focus detection areas 101 to 103 as shown in FIGS. 2 and 3, and in this arrangement, the photoelectric conversion unit 13 of the focus detection pixels 313 is a microlens. The photoelectric conversion unit 14 of the focus detection pixel 314 is arranged at the lower half position of the microlens 10.

  The focus detection pixels 313 and 314 are not provided with a color filter to increase the amount of light, and the spectral characteristics of the focus detection pixels 313 and 314 include the spectral sensitivity of a photodiode that performs photoelectric conversion and the spectral characteristics of an infrared cut filter (not shown). Spectral characteristics (see FIG. 7). That is, the spectral characteristics are obtained by adding the spectral characteristics of the green pixel, the red pixel, and the blue pixel shown in FIG. 6, and the light wavelength region of the sensitivity includes the light wavelength regions of the sensitivity of the green pixel, the red pixel, and the blue pixel. ing.

  The focus detection pixels 313 and 314 for focus detection are arranged in a column where B and G of the imaging pixel 310 should be arranged. The focus detection pixels 313 and 314 for focus detection are arranged in a column in which B and G of the imaging pixel 310 are to be arranged because interpolation processing for obtaining an image signal for imaging at the position of the focus detection pixel is performed. This is because the interpolation error of the blue pixel is less noticeable than the interpolation error of the red pixel due to human visual characteristics when an interpolation error occurs in FIG.

  The photoelectric conversion unit 11 of the imaging pixel 310 is designed so as to receive all the light beams that pass through the exit pupil diameter (for example, F1.0) of the brightest interchangeable lens by the microlens 10. In addition, the photoelectric conversion units 13 and 14 of the focus detection pixels 313 and 314 are designed to have a shape such that the microlens 10 receives all light beams passing through a predetermined region (for example, F2.8) of the exit pupil of the interchangeable lens. The

  FIG. 8 is a cross-sectional view of the imaging pixel 310. In the imaging pixel 310, the microlens 10 is disposed in front of the photoelectric conversion unit 11 for imaging, and the shape of the photoelectric conversion unit 11 is projected forward by the microlens 10. The photoelectric conversion unit 11 is formed on the semiconductor circuit substrate 29. A color filter (not shown) is arranged between the microlens 10 and the photoelectric conversion unit 11.

  FIG. 9A is a cross-sectional view of the focus detection pixel 313. In the focus detection pixel 313 disposed in the focus detection area 101 at the center of the screen, the microlens 10 is disposed in front of the photoelectric conversion unit 13, and the shape of the photoelectric conversion unit 13 is projected forward by the microlens 10. The photoelectric conversion unit 13 is formed on the semiconductor circuit substrate 29, and the microlens 10 is integrally and fixedly formed thereon by a semiconductor image sensor manufacturing process. Note that the cross-sectional structure of the focus detection pixels 313 arranged in the focus detection areas 102 and 103 at the top and bottom of the screen is the same as the cross-sectional structure shown in FIG.

  FIG. 9B is a cross-sectional view of the focus detection pixel 314. In the focus detection pixel 314 disposed in the focus detection area 101 in the center of the screen, the microlens 10 is disposed in front of the photoelectric conversion unit 14, and the shape of the photoelectric conversion unit 14 is projected forward by the microlens 10. The photoelectric conversion unit 14 is formed on the semiconductor circuit substrate 29, and the microlens 10 is integrally and fixedly formed thereon by the manufacturing process of the semiconductor image sensor. Note that the cross-sectional structure of the focus detection pixels 314 arranged in the focus detection areas 102 and 103 at the top and bottom of the screen is the same as the cross-sectional structure shown in FIG.

  FIG. 10 shows a configuration of a pupil division type phase difference detection type focus detection optical system using a microlens. The focus detection pixel portion is shown in an enlarged manner. In the figure, reference numeral 90 denotes an exit pupil set at a distance d forward from the microlens arranged on the planned imaging plane of the interchangeable lens 202 (see FIG. 1). This distance d is a distance determined according to the curvature and refractive index of the microlens, the distance between the microlens and the photoelectric conversion unit, and is referred to as a distance measuring pupil distance in this specification. 91 is an optical axis of the interchangeable lens, 10a to 10d are microlenses, 13a, 13b, 14a, and 14b are photoelectric conversion units, 313a, 313b, 314a, and 314b are focus detection pixels, and 73, 74, 83, and 84 are focus detection light fluxes. It is.

  Reference numeral 93 denotes an area of the photoelectric conversion units 13a and 13b projected by the microlenses 10a and 10c, and is referred to as a distance measuring pupil in this specification. In FIG. 10, an elliptical region is shown for easy understanding, but the shape of the photoelectric conversion unit is actually an enlarged projection shape. Similarly, 94 is an area of the photoelectric conversion units 14a and 14b projected by the microlenses 10b and 10d, and is called a distance measuring pupil in this specification. In FIG. 10, an elliptical region is shown for easy understanding, but the shape of the photoelectric conversion unit is actually an enlarged projection shape.

  In FIG. 10, four focus detection pixels 313a, 313b, 314a, and 314b adjacent to the photographing optical axis are schematically illustrated. However, other focus detection pixels in the focus detection area 101 also have a peripheral portion of the screen. Also in the focus detection pixels in the focus detection areas 102 and 103, the photoelectric conversion units are configured to receive the light beams coming from the corresponding distance measurement pupils 93 and 94 to the respective microlenses. The arrangement direction of the focus detection pixels is made to coincide with the arrangement direction of the pair of distance measuring pupils, that is, the arrangement direction of the pair of photoelectric conversion units.

  The microlenses 10a to 10d are disposed in the vicinity of the planned imaging plane of the interchangeable lens 202 (see FIG. 1), and the shapes of the photoelectric conversion units 13a, 13b, 14a, and 14b disposed behind the microlenses 10a to 10d. Is projected onto the exit pupil 90 separated from the microlenses 10a to 10c by the distance measurement pupil distance d, and the projection shape forms distance measurement pupils 93 and 94. That is, the relative relationship between the microlens and the photoelectric conversion unit in each focus detection pixel is such that the projection shape (ranging pupils 93 and 94) of the photoelectric conversion unit of each focus detection pixel matches on the exit pupil 90 at the projection distance d. Thus, the projection position of the photoelectric conversion unit in each focus detection pixel is determined.

  The photoelectric conversion unit 13a passes through the distance measuring pupil 93 and outputs a signal corresponding to the intensity of the image formed on the microlens 10a by the light beam 73 directed to the microlens 10a. Similarly, the photoelectric conversion unit 13b outputs a signal corresponding to the intensity of the image formed on the microlens 10c by the light beam 83 passing through the distance measuring pupil 93 and directed to the microlens 10c. Further, the photoelectric conversion unit 14a outputs a signal corresponding to the intensity of the image formed on the microlens 10b by the light beam 74 passing through the distance measuring pupil 94 and directed to the microlens 10b. Similarly, the photoelectric conversion unit 14b outputs a signal corresponding to the intensity of the image formed on the microlens 10d by the light beam 84 passing through the distance measuring pupil 94 and directed to the microlens 10d.

  A large number of the two types of focus detection pixels described above are arranged in a straight line, and the output of the photoelectric conversion unit of each pixel is grouped into a distance measurement pupil 93 and an output group corresponding to the distance measurement pupil 94, thereby measuring the distance measurement pupil 93 and the measurement pupil 93. Information on the intensity distribution of the pair of images formed on the pixel array by the focus detection light beams that respectively pass through the distance pupil 94 is obtained. By applying an image shift detection calculation process (correlation calculation process, phase difference detection process), which will be described later, to this information, the image shift amount of a pair of images is detected by a so-called pupil division type phase difference detection method. Furthermore, by performing a conversion operation according to the center of gravity distance of the pair of distance measuring pupils to the image shift amount, the current image plane relative to the planned image plane (the focus corresponding to the position of the microlens array on the planned image plane) The deviation (defocus amount) of the imaging plane at the detection position is calculated.

  FIG. 11 is a diagram showing the concept of the circuit configuration of the image sensor 212. The image sensor 212 is configured as a CMOS image sensor. In FIG. 11, for ease of explanation, the circuit configuration of the image sensor 212 is simply described as a layout of 8 pixels in the horizontal direction and 4 pixels in the vertical direction. In the vertical direction, focus detection pixels (indicated by ◯ in the figure) are arranged in the second and sixth columns, and imaging pixels (indicated by □ in the figure) are arranged in the other columns.

  The line memory 320 is a buffer that samples and holds the pixel signals of the pixels for one column and temporarily holds them, and the control signal ΦS that the horizontal scanning circuit 301 generates the pixel signals of the same column output to the signal line 501. Simultaneously sample and hold based on Note that the pixel signals held in the line memory 320 are reset in synchronization with the rise of the control signals ΦH1 to ΦH8.

  Outputs of pixel signals from the imaging pixels and focus detection pixels are controlled independently for each column by control signals (ΦH1 to ΦH8) generated by the horizontal scanning circuit 301. The pixel signals of the pixels in the column selected by the control signals (ΦH1 to ΦH8) are output to the signal line 501. The pixel signals held in the line memory 320 are sequentially transferred to the output circuit 330 by the control signals (ΦV1 to ΦV4) generated by the vertical scanning circuit 302, amplified by the output circuit 330 with a preset amplification degree, and output to the outside. Is done. After the pixel signal is sampled and held, the imaging pixel and the focus detection pixel are reset by the control signals (ΦR1 to ΦR8) generated by the reset circuit 303, and charge accumulation for the next pixel signal is started.

  The defective pixel memory 304 is a nonvolatile memory such as an EEPROM that stores position information of defective pixels (defective imaging pixels and defective focus detection pixels). When a defective pixel is found by inspection at the manufacturing stage, position information of the defective pixel is written and stored in the defective pixel memory 304. In this embodiment, an example in which the defective pixel memory 304 is added to the semiconductor substrate of the image pickup device 212 by a semiconductor manufacturing process is shown. However, the image pickup device 212 is usually enclosed in a dustproof case, and the image pickup device unit (not used). Since it is installed in the camera as shown in the figure, a memory may be installed in the image sensor unit to store data of defective pixels. Or you may make it memorize | store the data of a defective pixel in the memory which memorize | stores the various data with which a camera is equipped.

  FIG. 12 is a flowchart illustrating an imaging operation of the digital still camera (imaging device) according to the embodiment. When the power of the camera is turned on in step 100, the body drive control device 214 starts the imaging operation after step 110. In step 110, the image pickup pixel data is thinned out and displayed on the electronic viewfinder. In subsequent step 120, a pair of image data corresponding to the pair of images is read from the focus detection pixel array. The focus detection area is assumed to be selected in advance by the photographer using one of the focus detection areas 101 to 103 using a focus detection area selection member (not shown).

  In step 125, defective focus detection pixel information is read from the defective pixel memory 304 (see FIG. 11) built in the image sensor 212, and when there is a defective focus detection pixel in the selected focus detection area, the defective focus detection pixel is detected. Output data when the defect focus detection pixel is normal is estimated by interpolation calculation (detailed later).

  In step 130, based on the read pair of image data (normal focus detection pixel output data and defect focus detection pixel output data estimated by interpolation in step 125), an image shift detection calculation process (correlation calculation) to be described later. Process) to calculate the image shift amount and convert it to a defocus amount. In step 140, it is checked whether or not the focus is close, that is, whether or not the absolute value of the calculated defocus amount is within a predetermined value. If it is determined that the lens is not in focus, the process proceeds to step 150, where the defocus amount is transmitted to the lens drive control device 206, and the focusing lens 210 of the interchangeable lens 202 is driven to the focus position. Then, it returns to step 110 and repeats the operation | movement mentioned above.

  Even when focus detection is impossible, the process branches to this step, a scan drive command is transmitted to the lens drive control device 206, and the focusing lens 210 of the interchangeable lens 202 is driven to scan from infinity to the nearest. Then, it returns to step 110 and repeats the operation | movement mentioned above.

  If it is determined in step 140 that the focus is close to the in-focus state, the process proceeds to step 160, where it is determined whether or not a shutter release has been performed by operating a shutter button (not shown). If it is determined that the shutter release has not been performed, the process returns to step 110 to repeat the above-described operation. On the other hand, if it is determined that the shutter release has been performed, the process proceeds to step 170, where an aperture adjustment command is transmitted to the lens drive control device 206, and the aperture value of the interchangeable lens 202 is controlled to a control F value (F set by the photographer or automatically). Value). When the aperture control is finished, the image sensor 212 is caused to perform an imaging operation, and image data is read from the image pickup pixel 310 and all the focus detection pixels 313 and 314 of the image pickup element 212.

  In step 180, pixel data of each pixel position in the focus detection pixel column is subjected to pixel interpolation based on data of imaging pixels around the focus detection pixel. In the subsequent step 190, image data composed of the imaged pixel data and the interpolated data is stored in the memory card 219, and the process returns to step 110 to repeat the above-described operation.

Details of the image shift detection calculation process (correlation calculation process) in step 130 of FIG. 12 will be described. The pair of images detected by the focus detection pixels has a possibility that the distance measurement pupil is displaced by the aperture of the lens and the balance of the light quantity is lost. Perform the operation. A correlation calculation shown in the equation (1) is performed on a pair of data strings (A11 to A1M, A21 to A2M: M is the number of data) read from the focus detection pixel string, and a correlation amount C (k) is calculated.
C (k) = Σ | A1n · A2n + 1 + k−A2n + k · A1n + 1 | (1)
In equation (1), the Σ operation is accumulated for n. The range taken by n is limited to a range in which data of A1n, A1n + 1, A2n + k, and A2n + 1 + k exist according to the image shift amount k. The image shift amount k is an integer and is a relative shift amount with the data interval of the data string as a unit.

As shown in FIG. 13A, the calculation result of the expression (1) shows that the correlation amount C (k) is minimal at a shift amount with high correlation between a pair of data (k = kj = 2 in FIG. 13A). (The smaller the value, the higher the degree of correlation). The shift amount x that gives the minimum value C (k) with respect to the continuous correlation amount is obtained by using the three-point interpolation method according to the following equations (2) to (5).
x = kj + D / SLOP (2),
C (x) = C (kj) − | D | (3),
D = {C (kj-1) -C (kj + 1)} / 2 (4),
SLOP = MAX {C (kj + 1) -C (kj), C (kj-1) -C (kj)} (5)

  Whether or not the shift amount x calculated by the equation (2) is reliable is determined as follows. As shown in FIG. 13B, when the degree of correlation between a pair of data is low, the value of the interpolated minimum value C (x) of the correlation amount increases. Therefore, when C (x) is equal to or greater than a predetermined threshold value, it is determined that the calculated shift amount has low reliability, and the calculated shift amount x is canceled. Alternatively, in order to normalize C (x) with the contrast of data, when the value obtained by dividing C (x) by SLOP that is proportional to the contrast is equal to or greater than a predetermined value, the reliability of the calculated shift amount Is determined to be low, and the calculated shift amount x is canceled. Alternatively, when SLOP that is a value proportional to the contrast is equal to or less than a predetermined value, it is determined that the subject has low contrast and the reliability of the calculated shift amount is low, and the calculated shift amount x is canceled. As shown in FIG. 13C, when the correlation between the pair of data is low and there is no drop in the correlation amount C (k) between the shift ranges kmin to kmax, the minimum value C (x) is obtained. In such a case, it is determined that the focus cannot be detected.

  The correlation calculation formula is not limited to the above formula (1), and it is a type that can maintain the image shift detection accuracy with respect to the light quantity balance even when the distance measurement pupil is displaced by the aperture of the lens and the light quantity balance is lost. Any arithmetic expression may be used as long as it is a correlation arithmetic expression.

If it is determined that the calculated shift amount x is reliable, it is converted into an image shift amount shft according to equation (6).
shft = PY · x (6)
In the equation (6), PY is a detection pitch. The image shift amount calculated by the equation (6) is multiplied by a predetermined conversion coefficient k to be converted into a defocus amount def.
def = k · shft (7)

  Next, the interpolation calculation process of the output data of the defective focus detection pixel in step 125 in FIG. 12 will be described. 14 and 15 are partially enlarged views of the image sensor 212 shown in FIG. In these drawings, R, G, and B added to each pixel indicate the spectral characteristics of each imaging pixel, and A0 * (* = 0, 1, 2,...) Indicates the focus detection pixel 313 and A1 *. (* = 0, 1, 2,...) Indicate the focus detection pixels 314, respectively.

<< Example 1 of interpolation calculation of defective focus detection pixels >>
FIG. 14 shows a pixel arrangement of 5 vertical pixels and 5 horizontal pixels centering on the defective focus detection pixel when the defective focus detection pixel is the focus detection pixel 313. As shown in the following equation (8), the interpolation data A0x of the defect focus detection pixel to be obtained is an estimated value A0av of the defect focus detection pixel estimated from the data of the normal focus detection pixel located between the defect focus detection pixels. It is obtained by averaging the estimated value A0w of the defect focus detection pixel estimated from the data of normal imaging pixels around the defect focus detection pixel.
A0x = (A0av + A0w) / 2 (8)
In equation (8), A0av in the first term on the right side is a focus detection pixel of the same type as the defect focus detection pixel as shown in equation (9) below, and is a normal focus at a position sandwiching the defect focus detection pixel. This is the average data of the detection pixel data.
A0av = (A00 + A01) / 2 (9)

In equation (8), A0w in the second term on the right side is an estimated value of defective focus detection pixel data derived based on the equivalent relationship between the focus detection pixel data and the imaging pixel data. That is, the imaging pixel and the focus detection pixel have spectral characteristics as shown in FIGS. 6 and 7, and the output of the focus detection pixel is approximately expressed as a linear sum of the output of the imaging pixel. In general, when the sum of the data of the focus detection pixels 313 and 314 is the data A of the focus detection pixel, there is an approximation between the data A of the focus detection pixel and the data R, G, and B of the imaging pixel. The following equation (10) is established.
A = Kr · R + Kg · G + Kb · B (10)
In the equation (10), Kr, Kg, and Kb are predetermined coefficients, which are determined by actual measurement.

When A, R, G, and B in the equation (10) are locally estimated from the data of the focus detection pixels and the imaging pixels around the defect focus detection pixel, the following equation (11) is obtained.
A = A0w + (A10 + A11) / 2
R = (R00 + R01 + R10 + R11) / 4
G = (G00 + G01) / 2
B = (B00 + B01) / 2 (11)

When the equation (11) is substituted into the equation (10) and rearranged, the estimated value A0w is obtained by the following equation (12).
A0w = Kr. (R00 + R01 + R10 + R11) / 4 + Kg. (G00 + G01) / 2 + Kb. (B00 + B01) / 2- (A10 + A11) / 2 (12)

  In the first interpolation calculation example, the defect focus detection pixel interpolation data A0x, the defect focus detection pixel estimation value A0av estimated from the normal focus detection pixel data at the position sandwiching the defect focus detection pixel, and the defect focus detection pixel An example is shown in which it is obtained as an average with the estimated value A0w of the defect focus detection pixel estimated from the data of the surrounding normal imaging pixels. According to this calculation method, the interpolation data A0x of the defective focus detection pixel can be accurately estimated. The two estimated values A0av and A0w may be weighted and averaged (A0x = K1 · A0av + K2 · A0w, K1 and K2 are weights). Further, although the estimation accuracy is slightly lowered, the interpolation data A0x of the defect focus detection pixel is also used as the defect focus detection pixel estimation value A0av estimated from the data of the normal focus detection pixel at the position sandwiching the defect focus detection pixel. Good (A0x = A0av). Alternatively, the defect focus detection pixel interpolation data A0x may be the defect focus detection pixel estimated value A0w estimated from the data of normal imaging pixels around the defect focus detection pixel (A0x = A0w).

<< Example 2 of interpolation calculation of defective focus detection pixels >>
FIG. 15 shows a pixel arrangement of 5 vertical pixels and 5 horizontal pixels centering on the defective focus detection pixel when the defective focus detection pixel is the focus detection pixel 314. The interpolation data A1x of the defect focus detection pixel to be obtained is an estimated value A1av of the defect focus detection pixel estimated from the data of the normal focus detection pixel at the position sandwiching the defect focus detection pixel, as shown in the following equation (13): It can be obtained by averaging the estimated value A1w of the defect focus detection pixel estimated from the data of normal imaging pixels around the defect focus detection pixel.
A1x = (A1av + A1w) / 2 (13)
In equation (13), A1av in the first term on the right side is the same type of focus detection pixel as the defect focus detection pixel as shown in the following equation (14), and is a normal focus at a position sandwiching the defect focus detection pixel. This is the average data of the detection pixel data.
A1av = (A10 + A11) / 2 (14)

In equation (13), A1w in the second term on the right side is an estimated value of defect focus detection pixel data derived based on the equivalent relationship between the focus detection pixel data and the imaging pixel data. As described above, since the output of the focus detection pixel is approximately expressed as a linear sum of the output of the imaging pixel, if the sum of the data of the focus detection pixels 313 and 314 is the data A of the focus detection pixel, the focus detection pixel (10) is approximately established between the data A and the sum of the data R, G, and B of the imaging pixel. In the equation (10), when R, G, and B are locally estimated from the data of the focus detection pixels and the imaging pixels around the defect focus detection pixel, the following equation (15) is obtained.
A = A1w + (A00 + A01) / 2
R = (R00 + R01) / 2
G = (G00 + G01 + G10 + G11) / 4
B = (B00 + B01 + B10 + B11) / 4 (15)

When the equation (15) is substituted into the equation (10) and rearranged, the estimated value A1w is obtained by the following equation (16).
A1w = Kr · (R00 + R01) / 2 + Kg · (G00 + G01 + G10 + G11) / 4 + Kb · (B00 + B01 + B10 + B11) / 2- (A00 + A01) / 2 (16)

  In the interpolation calculation example 2, the defect focus detection pixel interpolation data A1x, the defect focus detection pixel estimated value A1av estimated from the normal focus detection pixel data at the position sandwiching the defect focus detection pixel, and the defect focus detection pixel An example is shown in which it is obtained as an average with the estimated value A1w of the defect focus detection pixel estimated from the data of the surrounding normal imaging pixels. According to this calculation method, the interpolation data A1x of the defective focus detection pixel can be accurately estimated. The two estimated values A1av and A1w may be weighted and averaged (A1x = K1 · A1av + K2 · A1w, K1 and K2 are weights). Further, although the estimation accuracy is slightly lowered, the interpolation data A1x of the defect focus detection pixel is also used as the defect focus detection pixel estimation value A1av estimated from the data of the normal focus detection pixel located between the defect focus detection pixels. Good (A1x = A1av). Alternatively, the defect focus detection pixel interpolation data A1x may be an estimated value A1w of the defect focus detection pixel estimated from data of normal imaging pixels around the defect focus detection pixel (A1x = A1w).

<< Other Embodiments of the Invention >>
The arrangement of the focus detection areas in the image sensor is not limited to that shown in FIG. 2, and the focus detection areas can be arranged in the diagonal direction and in other positions in the horizontal and vertical directions.

  In the imaging device 212 of the embodiment shown in FIG. 3, an example in which the focus detection pixels 313 and 314 include one photoelectric conversion unit in one pixel is shown. However, a pair of photoelectric conversion units is provided in one pixel. May be provided. FIG. 16 is a partial enlarged view of an image sensor 212A of a modified example corresponding to the image sensor 212 shown in FIG. 3, and the focus detection pixel 311 includes a pair of photoelectric conversion units in one pixel. The focus detection pixel 311 shown in the drawing performs a function corresponding to the pair of the focus detection pixel 313 and the focus detection pixel 314 shown in FIG.

  The focus detection pixel 311 includes a microlens 10 and a pair of photoelectric conversion units 13 and 14 as shown in FIG. The focus detection pixel 311 is not provided with a color filter in order to increase the amount of light, and its spectral characteristic is a spectral that combines the spectral sensitivity of a photodiode that performs photoelectric conversion and the spectral characteristic of an infrared cut filter (not shown). Characteristics (see FIG. 7). That is, the spectral characteristics are obtained by adding the spectral characteristics of the green pixel, the red pixel, and the blue pixel shown in FIG. 6, and the light wavelength region of the sensitivity includes the light wavelength regions of the sensitivity of the green pixel, the red pixel, and the blue pixel. ing.

  FIG. 18 is a diagram for explaining a focus detection operation of the pupil division method by the focus detection pixels of the image sensor 212A shown in FIG. In the figure, reference numeral 90 denotes an exit pupil set at a distance d in front of the microlens arranged on the planned imaging plane of the interchangeable lens. Here, the distance d is a distance determined according to the curvature and refractive index of the microlens, the distance between the microlens and the photoelectric conversion unit, and is hereinafter referred to as a distance measuring pupil distance. 91 is an optical axis of the interchangeable lens, 50 and 60 are microlenses, (53, 54), (63, 64) are photoelectric conversion units of a pair of focus detection pixels, and 73, 74, 83, and 84 are focus detection light beams. is there.

  Reference numeral 93 denotes an area of the photoelectric conversion units 53 and 63 projected by the microlenses 50 and 60, which will be referred to as a distance measuring pupil below. Similarly, 94 is an area of the photoelectric conversion units 54 and 64 projected by the microlenses 50 and 60, and is hereinafter referred to as a distance measuring pupil. In FIG. 18, a focus detection pixel (consisting of a microlens 50 and a pair of photoelectric conversion units 53 and 54) on the optical axis 91 and an adjacent focus detection pixel (from the microlens 60 and a pair of photoelectric conversion units 63 and 64). In the focus detection pixels arranged in the periphery on the imaging surface, the pair of photoelectric conversion units arrive at each microlens from the pair of distance measuring pupils 93 and 94, respectively. Receives light flux. The arrangement direction of the focus detection pixels is made to coincide with the arrangement direction of the pair of distance measurement pupils.

  The microlenses 50 and 60 are disposed in the vicinity of the planned imaging plane of the optical system, and the shape of the pair of photoelectric conversion units 53 and 54 disposed behind the microlens 50 disposed on the optical axis 91 is formed. Projection is performed on the exit pupil 90 separated from the microlenses 50 and 60 by the distance measurement pupil distance d, and the projection shape forms distance measurement pupils 93 and 94. Further, the microlens 60 disposed adjacent to the microlens 50 projects the shape of the pair of photoelectric conversion units 63 and 64 disposed behind the microlens 50 onto the exit pupil 90 separated by the distance measuring pupil distance d. The projection shape forms distance measuring pupils 93 and 94. That is, the positions of the microlens and the photoelectric conversion unit of each pixel so that the projection shapes (ranging pupils 93 and 94) of the photoelectric conversion unit of each focus detection pixel match on the exit pupil 90 at the distance measurement pupil distance d. The relationship has been determined.

  The photoelectric conversion unit 53 outputs a signal corresponding to the intensity of the image formed on the microlens 50 by the focus detection light beam 73 passing through the distance measuring pupil 93 and traveling toward the microlens 50. In addition, the photoelectric conversion unit 54 outputs a signal corresponding to the intensity of the image formed on the microlens 50 by the focus detection light beam 74 that passes through the distance measuring pupil 94 and travels toward the microlens 50. Similarly, the photoelectric conversion unit 63 outputs a signal corresponding to the intensity of the image formed on the microlens 60 by the focus detection light beam 83 that passes through the distance measuring pupil 93 and travels toward the microlens 60. In addition, the photoelectric conversion unit 64 outputs a signal corresponding to the intensity of the image formed on the microlens 60 by the focus detection light beam 84 that passes through the distance measuring pupil 94 and travels toward the microlens 60.

  A large number of such focus detection pixels are arranged in a straight line, and the output of the pair of photoelectric conversion units of each pixel is grouped into an output group corresponding to the distance measurement pupil 93 and the distance measurement pupil 94, whereby the distance measurement pupil 93 and Information on the intensity distribution of the pair of images formed on the focus detection pixel array by the focus detection light fluxes that pass through the distance measuring pupils 94 is obtained. By applying an image shift detection calculation process (correlation calculation process, phase difference detection process), which will be described later, to this information, an image shift amount of a pair of images is detected by a so-called pupil division method. Furthermore, by applying a predetermined conversion process to the image shift amount, the deviation of the current imaging plane (imaging plane at the focus detection position corresponding to the position of the microlens array on the planned imaging plane) with respect to the planned imaging plane (Defocus amount) is calculated.

  An interpolation calculation of defective focus detection pixels for an image sensor in which focus detection pixels having a pair of photoelectric conversion units are arranged for one microlens as in the image sensor 212A illustrated in FIG. 16 will be described. 19 and 20 are enlarged views of the vicinity of the defective focus detection pixel of the image sensor 212A shown in FIG. In these figures, R, G, and B added to each pixel indicate spectral characteristics of each imaging pixel, and A0 * (* = 0, 1, 2,...) Indicates a pair of photoelectric conversions of the focus detection pixel. .., A1 * (* = 0, 1, 2,...) Indicates the other photoelectric conversion unit of the focus detection pixel.

<< Example 3 of interpolation calculation of defective focus detection pixels >>
FIG. 19 shows a case where the output of one of the pair of photoelectric conversion units of the defect focus detection pixel is abnormal and the output of the other photoelectric conversion unit is normal, and the defect focus detection pixel is an imaging pixel. This shows a pixel arrangement of 5 pixels in the vertical direction and 5 pixels in the horizontal direction with the defective focus detection pixel at the center when it is at the position of the blue pixel in the array. Interpolation data A0x of one photoelectric conversion unit of the defect focus detection pixel to be obtained is defect focus detection estimated from data of normal focus detection pixels at positions sandwiching the defect focus detection pixel as shown in the following equation (17). It is obtained by averaging the estimated pixel value A0av and the estimated value A0w of the defect focus detection pixel estimated from the data of the normal imaging pixels around the defect focus detection pixel.
A0x = (A0av + A0w) / 2 (17)
In equation (17), A0av in the first term on the right side is the same type of photoelectric conversion unit as the abnormal output photoelectric conversion unit of the defect focus detection pixel as shown in the following equation (18), and the defect focus detection pixel is It is the average data of the data of the photoelectric conversion part of the normal focus detection pixel in the sandwiched position.
A0av = (A00 + A01) / 2 (18)

In equation (17), A0w in the second term on the right side is an estimated value of defective focus detection pixel data derived based on the equivalent relationship between the focus detection pixel data and the imaging pixel data. As described above, since the output of the focus detection pixel is approximately expressed as a linear sum of the output of the imaging pixel, the sum of the output data of the pair of photoelectric conversion units of the focus detection pixel 311 is the data A of the focus detection pixel. Then, the expression (10) is approximately established between the data A of the focus detection pixel and the sum of the data R, G, and B of the imaging pixel. In the equation (10), when R, G, and B are locally estimated from the data of the focus detection pixels and the imaging pixels around the defect focus detection pixel, the following equation (19) is obtained.
A = A0w + A10,
R = (R00 + R01 + R10 + R11) / 4
G = (G00 + G01) / 2
B = (B00 + B01) / 2 (19)

When the equation (19) is substituted into the equation (10) and rearranged, the estimated value A0w is obtained by the equation (20).
A0w = Kr. (R00 + R01 + R10 + R11) / 4 + Kg. (G00 + G01) / 2 + Kb. (B00 + B01) / 2-A10 (20)

  In the interpolation calculation example 3, the defect focus detection pixel interpolation data A0x is obtained by estimating the defect focus detection pixel estimated value A0av estimated from the data of the normal focus detection pixel located between the defect focus detection pixels and the defect focus detection pixel. An example is shown in which it is obtained as an average with the estimated value A0w of the defect focus detection pixel estimated from the data of the surrounding normal imaging pixels. According to this calculation method, the interpolation data A0x of the defective focus detection pixel can be accurately estimated. The two estimated values A0av and A0w may be weighted and averaged (A0x = K1 · A0av + K2 · A0w, K1 and K2 are weights). Further, although the estimation accuracy is slightly lowered, the interpolation data A0x of the defect focus detection pixel is also used as the defect focus detection pixel estimation value A0av estimated from the data of the normal focus detection pixel at the position sandwiching the defect focus detection pixel. Good (A0x = A0av). Alternatively, the defect focus detection pixel interpolation data A0x may be the defect focus detection pixel estimated value A0w estimated from the data of normal imaging pixels around the defect focus detection pixel (A0x = A0w).

<< Example 4 of interpolation calculation of defective focus detection pixels >>
FIG. 20 shows a case where the output of one of the pair of photoelectric conversion units of the defect focus detection pixel is abnormal and the output of the other photoelectric conversion unit is normal, and the defect focus detection pixel is an imaging pixel. This shows a pixel arrangement of 5 pixels in the vertical direction and 5 pixels in the horizontal direction with the defective focus detection pixel at the center when the green pixel position is in the array. One interpolation data A1x of the defect focus detection pixel to be obtained is the estimated value of the defect focus detection pixel estimated from the data of the normal focus detection pixel at the position sandwiching the defect focus detection pixel as shown in the following equation (21). It is obtained by averaging A1av and the estimated value A1w of the defect focus detection pixel estimated from the data of normal imaging pixels around the defect focus detection pixel.
A1x = (A1av + A1w) / 2 (21)
In equation (21), A1av in the first term on the right side is the same type of photoelectric conversion unit as the abnormal output photoelectric conversion unit of the defect focus detection pixel as shown in equation (22), and sandwiches the defect focus detection pixel. It is the average data of the data of the photoelectric conversion part of the normal focus detection pixel in a position.
A1av = (A10 + A11) / 2 (22)

In equation (21), A1w in the second term on the right side is an estimated value of defect focus detection pixel data derived based on the equivalent relationship between the focus detection pixel data and the imaging pixel data. As described above, since the output of the focus detection pixel is approximately expressed as a linear sum of the output of the imaging pixel, the sum of the output data of the pair of photoelectric conversion units of the focus detection pixel 311 is the data A of the focus detection pixel. Then, the expression (10) is approximately established between the data A of the focus detection pixel and the sum of the data R, G, and B of the imaging pixel. In the equation (10), when R, G, and B are locally estimated from the data of the focus detection pixels and the imaging pixels around the defect focus detection pixel, the following equation (23) is obtained.
A = A1w + A00,
R = (R00 + R01) / 2
G = (G00 + G01 + G10 + G11) / 4
B = (B00 + B01 + B10 + B11) / 4 (23)

When the equation (23) is substituted into the equation (10) and rearranged, the estimated value A1w is obtained by the equation (24).
A1w = Kr · (R00 + R01) / 2 + Kg · (G00 + G01 + G10 + G11) / 4 + Kb · (B00 + B01 + B10 + B11) / 2−A00 (24)

  In the interpolation calculation example 4, the defect focus detection pixel interpolation data A1x, the defect focus detection pixel estimated value A1av estimated from the data of the normal focus detection pixel at the position sandwiching the defect focus detection pixel, and the defect focus detection pixel An example is shown in which it is obtained as an average with the estimated value A1w of the defect focus detection pixel estimated from the data of the surrounding normal imaging pixels. According to this calculation method, the interpolation data A1x of the defective focus detection pixel can be accurately estimated. The two estimated values A1av and A1w may be weighted and averaged (A1x = K1 · A1av + K2 · A1w, K1 and K2 are weights). Further, although the estimation accuracy is slightly lowered, the interpolation data A1x of the defect focus detection pixel is also used as the defect focus detection pixel estimation value A1av estimated from the data of the normal focus detection pixel located between the defect focus detection pixels. Good (A1x = A1av). Alternatively, the defect focus detection pixel interpolation data A1x may be an estimated value A1w of the defect focus detection pixel estimated from data of normal imaging pixels around the defect focus detection pixel (A1x = A1w).

  21 and 22 are enlarged views of the vicinity of the defective focus detection pixel of the image sensor 212A shown in FIG. In these figures, R, G, and B added to each pixel indicate spectral characteristics of each imaging pixel, and A0 * (* = 0, 1, 2,...) Indicates a pair of photoelectric conversions of the focus detection pixel. .., A1 * (* = 0, 1, 2,...) Indicates the other photoelectric conversion unit of the focus detection pixel.

<< Example 5 of interpolation calculation of defective focus detection pixels >>
FIG. 21 shows a case where the outputs of both of the pair of photoelectric conversion units of the defect focus detection pixel are abnormal, and the defect focus detection pixel is detected when the defect focus detection pixel is at the position of the blue pixel in the imaging pixel array. The pixel arrangement for 5 pixels vertically and 5 pixels horizontally is shown at the center. The interpolation data A0x and A1x of the pair of photoelectric conversion units of the defect focus detection pixel to be obtained are the defects estimated from the data of the normal focus detection pixel at the position sandwiching the defect focus detection pixel as shown in the following equation (25). It is obtained by averaging the estimated values A0av and A1av of the focus detection pixels and the estimated values A0w and A1w of the defect focus detection pixels estimated from data of normal imaging pixels around the defect focus detection pixels.
A0x = (A0av + A0w) / 2
A1x = (A1av + A1w) / 2 (25)
In Equation (25), A0av and A1av in the first term on the right side of both equations are the same type of photoelectric conversion unit as the abnormal output photoelectric conversion unit of the defective focus detection pixel, as shown in the following Equation (26): It is the average data of the data of the photoelectric conversion part of the normal focus detection pixel located between the defective focus detection pixels.
A0av = (A00 + A01) / 2
A1av = (A10 + A01) / 2 (26)

In Expression (25), A0w and A1w in the second term on the right side of both expressions are estimated values of defective focus detection pixel data derived based on the equivalent relationship between focus detection pixel data and imaging pixel data. is there. As described above, since the output of the focus detection pixel is approximately expressed as a linear sum of the output of the imaging pixel, the sum of the output data of the pair of photoelectric conversion units of the focus detection pixel 311 is the data A of the focus detection pixel. Then, the expression (10) is approximately established between the data A of the focus detection pixel and the sum of the data R, G, and B of the imaging pixel. In the equation (10), when R, G, and B are locally estimated from the data of the focus detection pixels and the imaging pixels around the defect focus detection pixel, the following equation (27) is obtained.
A = A0w + A1w = (A00 + A01 + A10 + A11) / 2
A0w / A1w = (A00 + A01) / (A10 + A11),
R = (R00 + R01 + R10 + R11) / 4
G = (G00 + G01) / 2
B = (B00 + B01) / 2 (27)

By substituting equation (27) into equation (10) and rearranging, estimated values A0w and A1w can be obtained by equation (28).
A0w = {Kr · (R00 + R01 + R10 + R11) / 4 + Kg · (G00 + G01) / 2 + Kb · (B00 + B01) / 2} / {1+ (A10 + A11) / (A00 + A01)},
A1w = {Kr · (R00 + R01 + R10 + R11) / 4 + Kg · (G00 + G01) / 2 + Kb · (B00 + B01) / 2} / {1+ (A00 + A01) / (A10 + A11)} (28)

  In the interpolation calculation example 5, the defect focus detection pixel estimated values A0av and A1av obtained by estimating the defect focus detection pixel interpolation data A0x and A1x from the normal focus detection pixel data at the position sandwiching the defect focus detection pixel, and the defect An example in which the average of the estimated values A0w and A1w of the defect focus detection pixels estimated from the data of normal imaging pixels around the focus detection pixels is shown. According to this calculation method, the interpolation data A0x and A1x of the defective focus detection pixel can be accurately estimated. The two sets of estimated values (A0av, A0w) and (A1av, A1w) may be weighted and averaged (A0x = K1, A0av + K2, A0w, A1x = K3, A1av + K4, A1w, K1, K2, K3, K4). Is weight). Further, although the estimation accuracy is slightly lowered, the defect focus detection pixel estimated value A0av estimated from the defect focus detection pixel interpolation data A0x, A1x from the data of normal focus detection pixels at positions sandwiching the defect focus detection pixel. , A1av may be used (A0x = A0av, A1x = A1av). Alternatively, the defect focus detection pixel interpolation data A0x and A1x may be used as the defect focus detection pixel estimation values A0w and A1w estimated from the data of normal imaging pixels around the defect focus detection pixel (A0x = A0w, A1x = A1w).

<< Defect focus detection pixel interpolation calculation example 6 >>
FIG. 22 shows a case where the outputs of both of the pair of photoelectric conversion units of the defective focus detection pixel are abnormal, and when the defective focus detection pixel is at the position of the green pixel in the imaging pixel array, The pixel arrangement for 5 pixels vertically and 5 pixels horizontally is shown at the center. The interpolation data A0x and A1x of the pair of photoelectric conversion units of the defect focus detection pixel to be obtained are the defect focus estimated from the data of the normal focus detection pixel at the position sandwiching the defect focus detection pixel as shown in the equation (29). It is obtained by averaging the estimated values A0av and A1av of the detected pixels and the estimated values A0w and A1w of the defect focus detection pixels estimated from the data of normal imaging pixels around the defect focus detection pixels.
A0x = (A0av + A0w) / 2
A1x = (A1av + A1w) / 2 (29)
In Equation (29), A0av and A1av in the first term on the right side of both equations are the same type of photoelectric conversion unit as the abnormal output photoelectric conversion unit of the defective focus detection pixel as shown in the following Equation (30): It is the average data of the data of the photoelectric conversion part of the normal focus detection pixel located between the defective focus detection pixels.
A0av = (A00 + A01) / 2
A1av = (A10 + A01) / 2 (30)

In Equation (29), A0w and A1w in the second term on the right side of both equations are estimated values of defective focus detection pixel data derived based on the equivalent relationship between the focus detection pixel data and the imaging pixel data. is there. As described above, since the output of the focus detection pixel is approximately expressed as a linear sum of the output of the imaging pixel, the sum of the output data of the pair of photoelectric conversion units of the focus detection pixel 311 is the data A of the focus detection pixel. Then, the expression (10) is approximately established between the data A of the focus detection pixel and the sum of the data R, G, and B of the imaging pixel. In the equation (10), when R, G, and B are locally estimated from the data of the focus detection pixels and the imaging pixels around the defect focus detection pixel, the following equation (31) is obtained.
A = A0w + A1w = (A00 + A01 + A10 + A11) / 2
A0w / A1w = (A00 + A01) / (A10 + A11),
R = (R00 + R01) / 2
G = (G00 + G01 + G10 + G11) / 4
B = (B00 + B01 + B10 + B11) / 4 (31)

By substituting equation (31) into equation (10) and rearranging, estimated values A0w and A1w can be obtained by equation (32).
A0w = {Kr · (R00 + R01) / 2 + Kg · (G00 + G01 + G10 + G11) / 4 + Kb · (B00 + B01 + B10 + B11) / 4} / {1+ (A10 + A11) / (A00 + A01)},
A1w = {Kr · (R00 + R01) / 2 + Kg · (G00 + G01 + G10 + G11) / 4 + Kb · (B00 + B01 + B10 + B11) / 4} / {1+ (A00 + A01) / (A10 + A11)} (32)

  In the interpolation calculation example 6, the defect focus detection pixel estimated values A0av and A1av estimated from the defect focus detection pixel interpolation data A0x and A1x from the normal focus detection pixel data at the position sandwiching the defect focus detection pixel, and the defect An example in which the average of the estimated values A0w and A1w of the defect focus detection pixels estimated from the data of normal imaging pixels around the focus detection pixels is shown. According to this method, the interpolation data A0x and A1x of the defective focus detection pixel can be accurately estimated. The two sets of estimated values (A0av, A0w) and (A1av, A1w) may be weighted and averaged (A0x = K1, A0av + K2, A0w, A1x = K3, A1av + K4, A1w, K1, K2, K3, K4). Is weight). Further, although the estimation accuracy is slightly lowered, the defect focus detection pixel estimated value A0av estimated from the defect focus detection pixel interpolation data A0x, A1x from the data of normal focus detection pixels at positions sandwiching the defect focus detection pixel. , A1av may be used (A0x = A0av, A1x = A1av). Alternatively, the defect focus detection pixel interpolation data A0x and A1x may be used as the defect focus detection pixel estimation values A0w and A1w estimated from the data of normal imaging pixels around the defect focus detection pixel (A0x = A0w, A1x = A1w).

  In the embodiment described above, the data interpolation of the defective focus detection pixels when the focus detection pixels are continuously arranged on one straight line has been described. However, by applying the same concept, the focus detection pixels Even when the pixels are thinned out and arranged, or when the focus detection pixels are arranged in a zigzag manner, data interpolation of the defective focus detection pixels can be performed.

  In the imaging devices 212 and 212A shown in FIGS. 3 and 16, the imaging pixel 310 includes a Bayer color filter, but the configuration and arrangement of the color filter are not limited to this, and the complementary color filter The present invention can also be applied to an arrangement other than (Green: G, Yellow: Ye, Magenta: Mg, Cyan: Cy) or an array other than the Bayer array. In the imaging elements 212 and 212A shown in FIGS. 3 and 16, an example in which the focus detection pixels 313 and 314 are not provided with a color filter is shown. However, one of the color filters of the same color as the imaging pixel 310 (for example, Even when a green filter is provided, the present invention can be applied.

  Further, in the focus detection pixels 311, 313, and 314 shown in FIGS. 5 and 17 of the embodiment described above, an example in which the shape of the photoelectric conversion unit is semicircular or rectangular is shown. The shape of the part is not limited to these, and may be other shapes. For example, the shape of the photoelectric conversion unit of the focus detection pixel can be an ellipse or a polygon.

  Further, in the imaging elements 212 and 212A shown in FIGS. 3 and 16, the example in which the imaging pixels and the focus detection pixels are arranged in a dense square lattice arrangement is shown, but a dense hexagonal lattice arrangement (honeycomb arrangement) may be used.

  In the above-described embodiment, the imaging elements 212 (see FIG. 3) and 212A (see FIG. 16) in which the imaging pixels and the focus detection pixels are two-dimensionally arranged have been described as an example. The present invention can also be applied to the focus detection dedicated elements arranged on a plane, and similar effects can be obtained. In this case, a light transmitting / reflecting member such as a half mirror is provided between the interchangeable lens 202 and the image sensor 212 shown in FIG. 1, and the image sensor 212 is an image-capturing element in which imaging pixels are two-dimensionally arranged. The transmission light from the light transmission reflection member may be received by the imaging dedicated element, and the reflection light from the light transmission reflection member may be received by the focus detection dedicated element.

  In the above-described embodiment, the focus detection operation by the pupil division method using the microlens has been described. However, the present invention is not limited to the focus detection of such a method, and is disclosed in Japanese Patent Laid-Open No. 2008-015157. The present invention can also be applied to a pupil detection type phase difference detection type focus detection apparatus using a polarizing element.

  Note that the imaging apparatus is not limited to a digital still camera or a film still camera in which an interchangeable lens is mounted on the camera body as described above. For example, the present invention can be applied to a lens-integrated digital still camera, film still camera, or video camera. Furthermore, the present invention can be applied to a small camera module built in a mobile phone, a surveillance camera, a visual recognition device for a robot, an in-vehicle camera, and the like.

  According to the above-described embodiment and its modifications, the following operational effects can be achieved. First, an embodiment is an image pickup device in which an image pickup pixel 310 and focus detection pixels 311, 313, and 314 are two-dimensionally arranged, and an interchangeable lens is formed by arranging a plurality of focus detection pixels 311, 313, and 314. When there are defective focus detection pixels among the imaging elements 212 and 212A that generate a pair of image signals corresponding to a pair of images formed by a pair of light beams that pass through 202 and a plurality of focus detection pixels 311, 313, and 314 In addition, the output signal of the defective focus detection pixel is calculated by interpolation based on the output signals of the pixels around the defective focus detection pixel, and the output signals of the focus detection pixels 311, 313, and 314 are calculated. Based on the pair of image signals generated by the output signal, a relative shift amount of the pair of images is detected, and an exchange lens is detected based on the detected shift amount of the pair of images. And a body drive control device 214 which calculates a focusing state of 202. By providing such a configuration, it is possible to perform focus detection normally even if an image sensor including defective focus detection pixels is used, and to prevent a decrease in the yield of the image sensor. Effects can be obtained. In particular, the presence of defective focus detection pixels decreases the focus detection accuracy as the contrast change of the subject image increases, that is, the spatial frequency increases. However, according to one embodiment, it is possible to prevent a decrease in focus detection accuracy.

  Next, according to one embodiment, since the output signal of the defective focus detection pixel is calculated by interpolation based on the output signal of the imaging pixels around the defective focus detection pixel, it is arranged around the focus detection pixel. The output signal of the defective focus detection pixel can be obtained by interpolation using the output signals of many imaging pixels.

  Furthermore, according to one embodiment, since the output signal of the defective focus detection pixel is calculated by interpolation based on the output signals of the imaging pixels around the defective focus detection pixel and the focus detection pixel, the defective focus detection pixel Can be accurately obtained by interpolation.

  According to one embodiment, since the output signal of the defective focus detection pixel not provided with the color filter is calculated as a linear sum of the output signals of the imaging pixels provided with the respective color filters, focus detection is performed. The output signals of the defective focus detection pixels can be accurately interpolated using the output signals of many imaging pixels arranged around the pixels.

  According to one embodiment, an image sensor 212 (a first focus detection pixel 313 that outputs one image signal of a pair of image signals and a second focus detection pixel 314 that outputs the other image signal. 3), the first focus detection pixel 313 or the second focus detection pixel 313 of the same type as the defect focus detection pixel out of the first focus detection pixel 313 and the second focus detection pixel 314 arranged around the defect focus detection pixel. Since the output signal of the defective focus detection pixel is calculated by interpolation based on the output signal of the focus detection pixel 314, the output signal of the defective focus detection pixel can be accurately interpolated.

Cross-sectional view of the camera showing the configuration of the camera of one embodiment The figure which shows the focus detection position on the photographing screen of the interchangeable lens Front view showing detailed configuration of image sensor Front view showing configuration of imaging pixel Front view showing configuration of focus detection pixel Diagram showing spectral characteristics of imaging pixels Diagram showing spectral characteristics of focus detection pixels Sectional view showing structure of imaging pixel Sectional view showing structure of focus detection pixel The figure which shows the structure of the focus detection optical system of the pupil division type phase difference detection method using a micro lens The figure which shows the concept of the circuit structure of an image sensor The flowchart which shows the imaging operation of the digital still camera (imaging device) of one embodiment Diagram explaining the reliability of focus detection results The figure for demonstrating the interpolation calculation example 1 of a defect focus detection pixel The figure for demonstrating the interpolation calculation example 2 of a defect focus detection pixel Front view of image sensor of modification Front view showing configuration of focus detection pixel The figure which shows the structure of the focus detection optical system of the pupil division type phase difference detection method using a micro lens The figure for demonstrating the interpolation calculation example 3 of a defect focus detection pixel The figure for demonstrating the interpolation calculation example 4 of a defect focus detection pixel The figure for demonstrating the interpolation calculation example 5 of a defect focus detection pixel The figure for demonstrating the interpolation calculation example 6 of a defect focus detection pixel

Explanation of symbols

201; Camera, 202; Interchangeable lens, 212, 212A; Imaging element, 214; Body drive control device, 310; Imaging pixel, 311, 313, 314;

Claims (10)

An image pickup device in which an image pickup pixel and a focus detection pixel are two-dimensionally arranged, and a pair corresponding to a pair of images formed by a pair of light beams passing through an imaging optical system by the arrangement of the plurality of focus detection pixels An image sensor for generating an image signal of
An interpolation means for calculating an output signal of the defective focus detection pixel by interpolation based on an output signal of a pixel around the defective focus detection pixel when there is a defective focus detection pixel among the plurality of focus detection pixels;
Detection for detecting a relative shift amount of the pair of images based on a pair of image signals generated by the output signal of the focus detection pixel and the output signal of the defective focus detection pixel calculated by the interpolation means. Means,
Calculating means for calculating a focus adjustment state of the imaging optical system based on a shift amount of the pair of images detected by the detecting means;
The interpolation means calculates the output signal of the defective focus detection pixel by interpolation based on the output signal of the imaging pixel around the defective focus detection pixel,
The imaging pixel is provided with a plurality of types of color filters,
The interpolating unit calculates an output signal of the defective focus detection pixel not provided with the color filter as a linear sum of output signals of the imaging pixels provided with the color filters. Detection device. An image pickup device in which an image pickup pixel and a focus detection pixel are two-dimensionally arranged, and a pair corresponding to a pair of images formed by a pair of light beams passing through an imaging optical system by the arrangement of the plurality of focus detection pixels An image sensor for generating an image signal of
An interpolation means for calculating an output signal of the defective focus detection pixel by interpolation based on an output signal of a pixel around the defective focus detection pixel when there is a defective focus detection pixel among the plurality of focus detection pixels;
Detection for detecting a relative shift amount of the pair of images based on a pair of image signals generated by the output signal of the focus detection pixel and the output signal of the defective focus detection pixel calculated by the interpolation means. Means,
Calculating means for calculating a focus adjustment state of the imaging optical system based on a shift amount of the pair of images detected by the detecting means;
The interpolation means calculates an output signal of the defective focus detection pixel by interpolation based on an output signal of the imaging pixel and the focus detection pixel around the defective focus detection pixel,
The imaging pixel is provided with a plurality of types of color filters,
The interpolating unit calculates an output signal of the defective focus detection pixel not provided with the color filter as a linear sum of output signals of the imaging pixels provided with the color filters. Detection device. An image pickup device in which an image pickup pixel and a focus detection pixel are two-dimensionally arranged, and a pair corresponding to a pair of images formed by a pair of light beams passing through an imaging optical system by the arrangement of the plurality of focus detection pixels An image sensor for generating an image signal of
When there is a defective focus detection pixel among the plurality of focus detection pixels, an output signal of the defective focus detection pixel based on an output signal of the imaging pixel and the focus detection pixel located around the defective focus detection pixel Interpolation means for calculating by interpolation,
Detection for detecting a relative shift amount of the pair of images based on a pair of image signals generated by the output signal of the focus detection pixel and the output signal of the defective focus detection pixel calculated by the interpolation means. Means,
A focus detection apparatus comprising: a calculation unit that calculates a focus adjustment state of the imaging optical system based on a shift amount of the pair of images detected by the detection unit. An imaging device in which an imaging pixel and a focus detection pixel are two-dimensionally arranged, and the focus detection pixel receives a first and a second of a pair of focus detection light beams passing through an imaging optical system, respectively. A pair corresponding to a pair of images formed by the pair of focus detection light beams, each having three or more focus detection pixels alternately arranged adjacent to each other in one direction. An image sensor that outputs an image signal of
When the first focus detection pixel is a defective focus detection pixel, at least based on the output signal of the first focus detection pixel adjacent to each of the second focus detection pixels located on both sides of the defective focus detection pixel. Interpolation means for calculating the output signal of the defective focus detection pixel by interpolation,
Detection for detecting a relative shift amount of the pair of images based on a pair of image signals generated by the output signal of the focus detection pixel and the output signal of the defective focus detection pixel calculated by the interpolation means. Means,
Calculating means for calculating a focus adjustment state of the imaging optical system based on a shift amount of the pair of images detected by the detecting means ;
The interpolation means outputs an output signal of a first focus detection pixel adjacent to each of second focus detection pixels located on both sides of the defective focus detection pixel, and an output of the imaging pixels around the defective focus detection pixel. A focus detection apparatus that calculates an output signal of the defective focus detection pixel by interpolation based on a signal . An imaging device in which imaging pixels and focus detection pixels are two-dimensionally arranged, wherein a plurality of the focus detection pixels are arranged in one direction, and one of a pair of focus detection light beams passing through the imaging optical system and An imaging device having first and second photoelectric conversion units for receiving the other and outputting a pair of image signals corresponding to a pair of images formed by a pair of focus detection light beams passing through the imaging optical system; ,
When there is a defect in the output signal from the first photoelectric conversion unit of the focus detection pixel, the output signal from the first photoelectric conversion unit of the focus detection pixel located on both sides of the defective focus detection pixel, respectively. Interpolation means for calculating an output signal of the first photoelectric conversion unit of the defective focus detection pixel by interpolation;
Detection for detecting a relative shift amount of the pair of images based on a pair of image signals generated by the output signal of the focus detection pixel and the output signal of the defective focus detection pixel calculated by the interpolation means. Means,
Based on the shift amount of the pair of images detected by the detection means, and a calculation means for calculating a focusing state of the imaging optical system,
The interpolation means is based on an output signal of a first photoelectric conversion unit of a focus detection pixel located on both sides of the defective focus detection pixel and an output signal of the imaging pixel around the defective focus detection pixel. A focus detection apparatus that calculates an output signal of a first photoelectric conversion unit of a defective focus detection pixel by interpolation . An imaging element in which an imaging pixel and a pair of focus detection pixels that receive a pair of focus detection light beams passing through an imaging optical system are two-dimensionally arranged, and a plurality of the focus detection pixels are adjacent to each other on a straight line An image sensor that is disposed and outputs a pair of image signals corresponding to a pair of images formed by the pair of focus detection light beams;
When there is a defect focus detection pixel among the plurality of focus detection pixels, the defect focus detection is performed based on an output signal of at least one of the imaging pixel and the focus detection pixel in the vicinity of the defect focus detection pixel. Interpolation means for calculating pixel output signals by interpolation;
Detection for detecting a relative shift amount of the pair of images based on a pair of image signals generated by the output signal of the focus detection pixel and the output signal of the defective focus detection pixel calculated by the interpolation means. Means,
Calculating means for calculating a focus adjustment state of the imaging optical system based on a shift amount of the pair of images detected by the detecting means ;
The focus detection apparatus characterized in that the interpolation means calculates an output signal of the defective focus detection pixel by interpolation based on output signals of both the imaging pixel and the focus detection pixel . The focus detection apparatus according to claim 6 ,
The focus detection pixel includes first and second focus detection pixels that respectively receive one and the other of a pair of focus detection light beams that pass through the imaging optical system,
The focus detection apparatus, wherein the first focus detection pixels and the second focus detection pixels are alternately arranged adjacent to each other. The focus detection apparatus according to claim 7 ,
The interpolation means includes the first focus detection pixel of the same type as the defective focus detection pixel or the first focus detection pixel of the first focus detection pixel and the second focus detection pixel arranged around the defective focus detection pixel. A focus detection apparatus, wherein an output signal of the defective focus detection pixel is calculated by interpolation based on an output signal of a bifocal detection pixel. In the focus detection apparatus according to any one of claims 1 to 8 ,
Storage means for storing position information of the defective focus detection pixels;
The focus detection apparatus characterized in that the interpolation means calculates an output signal of the defect focus detection pixel by interpolation based on position information of the defect focus detection pixel stored in the storage means. Imaging apparatus characterized by comprising the focus detection apparatus according to any one of claims 1-9.

Images