JP2012113064A - Imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device having focus assisting means of simple configuration.SOLUTION: The imaging device includes: an imaging element 22 having a plurality of pixels arranged in a two dimensional manner, photoelectrically converting a first object image signal formed by a light flux from an imaging optical system and second and third object images formed by pupil division among the light fluxes from the imaging optical system, and outputting second and third object image signals; a correlation calculation part that calculates the distance between two images on the basis of the second and third object image signals output from the imaging element 22; a focus assisting figure generation part for generating a focus assisting figure on the basis of the distance between the two images calculated by the correlation calculation part and the first object image signal output from the imaging element 22; and a display image configuration part for configuring a display image on the basis of the first object image output from the imaging element 22 and the focus assisting figure generated by the focus assisting figure generation part.

Description

  The present invention relates to an imaging apparatus.

  2. Description of the Related Art Conventionally, in an imaging apparatus using an imaging device in which imaging pixels and focus detection pixels are two-dimensionally arranged, for example, a split image is displayed as a guide image for assisting focus during manual focus (MF). . For example, Patent Document 1 proposes an imaging device that is easy to use based on a guide image.

  Patent Document 1 discloses a configuration for displaying a focus guide image that is easy to understand visually during focusing. In this configuration, the image acquisition unit generates a first image and a second image by photoelectrically converting the first subject image and the second subject image formed by the light beams divided by the pupil division unit, respectively. . The display means displays an image. The processing means causes the display means to display a superimposed image in which the first image and the second image are superimposed.

JP 2009-163220 A

In the configuration of Patent Document 1, since a split image is generated from at least the second and third subject images formed by the light beam divided by the pupil dividing unit, there is a problem that the processing becomes complicated.
In addition, it is desirable for the photographer to be easy to use during focusing.

  The present invention has been made in view of the above, and an object of the present invention is to provide an imaging apparatus having focus assisting means that is simple in processing and simple in configuration.

In order to solve the above-described problems and achieve the object, the imaging apparatus of the present invention includes:
A first subject image signal formed by a light beam from the imaging optical system and a second light beam divided from the pupil of the light beam from the imaging optical system, which has a plurality of pixels arranged two-dimensionally. An image sensor that photoelectrically converts each of the subject image and the third subject image and outputs a second subject image signal and a third subject image signal;
A correlation calculation unit that calculates the interval between the two images based on the second subject image signal and the third subject image signal output by the image sensor;
A focus auxiliary diagram generation unit that generates a focus auxiliary diagram based on the interval between the two images calculated by the correlation calculation unit and the first subject image signal output by the image sensor;
And a display image configuration unit configured to form a display image based on the first subject image output from the image sensor and the focus auxiliary diagram generated by the focus auxiliary diagram generation unit.

  Further, according to a preferred aspect of the present invention, there is further provided a display unit for displaying an image corresponding to the first subject image signal instead of the focus assisting diagram when the interval between the two images is larger than a predetermined value. It is desirable.

Further, according to a preferred aspect of the present invention, it further includes a correction value storage unit that stores a plurality of correction values,
The focus auxiliary diagram generation unit corrects the interval between the two images calculated by the correlation calculation unit with a correction value selected from a plurality of correction values stored in the correction value storage unit, and based on the corrected two image interval It is desirable to generate a focus assistance diagram.

Further, according to a preferred aspect of the present invention, the optical system further includes a diaphragm portion that is arranged in the optical path of the photographing optical system and the aperture diameter is adjustable.
It is desirable that the focus auxiliary diagram generation unit corrects the interval between the two images calculated by the correlation calculation unit based on the aperture diameter of the diaphragm unit at the time of shooting.

  According to the present invention, it is possible to provide an imaging apparatus having focus assisting means that is simple in processing and simple in configuration.

It is a figure which shows the internal structure of the digital camera which concerns on 1st Embodiment of this invention. It is a figure which shows the structure of the exit pupil of the digital camera which concerns on this embodiment. It is a figure which shows the structure of the photoelectric conversion part of the image pick-up element in this embodiment. It is sectional drawing which shows the structure of two adjacent pixels of the image pick-up element in this embodiment. It is a figure which shows the internal structure of the image pick-up element in this embodiment. It is a figure which shows the internal structure of the image pick-up element in this embodiment. It is a top view which shows the structure (variation 1) of an image pick-up element. It is a top view which shows the structure (variation 2) of an image pick-up element. It is a top view which shows the structure (variation 3) of an image pick-up element. It is a top view which shows the structure (variation 4) of an image pick-up element. It is a functional block diagram which shows the structure of the digital camera which concerns on this embodiment. It is a flowchart which shows the processing procedure of focusing at the time of image photography. It is a flowchart which shows the flow which calculates 2 image interval. It is a flowchart which shows the flow of an imaging | photography subroutine. It is a figure explaining a focus auxiliary | assistant figure. It is a figure explaining the relationship between a split image and an aperture stop. It is a figure explaining the weighted correlation calculation result.

  Hereinafter, embodiments of an imaging apparatus according to the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited by the following embodiment.

(First embodiment)
(Digital camera)
First, a camera including an imaging device according to an embodiment of the present invention will be described.
FIG. 1 is a diagram showing an internal configuration of a digital camera 11 according to an embodiment of the present invention.
The digital camera 11 includes an interchangeable lens 12 and a camera body 13, and the interchangeable lens 12 is attached to the camera body 13 by a mount unit 14.

The interchangeable lens 12 includes a lens control unit 30, a lens driving unit 16, a diaphragm driving unit 15, a zooming lens 18, a lens 19, a focusing lens 20, and a diaphragm 21. The lens control unit 30 includes peripheral components such as a microcomputer and a memory. The lens control unit 30 controls driving of the focusing lens 20 and the aperture 21, detects the state of the aperture 21, the zooming lens 18 and the focusing lens 20, and the body control unit 24. Lens information and camera information are received.
In addition to the lens state, specification values relating to the lens data itself are stored in the lens data storage unit 32.

  The aperture driving unit 15 controls the aperture diameter of the aperture 21 via the lens control unit 30 based on the signal from the body control unit 24. Further, the lens driving unit 16 drives the zooming lens 18 and the focusing lens 20 via the lens control unit 30 based on the signal from the body control unit 24.

  The camera body 13 includes an imaging element 22, a body control unit 24, a liquid crystal display element driving circuit 25, a liquid crystal display element 26, an eyepiece lens 27, a memory card 29, and the like. Pixels to be described later are arrayed two-dimensionally on the image sensor 22, and the subject image formed by the interchangeable lens 12 is captured by being arranged on the planned image formation surface of the interchangeable lens 12. Focus detection pixels (hereinafter referred to as AF pixels) are arranged at predetermined focus detection positions of the image sensor 22.

  Here, the interchangeable lens 12 corresponds to an imaging optical system, and the imaging element 22 corresponds to an imaging element.

  The body control unit 24 includes peripheral components such as a microcomputer and a memory, and reads out an image signal from the image sensor 22, corrects the image signal, and detects the focus adjustment state of the interchangeable lens 12 via the image sensor drive circuit 28. In addition, it receives lens information from the lens control unit 30, transmits camera information (defocus amount), and controls the operation of the entire digital camera. The body control unit 24 and the lens control unit 30 communicate via the electrical contact unit 23 of the mount unit 14 to exchange various information.

  The liquid crystal display element driving circuit 25 drives the liquid crystal display element 26 of the liquid crystal viewfinder. The photographer observes an image displayed on the liquid crystal display element 26 through the eyepiece lens 27. The memory card 29 is removable from the camera body 13 and is a portable storage medium that stores and stores image signals.

  The subject image formed on the image sensor 22 through the interchangeable lens 12 is photoelectrically converted by the image sensor 22, and the output is sent to the body controller 24. The body control unit 24 calculates the lens drive amount based on the defocus amount at a predetermined focus detection position based on the output data (first image signal, second image signal) of the AF pixels on the image sensor 22. The lens driving amount is sent to the lens driving unit 16 via the lens control unit 30. The body control unit 24 stores the image signal generated based on the output of the image sensor 22 in the memory card 29 and sends the image signal to the liquid crystal display element drive circuit 25 to cause the liquid crystal display element 26 to display an image. .

  The camera body 13 is provided with operation members (not shown) (shutter buttons, focus detection position setting members, etc.), and the body control unit 24 detects operation state signals from these operation members, and according to the detection results. The following operations (imaging operation, focus detection position setting operation, image processing operation) are controlled.

  The lens control unit 30 changes the lens information according to the focusing state, zooming state, aperture setting state, aperture opening F value, and the like. Specifically, the lens control unit 30 monitors the positions of the lens 18 and the focusing lens 20 and the aperture position of the aperture 21 and calculates lens information according to the monitor information, or a lookup table prepared in advance. For example, lens information corresponding to the monitor information is selected from the lens data storage unit 32. The lens control unit 30 drives the focusing lens 20 to a focal point by a driving source such as a motor (not shown) based on the received lens driving amount.

(Configuration of image sensor)
As for the configuration of the digital camera 11 described above, the configuration using the same reference numerals is common to all the following embodiments. Next, the configuration of the imaging element 22 of the imaging device included in the digital camera 11 will be described.

2A to 2E are diagrams illustrating the configuration of the exit pupil of the imaging apparatus according to the embodiment of the present invention. As shown in FIGS. 2A to 2E, the exit pupil P in the digital still camera 11 has focus detection pixels corresponding to at least two types of pupil regions of left, right, upper and lower.
Specific examples are as follows (1) to (5).

(1) The exit pupil P is vertically divided into two, and the left pupil detection pixel L and the right pupil detection pixel R are arranged (FIG. 2A).
(2) The exit pupil P is horizontally divided into two, and the upper pupil detection pixel U and the lower pupil detection pixel D are arranged (FIG. 2B)
(3) The left pupil detection pixel L and the right pupil detection pixel R are arranged on the left and right, and a part of them is overlapped (FIG. 2C).
(4) Upper pupil detection pixel U and lower pupil detection pixel D are arranged one above the other and partially overlapped (FIG. 2D)
(5) The left pupil detection pixel L and the lower pupil detection pixel D are arranged at arbitrary positions and a part thereof is overlapped (FIG. 2 (e)).

The shape of the distance measuring pupil is a semicircular shape or an elliptical shape. However, the shape is not limited to this, and other shapes such as a rectangular shape or a polygonal shape may be used.
2 (a) and 2 (b) may be combined to arrange the vertical and horizontal focus detection pixels, or FIG. 2 (c) and 2 (d) may be combined to arrange the vertical and horizontal focus detection pixels. Further, the focus detection pixels for detecting left and right and diagonal lines may be arranged by combining FIGS. 2C and 2E, but the present invention is not limited to this.

In the imaging apparatus according to the present embodiment, the first image signal obtained from the output of the photoelectric conversion unit that receives a light beam having a different pupil area and transmitted through one of the regions, and the light beam transmitted through the other region. The phase difference is detected based on the second image signal obtained from the output of the photoelectric conversion unit that receives the light, and the focus state of the photographing lens is detected.
A specific example of exit pupil division will be described below with reference to FIGS.

(Division of photoelectric conversion part)
First, an example in which the exit pupil is divided by dividing the photoelectric conversion unit of the image sensor 22 will be described with reference to FIG.
FIG. 3 is a diagram illustrating a configuration of the photoelectric conversion unit of the image sensor 22.
The imaging device 22 transfers the photocharges accumulated in the n-type regions 32α and 32β and the n-type regions 32α and 32β that generate and accumulate photocharges together with the p-type well 31 and the p-type well 31 formed in the substrate. Floating diffusion portion (not shown) (hereinafter referred to as “FD portion”), a surface p + layer that collects photocharges in order to efficiently transfer the photocharges accumulated in the n-type regions 32α and 32β to the FD portion. 33α, 33β, a transfer gate (not shown) for transferring photoelectric charges to the FD portion, a SiO ₂ film 34 as a gate insulating film, a Bayer array color filter 35, and a microlens 36 for collecting light from a subject, Is provided.

  The microlens 36 is formed in a shape and position so that the pupil of the interchangeable lens 12 (FIG. 1) and the surface p + layers 33α and 33β are substantially conjugate. Photoelectric charges are typically generated in the region 37.

  In the example shown in FIG. 3, the photoelectric conversion unit is divided into an n-type region 32α and a surface p + layer 33α, and an n-type region 32β and a surface p + layer 33β, thereby dividing the exit pupil. Light rays L31 and L32 enter the n-type region 32α and the surface p + layer 33α, and the n-type region 32β and the surface p + layer 33β, respectively.

(Eccentric opening)
Next, an example in which the exit pupil is divided by decentering the opening of the pixel of the image sensor 22 with respect to the center of the photoelectric conversion element will be described with reference to FIG.
FIG. 4 is a cross-sectional view showing the structure of two adjacent pixels of the image sensor 22.

  The pixel 41 has a microlens 42, a smooth layer 43 for forming a plane for forming the microlens 42, a light shielding film 44 for preventing color mixture of color pixels, and a color filter layer in order from the top. A smoothing layer 45 for flattening the surface and a photoelectric conversion element 46 are arranged. Similarly to the pixel 41, the pixel 51 also includes a micro lens 52, a smooth layer 53, a light shielding film 54, a smooth layer 55, and a photoelectric conversion element 56 in order from the top.

  Further, in these pixels 41 and 51, the light shielding films 44 and 54 have openings 48 and 58 that are eccentric to the outside from the central portions 47 and 57 of the photoelectric conversion elements 46 and 56, respectively.

  In the example shown in FIG. 4, the opening of the pixel of the image sensor 22 is decentered with respect to the center of the photoelectric conversion element. For this reason, since the light rays L41 and L51 are incident on the photoelectric conversion elements 46 and 56, respectively, the exit pupil is divided.

Next, an example in which the exit pupil is divided by decentering the lens will be described with reference to FIG. FIG. 5 is a diagram illustrating an internal configuration of the image sensor.
In the image sensor of FIG. 5, on-chip lenses 61, 62, 63, and 64 on each pixel are configured independently.
In FIG. 5, the optical axes 61a and 63a of the on-chip lenses 61 and 63 of the pixels in the pixel set A are shifted to the left from the center of the pixels. Further, the optical axes 62a and 64a of the on-chip lenses 62 and 64 of the pixels in the pixel set B are shifted to the right from the center of the pixels.
By comparing the outputs from the two pixel sets A and B, the focus amount of the lens 18 can be calculated.

  In the on-chip lenses 61, 62, 63, and 64, two parameters such as the refractive power and the shapes of the optical axes 61a, 62a, 63a, and 64a can be controlled independently. If the number of pixels is sufficiently large, the pixel set A and the pixel set B can obtain the same light intensity distribution, and the phase difference AF can be performed using this. At this time, since the defocus amount in the entire screen can be detected, the three-dimensional information of the subject can be acquired.

  In the example shown in FIG. 5, the on-chip lens of the image sensor 22 is decentered with respect to the center of the pixel. For this reason, the light rays L61 and L62 are incident on the on-chip lenses 61 and 62, respectively, thereby dividing the exit pupil.

Next, an example of dividing the exit pupil using DML (digital microlens) will be described with reference to FIG. FIG. 6 is a cross-sectional view showing the internal structure of the image sensor.
In the imaging device shown in FIG. 6, the on-chip lens is configured by DML. The pixel 70 and the pixel 80 are adjacent pixels that receive light beams from different regions.

  In FIG. 6, the imaging device includes DMLs 71 and 81, a color filter 72, an aluminum wiring 73, a signal transmission unit 74, a planarization layer 75, light receiving elements 76 and 86 (for example, Si photodiodes), and a Si substrate 77. As shown in FIG. 6, the aluminum wiring 73, the signal transmission unit 74, the smoothing layer 75, the light receiving elements 76 and 86, and the Si substrate 77 constitute a semiconductor integrated circuit 78. Here, the configurations of the pixel 70 and the pixel 80 are the same except for the DMLs 71 and 81.

  FIG. 6 shows a state of light beams incident on the light receiving elements 76 and 86, respectively, out of the entire incident light beam. By using the DMLs 71 and 81, the light beams L71 and L81 enter the light receiving element 76 of the pixel 70 and the light receiving element 86 of the pixel 80, respectively, and the exit pupil is divided.

  As an imaging device (imager), for example, a CCD (charge coupled device), a CMOS (Complementary Metal Oxide Semiconductor), a back-illuminated CMOS, a sensor that can capture all three colors of R, G, and B in one pixel. (Forveon X3) can be used.

  In the following embodiments, the focus detection pixel receives a light beam transmitted through a different position of the pupil of the photographing lens by decentering an on-chip lens formed on the photographing lens side of the photoelectric conversion unit from the center of the pixel. It is configured as follows. As described above, pupil division means uses a light shielding member for the center of the pixel to decenter the opening, uses DML, or provides two photoelectric conversion units in one pixel. But you can.

The focus detection pixels are configured to receive light beams transmitted through different positions of the pupil of the photographing lens. For this reason, the signal level from the focus detection pixel may be different from the signal level output from the imaging pixel in the vicinity of the focus detection pixel. In order to obtain an image signal at the position of the focus detection pixel, it is preferable to adopt the following method (1) or (2).
(1) The gain is adjusted so that the signal of the focus detection pixel is equivalent to the signal level of the surrounding imaging pixels, and the signal is used as the image signal at the position of the focus detection pixel.
(2) Pixel interpolation is performed based on the signal of the focus detection pixel and the signal of the imaging pixel in the vicinity of the focus detection pixel to obtain an image signal at the position of the focus detection pixel.

The gain adjustment method is performed as follows.
First, the signal level output from the focus detection pixel is compared with the signal level output from the imaging pixel in the vicinity of the focus detection pixel. Subsequently, the gain is adjusted so that the signal level output from the focus detection pixel approaches the signal level output from a nearby imaging pixel. Thereafter, demosaicing is performed by using the signal obtained by adjusting the gain of the focus detection pixel signal as an image signal to obtain a final image.

The pixel interpolation method is preferably any one of the following (a) to (c), but is not limited to these, and is not limited to simple average calculation (including weighting), but also linear interpolation, quadratic or higher order polynomials It may be obtained by interpolation or median processing.
(A) The signal at the position of the focus detection pixel is interpolated based on the signal of the imaging pixel in the vicinity of the focus detection pixel, and the signal obtained by the interpolation is used as the image signal at the position of the focus detection pixel. Perform mosaicing to get the final image.
(B) The signal at the position of the focus detection pixel is interpolated based on the signal of the focus detection pixel and the signal of the imaging pixel in the vicinity of the focus detection pixel, and the signal obtained by the interpolation is used for focus detection. Demosaicing is performed as an image signal at the pixel position to obtain a final image.
(C) Interpolate the signal at the position of the focus detection pixel based on the signal of the imaging pixel near the focus detection pixel, and based on the signal obtained by the interpolation and the signal of the position of the focus detection pixel Then, the signal obtained by the interpolation is demosaiced as an image signal at the position of the focus detection pixel to obtain a final image.

A plurality of color filters are respectively disposed in the plurality of pixels of the image sensor. In Examples 3 and 4 to be described later, the transmission characteristics of the plurality of color filters are R (red), G (green), and B (blue).
The B filter is a color filter having the transmission characteristic on the shortest wavelength side among the transmission characteristics having different R, G, and B, and the R filter is a color filter having the transmission characteristic on the longest wavelength side. Has other transmission characteristics.
The plurality of color filters may include other combinations as long as they include at least a part of the visible region and have at least three different transmission characteristics.

The focus detection pixel uses the G filter as a color filter that weights the luminance signal most among the plurality of color filters, and restricts the incident direction of the incident light beam.
Note that the focus detection pixel is not limited to the G filter, and at least a part of the color filter that weights the luminance signal most among the plurality of color filters or the pixel in which the color filter having the highest transmittance is arranged. The incident direction of the incident light beam can be limited.

(Pixel array variation 1)
FIG. 7 is a plan view conceptually showing a pixel arrangement in the imager of the first embodiment.
The imager (imaging device) illustrated in FIG. 7 has the pixel center illustrated in FIGS. 3, 4, 5, and 6 with the center of each pixel and the center of the pupil or the area center of gravity of each photoelectric conversion region in the upward direction and the downward direction. It is composed of a combination of pixels shifted in the direction, right direction, and left direction.

FIG. 7 shows a photoelectric conversion region as desired from the optical axis direction of each pixel.
FIG. 7 shows an example of 10 pixels in the vertical direction (L01 to L10) and 10 pixels in the horizontal direction (F01 to F10), for a total of 100 pixels. However, the number of pixels is not limited to this. For example, the total number of pixels may exceed 10 million pixels.

  In the example shown in FIG. 7, there are four types of directions in which the area center of the photoelectric conversion region is shifted with respect to the pixel center: right side, left side, upper side, and lower side. In the following description, they are referred to as a right pixel 120R, a left pixel 120L, an upper pixel 120U, and a lower pixel 120D, respectively.

In FIG. 7, in the row L01, the left pixel 120L, the imaging pixel 121, the left pixel 120L, and the imaging pixel 121 are repeatedly arranged in order from the left (from F01).
In the row L02, the imaging pixel 121, the upper pixel 120U, the imaging pixel 121, and the lower pixel 120D are repeatedly arranged in order from the left.
In the row L03, the right pixel 120R, the imaging pixel 121, the right pixel 120R, and the imaging pixel 121 are repeatedly arranged in order from the left.
In the row L04, the imaging pixel 121, the upper pixel 120U, the imaging pixel 121, and the lower pixel 120D are repeatedly arranged in order from the left.
The rows after L05 are arranged to repeat the patterns of L01, L02, L03, and L04.

The arrangement in FIG. 7 is viewed as follows from the columns F01 to F10.
In the column F01, the left pixel 120L, the shooting pixel 121, the right pixel 120R, and the shooting pixel 121 are repeatedly arranged in order from the top (from L01).
In the column F02, the imaging pixel 121, the upper pixel 120U, the imaging pixel 121, and the upper pixel 120U are repeatedly arranged in order from the top.
The columns after F03 are arranged to repeat the patterns F01 and F02.

  In the following description, row numbers L01 to L10 and column numbers F01 to F10 are shown side by side when a specific pixel is shown. For example, the pixel corresponding to the column F01 in the L01 row is represented by “L01F01”.

  In the example illustrated in FIG. 7, for example, one of L01F01 (left pixel 120L), L02F02 (upper pixel 120U), L03F01 (right pixel 120R), and L02F04 (lower pixel 120D) is calculated from the pixel pitch. The distance between the centers of the pupils or the distance between the centers of gravity is smaller than the distance between the pixels.

  In the imaging apparatus according to the first embodiment, the phase difference is determined from the output signals (signals for ranging) of the cell group including the left pixel 120L and another cell group including the right pixel 120R. Information can be calculated to adjust the focus of the optical system.

  For example, the output waveform obtained from L01F01, L01F03, L01F05, L01F07, and L01F09, which are the left pixels 120L in the L01 row, and the output obtained from L03F01, L03F03, L03F05, L03F07, and L03F09 that are the right pixels 120R in the L03 row By comparing the waveforms with each other, defocus information and in-focus position information can be acquired by a so-called phase difference detection formula.

(Pixel array variation 2)
Next, another configuration example of the imaging element included in the imaging device will be described.

  In the arrangement of FIG. 8, the L05F01 pixel and the L05F05 pixel are left pupil detection pixels. The L05F03 pixel is a right pupil detection pixel.

  Thereby, focus detection with high accuracy can be performed.

(Pixel array variation 3)
Next, another configuration example of the imaging element included in the imaging device will be described.

  The arrangement of the color filters in FIG. 9 repeats these combination patterns in the horizontal direction, with the L01F01 pixel as the green filter G and the L01F02 pixel as the red filter R.

Further, the L02F01 pixel is a blue filter B and the L02F02 pixel is a green filter G, and these combination patterns are repeated in the horizontal direction.
Then, the pattern of the L01 column and the pattern of the L02 column are repeated in the vertical direction.

  Here, the L05F01 pixel and the L05F09 pixel in which the green filter G is disposed are the left pupil detection pixels. The L05F05 pixel is a right pupil detection pixel.

  This makes it possible to perform focus detection with high accuracy regardless of the color of the subject. Note that the combination of the color filter and the photoelectric conversion region in the direction of deviation from the pixel center is not limited to this.

(Pixel array variation 4)
Next, still another configuration example of the imaging element included in the imaging device will be described.

  The arrangement of the color filters in FIG. 10 repeats these combination patterns in the horizontal direction, with the L01F01 pixel as the green filter G and the L01F02 pixel as the red filter R.

Further, the L02F01 pixel is a blue filter B and the L02F02 pixel is a green filter G, and these combination patterns are repeated in the horizontal direction.
Then, the pattern of the L01 column and the pattern of the L02 column are repeated in the vertical direction.

Here, the L01F01 pixel, the L01F09 pixel, the L09F01 pixel, and the L09F09 pixel in which the green filter G is arranged are left pupil detection pixels.
The L05F01 pixel and the L05F09 pixel are right pupil detection pixels.
The L01F05 pixel and the L09F05 pixel are upper pupil detection pixels.
Further, the L05F05 pixel is a lower pupil detection pixel.

  This makes it possible to perform focus detection with high accuracy regardless of the color of the subject. Note that the combination of the color filter and the photoelectric conversion region in the direction of deviation from the pixel center is not limited to this.

FIG. 11 is a diagram illustrating functional blocks of the body control unit 24 of the imaging device 11.
From the image sensor 22, a signal for identifying which signal is a signal from the imaging pixel, a signal from the focus detection pixel, the first image signal, or the second image signal passes through the line a. To the A / D converter 131.

  The imaging element 22 has a plurality of pixels arranged two-dimensionally and is pupil-divided among the first subject image signal formed by the light beam from the imaging optical system and the light beam from the imaging optical system. The second subject image and the third subject image formed in this manner are photoelectrically converted, and the second subject image signal and the third subject image signal are output. Details will be described later.

  The image signal acquisition unit 133 acquires a signal that has passed through the signal processing unit 132. A signal from the imaging pixel is output to the liquid crystal display element driving circuit 25 by the display image forming unit 134. The liquid crystal display element 26 displays an image for photographing.

The display image configuration unit 134 configures a display image based on the first subject image output from the image sensor 22 and the focus auxiliary diagram generated by the focus auxiliary diagram generation unit 139.
The photographer can observe an image for photographing through the eyepiece lens 27.

  The recorded image configuration unit 139 configures recorded image data based on signals from the imaging pixels. The recorded image data is stored in the memory card 29.

A signal from the focus detection pixel is output from the image signal acquisition unit 133 to the correlation calculation unit 136.
The correlation calculation unit 136 performs a correlation calculation based on the first image signal and the second image signal. That is, the correlation calculation unit 136 calculates the interval between the two images based on the second subject image signal and the third subject image signal output from the image sensor 22.

The correction value storage unit 138 stores a weighting coefficient described later. The focus auxiliary diagram generation unit 139 generates a focus auxiliary diagram based on the calculation result of the correlation calculation unit 136 and the correction value (= weighting coefficient) stored in the correction value storage unit 138.
That is, the focus auxiliary diagram generation unit 139 generates a focus auxiliary diagram based on the interval between the two images calculated by the correlation calculation unit 136 and the first subject image signal output by the image sensor 22.

  The details of the flowchart for focusing by driving the lens by the correlation calculation unit 136 to the control unit 137 and the details of the focus auxiliary diagram will be described later.

When performing manual focusing on a lens capable of autofocusing, the above-described configuration can be used as it is.
On the other hand, when a lens that cannot be autofocused is used, the signal lines e, d, and f in the above figure are not connected.

Next, the focusing operation in the present embodiment will be described in more detail using a flowchart.
FIG. 12 is a flowchart showing a rough flow of shooting. In step S101, shooting is started. In step S102, the photographer selects whether or not focus assistance is required in manual focus.

  If there is no need for focus assistance in step S102, the display image construction unit 134 constructs a display image in step S103.

  When the focus assist diagram is not required, the photographer observes the image displayed on the liquid crystal display element 26 through the eyepiece lens 27 and performs a manual focusing operation. In step S104, the lens driving unit 16 drives the focusing lens 20 to a focal point by a driving source such as a motor (not shown) based on the lens driving amount.

  In step S105, it is determined whether to perform shooting. When the determination result in step S105 is Yes, a shooting subroutine is executed in step S106.

  Next, in step S102, when the focus assistance is necessary, the photographer further selects to set the focus confirmation priority mode or to the shooting confirmation priority mode.

  If the focus confirmation priority mode is selected in step S107, the process proceeds to step S108. In step S108, a two-image interval calculation subroutine is executed. In step S109, the lens control unit 30 reads the F number of the lens in the imaging state.

Then, the focus auxiliary diagram creation unit 139 selects an appropriate correction value from among a plurality of correction values stored in the correction value storage unit 138. Here, the correction value corresponds to “weighting” as described above.
By weighting the results of the correlation calculation unit, it is possible to increase or decrease the amount of deviation between the two images. The weighting amount may be input from the outside by the photographer. Thereby, conditions easy for the photographer to use can be set.

  The focus auxiliary diagram generation unit 139 creates a focus auxiliary diagram having contents to be described later. Then, the process proceeds to step S103 and photographing is performed.

  In step S107, if the image capture priority mode is selected, the process proceeds to step S113. In step S113, a two-image interval calculation subroutine is executed. In step S114, the focus assisting unit generating unit 139 generates a focus assisting diagram. Then, the process proceeds to step S103 and photographing is performed.

  FIG. 13 is a flowchart showing the procedure of the two-image interval calculation subroutine. In step S201, a two-image interval calculation subroutine is called. In step S202, the image signal acquisition unit 133 reads out two image signals of the first signal and the second signal.

  In step S203, the correlation calculation unit 136 performs a correlation calculation of the two image signals. In step S204, the interval between the two images is calculated as a result of the correlation calculation.

  FIG. 14 is a flowchart showing the procedure of the photographing subroutine. In step S301, a shooting subroutine is called. In step S302, the image signal acquisition unit 133 reads the signal of the first subject image.

  In step S303, the image constructing unit 135 constructs a stored image. In step S304, the recorded image is recorded on the memory card 29 or the like. In step S <b> 305, the liquid crystal display element 26 displays the image configured by the display image configuration unit 134.

  The pixel readout may be configured to perform all pixel readout and thinning readout processing. The image configuration includes interpolation processing of defective pixels. Alternatively, the pixel signal at the position of the focus detection target pixel may be obtained by pixel addition or pixel interpolation.

Moreover, it is not limited to the flow of the said figure.
For example, after the power is turned on, initial state detection, image sensor driving, preview image display, and the like may be performed. Alternatively, the flow of starting shooting may be shifted to the focus detection subroutine after half-pressing the shutter button, the confirmation image may be displayed after focusing, and the flow may be shifted to the shooting subroutine after the shutter button is fully pressed.

Next, focus assistance will be described. FIG. 15 is a diagram for explaining a focus assistance diagram.
FIGS. 15A, 15B, 15C, and 15D are diagrams for explaining focus assistance, that is, a split image in conventional manual focus.
FIGS. 15E and 15F are diagrams for explaining the focus assist in the manual focus according to the present embodiment.

First, a focus assist diagram in the conventional manual focus will be described.
In FIG. 15A, the image signal acquisition unit takes in the first subject image signal 401a formed by the light flux from the imaging optical system.

In FIG. 15B, the image signal acquisition unit 133 captures the second subject image signal 401b. Further, in FIG. 15C, the image signal acquisition unit 133 captures the third subject image signal 401c.
Here, it is necessary to read out not only one line of information for distance measurement pixels that are pupil-divided but also a signal amount that enables image construction in the target areas 400a and 400b. This complicates processing. Furthermore, in order to construct an image, second and third image construction units are required.

The focus auxiliary diagram generation unit generates a focus auxiliary diagram based on the second subject image 401b and the third subject image 401c. And as shown in FIG.15 (d), a display image structure part comprises the display image which is a split image from the 1st object image signal 401a and a focus auxiliary | assistant figure.
As described above, the conventional focus assist in manual focus has problems such as complicated processing and the necessity of the second and third image constructing units.

  Next, focus assistance of this embodiment will be described. In FIG. 15E, the image signal acquisition unit 133 captures the first subject image signal 501a formed by the light flux from the imaging optical system.

  Since it is sufficient that the interval between two images can be calculated from the second subject image signal and the third subject image signal, in principle, one line may be read out. For this reason, it is not necessary to capture and configure the entire subject image. Therefore, the process is simplified.

  For example, the correlation calculation unit 136 calculates the interval between two images based on the readout result of one line.

  The focus auxiliary diagram generation unit 139 generates a focus auxiliary diagram. As shown in FIG. 15 (f), the display image configuration unit 134 configures a display image that is a split image from the first subject image signal 501 a and a focus assisting diagram (parts of regions 500 a and 500 b indicated by dotted lines). .

  In the present embodiment, correlation calculation is performed based on one line of signals from different pupil regions. Then, the amount of deviation of the two images in the paper is calculated. For example, in the present embodiment, it is assumed that each is shifted by x in the horizontal direction of the paper.

  That is, in the image of FIG. 15 (f), the display image configuration unit 134 has moved completely, that is, the object region 500a moved by the distance x to the left and the object region 500b moved to the right by the distance x. It can be obtained by combining with the image signal 501a in the absence.

Next, the relationship between the change in aperture and the split image will be described.
FIGS. 16A and 16D show an optical path diagram and a split image when the diaphragm 21 is opened in the conventional configuration, respectively.
FIGS. 16B and 16E respectively show an optical path diagram and a split image when the diaphragm 21 is stopped in the conventional configuration.
As is apparent from these drawings, the two-image interval Da is changed to a shorter interval Db by narrowing the stop 21. Thus, by changing the aperture, the interval between two images for focusing changes.

  The conventional split image is generated from the first and second subject images formed by the light beams divided by the pupil dividing means. The amount of image shift between the first and second subject images varies depending on the aperture diameter. For this reason, if the aperture diameter is changed in the shooting state with the same focal length, the position of the split image changes, which is inconvenient.

  Therefore, in this embodiment, a correction value for correcting the deviation amount between the two image intervals corresponding to the opening diameter at the time of opening and the opening diameter at the time of photographing is calculated, and a split image is generated and displayed using the correction value.

The diaphragm 21 is disposed in the optical path of the photographing optical system, and the aperture diameter can be adjusted.
The focus auxiliary diagram generation unit 134 corrects the interval between the two images calculated by the correlation calculation unit 136 based on the aperture diameter of the diaphragm 21 at the time of shooting.

  FIGS. 16C and 16F show an optical path diagram and a split image when the diaphragm 21 is stopped in the configuration of the present embodiment, respectively. Here, as in the conventional example, when the aperture is changed, the interval between the two images changes.

The relationship between the defocus amount and the interval between two images is basically a linear relationship. That is, when the defocus amount is taken on the horizontal axis and the two-image interval is taken on the vertical axis, the two have a relationship indicated by a straight line.
When the focus is kept constant and only the aperture is changed, the slope of this straight line changes.

Therefore, in this embodiment, when the focus is maintained in a constant state and only the aperture is changed, correction is performed so that the inclination of the straight line does not change, that is, the interval between two images does not change.
Thus, the photographer can obtain a state in which the interval between the two images is constant even when the diaphragm 21 is changed. As a result, there is an effect that the photographer is easy to use.

  A plurality of correction values (= weighting) are stored in the correction value storage unit 138 in advance. Then, an optimum correction value is selected from the plurality of values.

Next, a configuration for correcting data in relation to the relationship between the defocus amount and the interval between two images (= correlation calculation result) will be described.
FIG. 17 is a diagram showing three types of correlation calculation results. This figure shows three cases: no weighting (solid line), weighting 1 (dashed line), and weighting 2 (one-dot broken line). Hereinafter, each case will be described.

The correction value storage unit 138 further includes a plurality of correction values.
The focus auxiliary diagram generation unit 139 corrects the interval between the two images calculated by the correlation calculation unit 136 based on the correction value selected from the plurality of correction values stored in the correction value storage unit 138 and corrects the corrected two values. A focus assist map is generated based on the image interval.

  The processing of the focus auxiliary diagram generation unit 139 will be described. First, the correlation calculation unit 136 calculates a two-image interval L from two subject images. As described above, the relationship between the defocus amount and the interval between two images is linear.

  The split image image shift amount x can be obtained by weighting the interval L between two images. For example, there is a weighting method as shown in the following equation (1).

x = α × L ^{(1 / k)} + β (1)
here,
0 <α, k = 2 × n + 1 (n = 0, 1, 2, 3...),
β is the correction amount

For example, when the parameters α and k are as follows, a deviation amount as shown in “weighting 1” in FIG. 17 can be expressed. This is an example in which the shift amount of the split image image multiplied by a coefficient is given to the interval between two images.
0 <α
1 = k

  In this case, as shown by the straight line shown by the broken line in FIG. 17, when the image is out of focus, it is possible to make it easier for the photographer to recognize the out-of-focus by displaying the image so as to be more blurred.

Further, for example, when the parameters α and k are as follows, a deviation amount as shown in “weighting 2” in FIG. 17 can be expressed. This is an example of a function in which x is large when the two-image interval L is near 0, and x decreases as the two-image interval L increases.
0 <α
1 <k

  In this case, as shown by the dashed line in FIG. 17, when the focus is achieved (near the origin), the amount of change in the focus ring drive amount and the degree of blur increases (the slope of the curve increases). This brings about an effect that the operability of focusing is further improved for the photographer.

As described above, in the present embodiment, only the means for generating a recorded image is provided, so that the processing is simple. That is, it is possible to provide an imaging apparatus having a focus assisting unit with a simple configuration.
In addition, the old type old lens cannot be autofocused. For this reason, using this embodiment is effective as an assist during focusing.
Further, in the case of a lens capable of autofocusing, there is an advantage that focusing is quicker when manual focus is used for discrimination of a main subject. In this case, usability is improved by using this embodiment.

In the configuration of the above-described embodiment, the display unit 26 can display a captured image instead of the focus assistance diagram when the interval between the two images is larger than a predetermined value.
In other words, when the interval between the two images is larger than the predetermined value, the display unit 26 displays an image corresponding to the first subject image signal instead of the focus assistance diagram.

When the interval between two images is larger than a predetermined value, that is, when the blur is large, the blur change due to focusing is easily recognized, so a live view image is displayed and the interval between the two images is smaller than the predetermined value. When the blur is small, detailed focus adjustment can be easily performed with the split image.
This brings about an effect that the operability of focusing is further improved for the photographer.

  As described above, the present invention is suitable for a manual focus imaging apparatus.

DESCRIPTION OF SYMBOLS 11 Digital still camera 12 Interchangeable lens 13 Camera body 15 Aperture drive part 16 Lens drive part 18 Zooming lens 19 Lens 20 Focusing lens 21 Variable aperture 22 Imaging element 24 Body control part 25 Liquid crystal display element drive circuit 26 Liquid crystal display element
27 eyepiece 28 image pickup device drive circuit 29 memory card 30 lens control unit 31a external input unit 31 p-type well 32α, 32β n-type region 33α, 33β surface p + layer 35 color filter 36 micro lens 41 pixel 42, 52 micro lens 44, 54 Light shielding film 48, 58 Aperture 61, 62, 63, 64 On-chip lens 61a, 62a, 63a, 64a Optical axis 70, 80 Pixel 71, 81 Refractive index distribution lens 72 Color filter 76, 86 Light receiving element 90 Moving / driving Unit 91 Dimming element 120L Left pupil detection pixel 120R Right pupil detection pixel 120U Upper pupil detection pixel 120D Lower pupil detection pixel 121 Imaging pixel 131 A / D conversion unit 132 Signal processing unit 133 Image signal acquisition unit 134 Display image component 135 Calculation unit / Correlation calculation unit 36 correlation calculation section 137 the control unit 138 correction value storing section 139 Focus Auxiliary diagram generation unit

Claims (4)

The first subject image signal formed by the light beam from the imaging optical system, and the first light beam divided from the light beam from the imaging optical system, which has a plurality of pixels arrayed two-dimensionally, is formed. An image sensor that photoelectrically converts each of the second subject image and the third subject image and outputs a second subject image signal and a third subject image signal;
A correlation calculation unit that calculates an interval between two images based on the second subject image signal and the third subject image signal output by the imaging device;
A focus auxiliary diagram generation unit that generates a focus auxiliary diagram based on the interval between the two images calculated by the correlation calculation unit and the first subject image signal output by the image sensor;
An image pickup apparatus comprising: a display image configuration unit configured to form a display image based on the first subject image output from the image sensor and the focus auxiliary diagram generated by the focus auxiliary diagram generation unit. .   2. The display apparatus according to claim 1, further comprising a display unit that displays an image corresponding to the first subject image signal instead of the focus assisting diagram when the interval between the two images is larger than a predetermined value. The imaging device described in 1. A correction value storage unit for storing a plurality of correction values;
The focus auxiliary diagram generation unit corrects an interval between the two images calculated by the correlation calculation unit based on the correction value selected from the plurality of correction values stored in the correction value storage unit, and corrects the correction. The imaging apparatus according to claim 1, wherein the focus auxiliary diagram is generated based on the two-image interval. It further has a diaphragm part that is arranged in the optical path of the photographing optical system and whose aperture diameter is adjustable,
The imaging apparatus according to claim 3, wherein the focus auxiliary diagram generation unit corrects an interval between the two images calculated by the correlation calculation unit based on an aperture diameter of the diaphragm unit at the time of photographing.

Images