JP6476630B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging element and an imaging apparatus.

2. Description of the Related Art There is known an imaging apparatus that generates left and right parallax images having parallax with a single shooting using a single shooting optical system.
[Prior art documents]
[Patent Literature]
[Patent Document 1] Japanese Patent Application Laid-Open No. 2003-7994

  When the interchangeable lens attached to the camera body is replaced, a parallax image cannot be generated properly when the angle of incidence of the outermost ray located on the outermost side of the subject luminous flux incident on the image sensor changes. There was a case.

  The image sensor according to the first aspect of the present invention has an aperture of an aperture mask provided corresponding to each of the parallax pixels that are two-dimensionally and periodically arranged as effective pixels that output image signals. A predetermined partial light beam that is deviated from the optical axis of each of the plurality of interchangeable lenses of the incident light beam, with respect to any of the plurality of interchangeable lenses that collect the subject image. It is positioned to pass at least a part of.

  An imaging apparatus according to a second aspect of the present invention includes an imaging device including parallax pixels that are two-dimensionally and periodically arranged as effective pixels that output an image signal and each have an aperture mask, and for any parallax pixels A determination unit that determines whether at least a part of a predetermined partial light beam deviated from the optical axis of the interchangeable lens among the object light beams incident from the mounted interchangeable lens passes through the opening of the aperture mask; When the determination unit determines not to pass through the opening, the determination unit includes a notification unit that notifies the user of the fact.

  An image pickup apparatus according to a third aspect of the present invention includes an image pickup device including parallax pixels that are two-dimensionally and periodically arranged as effective pixels that output an image signal and each has an aperture mask, and a zoom that collects a subject image. The aperture of the aperture mask is determined in advance so that the parallax pixels arranged at the end of the effective pixel region are deviated from the optical axis of the zoom lens out of the subject luminous flux with respect to a plurality of focal lengths of the zoom lens. It is positioned so as to pass at least a part of the obtained partial light flux.

  In the image pickup device according to the fourth aspect of the present invention, each of the divided pixels arranged two-dimensionally and periodically as an effective pixel that outputs an image signal includes a plurality of photodiodes formed in a unit pixel region. A predetermined portion of the incident light beam that is deviated from the optical axis of the interchangeable lens with respect to any of the interchangeable lenses that collect the subject image, even if the divided pixels are arranged at the end of the effective pixel region. At least a part of the luminous flux is positioned so as to enter a specific photodiode among the plurality of photodiodes.

  An imaging device according to a fifth aspect of the present invention includes divided pixels that are two-dimensionally and periodically arranged as effective pixels that output an image signal, and each include a plurality of photodiodes that are offset within a unit pixel region. At least part of a predetermined partial light beam deviated from the optical axis of the interchangeable lens out of the subject light flux incident from the interchangeable lens attached to the imaging element and any of the divided pixels is a plurality of photodiodes. A determination unit that determines whether the light enters the specific photodiode, and a notification unit that notifies the user when the determination unit determines that the light does not enter the specific photodiode.

  An imaging device according to a sixth aspect of the present invention includes divided pixels including a plurality of photodiodes that are two-dimensionally and periodically arranged as effective pixels that output an image signal, and each of which is offset in a unit pixel region. An image sensor and a zoom lens for condensing a subject image are provided, and the divided pixels are subject light fluxes regardless of the focal length of the zoom lens, even if the divided pixels are arranged at the end of the effective pixel region. Among them, at least a part of a predetermined partial light beam deviated from the optical axis of the zoom lens is positioned so as to enter a specific photodiode among the plurality of photodiodes.

  The summary of the invention does not enumerate all the features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

It is a figure explaining the structure of a digital camera. It is a conceptual diagram which represents notably the mode that a part of imaging device was expanded. It is a figure explaining the concept of defocus in a pixel without parallax. It is a figure explaining the concept of defocus in a parallax pixel. It is a figure which shows the light intensity distribution of a parallax pixel and a parallax pixel. It is a figure explaining an example of the opening shape of an opening mask in case there are two types of parallax pixels. It is the schematic showing the cross section of an image pick-up element. It is a figure explaining the change of the pixel sensitivity by the formation position of wiring, and the light ray incident angle to a pixel. It is a figure explaining the incident angle of the outermost light ray among the object light fluxes which inject into an image sensor. It is a figure explaining the structure of the pixel of an image pick-up element. It is a figure explaining the structure of the pixel of an image pick-up element. It is a figure explaining the structure of the pixel of the image pick-up element which concerns on a modification. It is a figure explaining the structure of the pixel of the image pick-up element which concerns on a modification. It is a figure explaining the structure of the pixel of the image pick-up element by which 2PD was employ | adopted. It is a figure explaining the example of a production | generation process of 2D image data and parallax image data.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  The digital camera according to the present embodiment, which is one form of the imaging device, is configured to generate images with a plurality of viewpoints for one scene by one shooting. Each image having a different viewpoint is called a parallax image. In the present embodiment, a case where a right parallax image and a left parallax image from two viewpoints corresponding to the right eye and the left eye are generated will be described. As will be described in detail later, the digital camera according to the present embodiment can generate a parallax-free image with no parallax from the central viewpoint as the viewpoint in the reference direction along with the parallax image. In some cases, the left viewpoint parallax pixels are referred to as parallax Lt pixels or left parallax pixels, and the right viewpoint parallax pixels are referred to as parallax Rt pixels or right parallax pixels. The left viewpoint parallax image may be referred to as a parallax Lt image, and the right viewpoint parallax image may be referred to as a parallax Rt image.

  FIG. 1 is a diagram illustrating the configuration of a digital camera 10 according to the present embodiment. The digital camera 10 is configured by mounting an interchangeable lens 300 on a camera body 200. The camera body 200 includes an image sensor 100, a camera body control unit 201, an A / D conversion circuit 202, a work memory 203, a drive unit 204, an image processing unit 205, a memory card IF 207, an operation unit 208, a display unit 209, and display control. The unit 210 is provided. The interchangeable lens 300 includes a photographing lens 20 as a photographing optical system, an aperture 22, an interchangeable lens control unit 301, and a lens driving unit 304. The camera body 200 includes a camera mount 213, and the interchangeable lens 300 includes a lens mount 303. When the camera mount 213 and the lens mount 303 are engaged, the connection between the communication terminal on the camera body 200 side and the communication terminal on the interchangeable lens 300 side is established, and communication such as a control signal can be performed. The camera body control unit 201 and the interchangeable lens control unit 301 control the camera body 200 and the interchangeable lens 300 in cooperation with each other while performing communication.

  There are a plurality of types of interchangeable lenses 300 having different focal lengths. The user can attach any one to the camera body 200 according to the purpose of shooting. The taking lens 20 provided in the interchangeable lens 300 guides the subject light flux incident along the optical axis 21 to the image sensor 100 disposed in the camera body 200. As shown in FIG. 1, the direction parallel to the optical axis 21 toward the image sensor 100 is defined as the Z-axis plus direction, the direction toward the back of the plane of the plane orthogonal to the Z-axis is the X-axis plus direction, and the upward direction on the plane is Y. The axis is defined as the plus direction. In the following several figures, the coordinate axes are displayed so that the orientation of each figure can be understood with reference to the coordinate axes of FIG.

  The taking lens 20 is composed of a plurality of optical lens groups, and forms an image of a subject light flux from the scene in the vicinity of its focal plane. In FIG. 1, for convenience of explanation, the photographic lens 20 is represented by a single virtual lens arranged in the vicinity of the pupil. In the vicinity of the pupil, a stop 22 for limiting the incident light beam concentrically with the optical axis as the center is disposed.

  The image sensor 100 is disposed near the focal plane of the photographic lens 20. The image sensor 100 is an image sensor such as a CMOS sensor in which a plurality of pixels are two-dimensionally arranged. The image sensor 100 is timing-controlled by the drive unit 204, converts the subject image formed on the light receiving surface into a pixel signal, and outputs the pixel signal to the A / D conversion circuit 202. The A / D conversion circuit 202 converts the pixel signal output from the image sensor 100 into a digital signal and outputs the digital signal to the work memory 203.

  The image processing unit 205 performs various image processing using the work memory 203 as a work space, and generates image data. As will be described later, the image data includes reference image data generated from the output of the non-parallax pixel of the image sensor 100 and parallax image data generated from the output of the parallax pixel of the image sensor 100. In addition, the image processing unit 205 also has general image processing functions such as adjusting image data in accordance with the selected image format. When generating image data in the JPEG file format, compression processing is performed after white balance processing, gamma processing, and the like are performed. The generated image data is converted into a display signal by the display control unit 210 and displayed on the display unit 209. The data is recorded on the memory card 220 attached to the memory card IF 207.

  A series of shooting sequences is started when the operation unit 208 receives a user operation and outputs an operation signal to the camera body control unit 201. Various operations such as AF and AE accompanying the shooting sequence are executed under the control of the camera body control unit 201.

  The digital camera 10 includes a parallax image shooting mode in addition to the normal 2D shooting mode. If the parallax image shooting mode is set, a parallax image can be shot. The user can select one of these modes by operating the operation unit 208 while viewing the display unit 209 on which the menu screen is displayed.

  The camera body control unit 201 includes a camera memory 214. When the interchangeable lens 300 is attached to the camera body 200, the camera body control unit 201 acquires lens information of the photographing lens 20 via the interchangeable lens control unit 301. In the present embodiment, identification information for identifying the interchangeable lens 300 is acquired as lens information. Also, a correspondence table regarding identification information for identifying an interchangeable lens described later is acquired from the camera memory 214. The camera body control unit 201 compares the lens information of the photographic lens 20 against the correspondence table, and the optical axis of the interchangeable lens 300 among the subject light flux incident from the interchangeable lens 300 attached to any parallax pixel. It is determined whether at least a part of the predetermined partial light beam deviated from 21 passes through the opening of the wiring described later. When it is determined that a predetermined partial light beam does not pass through the opening of the wiring, that is, when an identification number that matches the identification number of the interchangeable lens 300 is not included in the correspondence table, this is notified to the user. Specifically, the display unit 209 displays that a predetermined partial light flux does not pass through the opening of the wiring. The notification method to the user is not limited to this example. For example, you may alert | report by an audio | voice.

  The camera memory 214 is a non-volatile memory such as a flash memory, for example, and plays a role of storing a program for controlling the digital camera 10, various parameters, and the like. The camera memory 214 stores the above correspondence table. The correspondence table is a table in which identification information of the interchangeable lens in which the transmitted subject light flux falls within a preset incident angle range is described. Note that the correspondence table may be updated at any time by firmware upgrade.

  The interchangeable lens control unit 301 moves the photographing lens 20 via the lens driving unit 304. The interchangeable lens control unit 301 includes a lens memory 302. The interchangeable lens control unit 301 reads the lens information from the lens memory 302 that stores the lens information of the photographing lens 20 and transmits the lens information to the camera body control unit 201. The lens memory 302 stores identification information for identifying the photographing lens 20 as lens information of the photographing lens 20.

  FIG. 2 is a conceptual diagram conceptually showing a state in which a part of the image sensor 100 is enlarged. For example, 20 million or more pixels are arranged in a matrix in the pixel region. In the present embodiment, 64 pixels of adjacent 8 pixels × 8 pixels constitute one basic lattice. The basic grid includes four Bayer arrays having 4 × 2 × 2 basic units in the Y-axis direction and four in the X-axis direction. As shown in the figure, in the Bayer array, a green filter (G filter) is arranged for the upper left pixel and the lower right pixel, a blue filter (B filter) is arranged for the lower left pixel, and a red filter (R filter) is arranged for the upper right pixel.

  The basic grid includes parallax pixels and non-parallax pixels. The parallax pixel is a deviated pixel that receives a partial light beam deviated from the optical axis among incident light beams transmitted through the interchangeable lens 300, converts the light beam into a pixel signal, and outputs the pixel signal. As will be described in detail later, the parallax pixel is provided with a wiring that forms a deviated opening deviated from the pixel center so as to transmit only the partial light flux. The wiring functions as an opening mask that defines the offset opening. There are two types of parallax pixels: parallax Lt pixels and parallax Rt pixels. The parallax Lt pixel is a pixel that receives a partial light beam that has reached the left side with respect to the center of the pixel, converts it into a pixel signal, and outputs the pixel signal. The parallax Rt pixel is a pixel that receives a partial light beam that has reached the right side with respect to the center of the pixel, converts it into a pixel signal, and outputs the pixel signal. On the other hand, non-parallax pixels are non-decentered pixels without eccentricity. The non-parallax pixel receives the entire incident light beam that passes through the interchangeable lens 300, converts it into a pixel signal, and outputs it.

Note that FIG. 2 shows a state in which one basic lattice in the image sensor 100 is arranged as it is in accordance with the pixel arrangement. Although the pixel types are shown so that they can be understood, the output values corresponding to each pixel are actually arranged. A pixel in the basic grid is represented by _PIJ . For example, the upper left pixel is _{P 11,} the upper right pixel is _{P 81.} As shown in the figure, the parallax pixels are arranged as follows.

P ₁₁ : Parallax Lt pixel + G filter (= G (Lt))
P ₁₅ ... Parallax Rt pixel + G filter (= G (Rt))
P ₂₇ ... Parallax Lt pixel + R filter (= R (Lt))
P ₃₂ ... Parallax Lt pixel + B filter (= B (Lt))
P ₅₁ ... Parallax Rt pixel + G filter (= G (Rt))
P ₅₅ ... Parallax Lt pixel + G filter (= G (Lt))
P ₆₃ ... Parallax Rt pixel + R filter (= R (Rt))
P ₇₆ ... Parallax Rt pixel + B filter (= B (Rt))
The other pixels are non-parallax pixels, and are any of the non-parallax pixel + R filter, the non-parallax pixel + G filter, and the non-parallax pixel + B filter.

  When viewed as a whole of the image sensor 100, the parallax pixels are classified into one of a first group having a G filter, a second group having an R filter, and a third group having a B filter. The basic lattice includes at least one parallax Lt pixel and parallax Rt pixel belonging to each group among N pixels without parallax of the group. As in the example of the figure, these parallax pixels and non-parallax pixels may be arranged in a dispersed manner in the basic lattice. For example, it may be arranged two-dimensionally and periodically. Since the RGB color information can be acquired as the output of the parallax pixel without causing a bias in the spatial resolution for each color component by dispersively arranging, high-quality parallax image data can be obtained.

  In the basic lattice shown in FIG. 2, G (Lt) + G (Rt) = 2 + 2 = 4 for G (N) = 28, and R (Lt) for R (N) = 14. ) + R (Rt) = 2 and B (N) = 14, B (Lt) + B (Rt) = 2. The RGB ratio has the same R: G: B = 1: 2: 1 configuration as that of the Bayer array for each of the parallax Lt pixel, the parallax Rt pixel, and the non-parallax pixel. The pixel number ratio between the non-parallax pixel, the parallax Lt pixel, and the parallax Rt pixel is N: Lt: Rt = 14: 1: 1. The spatial resolution of the non-parallax pixels is kept close to the Bayer array.

  Next, the relationship between blur and parallax in monocular stereoscopic imaging will be described. Before describing the concept of defocusing when the parallax Lt pixel and the parallax Rt pixel receive light, first, the concept of defocusing in a pixel without parallax will be briefly described.

  FIG. 3 is a diagram for explaining the concept of defocus in a pixel without parallax. As shown in FIG. 3A, when an object point that is a subject exists at the focal position, the subject luminous flux that reaches the image sensor light receiving surface through the lens pupil is steep with the pixel at the corresponding image point as the center. The light intensity distribution is shown. That is, if non-parallax pixels that receive the entire effective luminous flux that passes through the lens pupil are arranged in the vicinity of the image point, the output value of the pixel corresponding to the image point is the largest, and the output value of the pixels arranged in the vicinity Drops rapidly.

  On the other hand, as shown in FIG. 3B, when the object point deviates from the focal position in the direction away from the light receiving surface of the image sensor, the subject luminous flux is received by the image sensor compared to the case where the object point exists at the focus position. It shows a gentle light intensity distribution on the surface. That is, the output value at the pixel of the corresponding image point is lowered, and a distribution having output values up to the peripheral pixels is shown.

  As shown in FIG. 3C, when the object point further deviates from the focal position, the subject luminous flux exhibits a gentler light intensity distribution on the image sensor light receiving surface. In other words, the output value at the pixel of the corresponding image point further decreases, and a distribution having output values up to the surrounding pixels is shown.

  As shown in FIG. 3D, when the object point is shifted from the focal position in the direction approaching the image sensor light receiving surface, the same light as when the object point is shifted in the direction away from the image sensor light receiving surface. The intensity distribution is shown.

  FIG. 4 is a diagram for explaining the concept of defocusing in the parallax pixels. The parallax Lt pixel and the parallax Rt pixel receive the subject luminous flux that arrives from one of the two parallax virtual pupils set as the optical axis target as a partial region of the lens pupil. In this specification, a method of capturing a parallax image by receiving subject light fluxes that arrive from different virtual pupils in a single lens pupil is referred to as a monocular pupil division imaging method.

  As shown in FIG. 4A, when an object point that is a subject exists at the focal position, steep light centering on the pixel of the corresponding image point regardless of the parallax virtual pupil, regardless of the parallax virtual pupil. The intensity distribution is shown. If the parallax Lt pixels are arranged in the vicinity of the image point, the output value of the pixel corresponding to the image point is the largest, and the output value of the pixels arranged in the vicinity rapidly decreases. Further, even when the parallax Rt pixels are arranged in the vicinity of the image point, the output value of the pixel corresponding to the image point is the largest, and the output value of the pixels arranged in the vicinity rapidly decreases. That is, even if the subject luminous flux passes through any parallax virtual pupil, the output value of the pixel corresponding to the image point is the largest, and the output value of the pixels arranged in the vicinity rapidly decreases. Match each other.

  On the other hand, as shown in FIG. 4B, when the object point deviates from the focal position in the direction away from the light receiving surface of the image sensor, the light intensity indicated by the parallax Lt pixel is greater than when the object point exists at the focal position. The distribution peak appears at a position away from the pixel corresponding to the image point in one direction, and its output value decreases. In addition, the width of the pixel having the output value is increased. That is, since the point image spreads in the horizontal direction of the light receiving surface of the image sensor, the amount of blur increases. The peak of the light intensity distribution indicated by the parallax Rt pixel appears at a position away from the pixel corresponding to the image point in the opposite direction to the one direction in the parallax Lt pixel and at an equal distance, and the output value similarly decreases. Similarly, the width of the pixel having the output value is increased. That is, the same light intensity distribution that is gentler than that in the case where the object point exists at the focal position appears at an equal distance from each other. The shift amount between the peaks of the light intensity distribution indicated by the parallax Lt pixel and the parallax Rt pixel corresponds to the parallax amount.

  Further, as shown in FIG. 4C, when the object point further deviates from the focal position, the same light intensity distribution that is more gentle than the state of FIG. . Since the spread of the point image becomes larger, the amount of blur increases. In addition, since the deviation between the peaks of the light intensity distribution indicated by the parallax Lt pixel and the parallax Rt pixel is large, the parallax amount is also increased. In other words, it can be said that the more the object point deviates from the focal position, the more the amount of blur and the amount of parallax increase.

  As shown in FIG. 4D, when the object point deviates from the focal position in the direction approaching the light receiving surface of the image sensor, the light intensity indicated by the parallax Rt pixel is opposite to the state of FIG. The distribution peak appears at a position away from the pixel corresponding to the image point in the one direction. The peak of the light intensity distribution indicated by the parallax Lt pixel appears at a position away from the one direction in the parallax Rt pixel. That is, it is determined in which direction the peak of the light intensity distribution indicated by the parallax Lt pixel and the parallax Rt pixel appears in the direction away from the pixel corresponding to the image point according to the direction of deviation of the object point.

  When the change of the light intensity distribution explained in FIG. 3 and the change of the light intensity distribution explained in FIG. 4 are respectively graphed, they are expressed as shown in FIG. FIG. 5 is a diagram illustrating the light intensity distribution of the non-parallax pixel and the parallax pixel. In the figure, the horizontal axis represents the pixel position, and the center position is the pixel position corresponding to the image point. The vertical axis represents the output value of each pixel. Since this output value is substantially proportional to the light intensity, it is shown as the light intensity in the figure.

  As described above, when the object point deviates from the focal position in the direction approaching the image sensor light receiving surface, the same light intensity distribution as when the object point deviates in the direction away from the image sensor light receiving surface is shown. Therefore, in the figure, the change in the light intensity distribution when the image sensor is shifted in the direction approaching the light receiving surface of the image sensor is omitted. When the object point deviates in the direction away from the image sensor light receiving surface with respect to the peak of the light intensity distribution indicated by the parallax Lt pixel and the parallax Rt pixel when the object point deviates from the focal position in the direction approaching the image sensor light receiving surface. Since it is the same as the peak of the light intensity distribution indicated by the parallax Lt pixel and the parallax Rt pixel, it is omitted.

  FIG. 5A is a graph showing changes in the light intensity distribution described in FIG. A distribution curve 1801 represents the light intensity distribution corresponding to FIG. 3A and shows the steepest state. The distribution curve 1802 represents the light intensity distribution corresponding to FIG. 3B, and the distribution curve 1803 represents the light intensity distribution corresponding to FIG. Compared with the distribution curve 1801, it can be seen that the peak value gradually decreases and has a broadening.

  FIG. 5B is a graph showing changes in the light intensity distribution described in FIG. A distribution curve 1804 and a distribution curve 1805 represent the light intensity distribution of the parallax Lt pixel and the light intensity distribution of the parallax Rt pixel in FIG. 4B, respectively. As can be seen from the figure, these distributions have a line-symmetric shape with respect to the center position. Further, a combined distribution curve 1806 obtained by adding them shows a similar shape to the distribution curve 1802 of FIG. 3B which is in a defocus state equivalent to that of FIG.

  A distribution curve 1807 and a distribution curve 1808 respectively represent the light intensity distribution of the parallax Lt pixel and the light intensity distribution of the parallax Rt pixel in FIG. As can be seen from the figure, these distributions are also symmetrical with respect to the center position. Further, a combined distribution curve 1809 obtained by adding them shows a similar shape to the distribution curve 1803 in FIG. 3C which is in a defocus state equivalent to that in FIG. Note that the light intensity distribution of the parallax Lt pixel and the light intensity distribution of the parallax Rt pixel in FIG. 4D are interchanged with the positions of the light intensity distribution of the parallax Lt pixel and the light intensity distribution of the parallax Rt pixel in FIG. Therefore, they correspond to the distribution curve 1808 and the distribution curve 1807, respectively.

  FIG. 6 is a diagram illustrating an example of the opening shape of the opening 104 when there are two types of parallax pixels. In FIG. 6A, the shape of the opening 104l in the opening mask of the parallax Lt pixel and the shape of the opening 104r of the parallax Rt pixel are obtained by dividing the shape of the opening 104n of the non-parallax pixel by the center line 322, respectively. The example which is the same as this shape is shown. That is, in FIG. 6A, the area of the opening 104n of the non-parallax pixel is the sum of the area of the opening 104l of the parallax Lt pixel and the area of the opening 104r of the parallax Rt pixel. In this case, in this specification, the opening 104n of the non-parallax pixel is referred to as a full-opening opening, and the opening 104l and the opening 104r are referred to as half-opening openings. In addition, when the opening is located at the center of the photoelectric conversion unit, the opening is directed in the reference direction. The opening 104l of the parallax Lt pixel and the opening 104r of the parallax Rt pixel are offset in opposite directions with respect to a virtual center line 322 passing through the center (pixel center) of the corresponding photoelectric conversion unit. . Accordingly, the opening 104l of the parallax Lt pixel and the opening 104r of the parallax Rt pixel each generate parallax in one direction with respect to the center line 322 and in the other direction opposite to the one direction.

  FIG. 6B shows the light intensity when the object point deviates from the focal position in the direction away from the light receiving surface of the image sensor in the pixel group in which only the pixels having the respective openings shown in FIG. Show the distribution. In the figure, the horizontal axis represents the pixel position in the pixel group, and the center position is the pixel position corresponding to the image point. The curve Lt corresponds to the distribution curve 1804 in FIG. 5B, and the curve Rt corresponds to the distribution curve 1805 in FIG. A curve N corresponds to a pixel without parallax, and shows a similar shape to the combined distribution curve 1806 in FIG. Each of the openings 104n, 104l, and 104r functions as an aperture stop. Therefore, the blur width of the non-parallax pixel having the opening 104n having an area double the opening 104l (opening 104r) is the parallax Lt pixel and the parallax Rt pixel indicated by the combined distribution curve 1806 in FIG. Is the same as the blur width of the curve.

  FIG. 6C shows the light intensity when the object point is deviated from the focal position in the direction approaching the light-receiving surface of the image sensor in the pixel group in which only the pixels having the openings shown in FIG. Show the distribution. In the figure, the horizontal axis represents the pixel position in the pixel group, and the center position is the pixel position corresponding to the image point. The curves Lt and Rt in FIG. 6C maintain the relationship that the blur width of the non-parallax pixel having the opening 104n is approximately the same as the blur width of the curve obtained by adding the parallax Lt pixel and the parallax Rt pixel. However, the positional relationship is reversed with respect to the curves Lt and Rt in FIG.

  FIG. 7 is a schematic diagram illustrating a cross section of the image sensor 100. In particular, it is a schematic diagram showing a cross section of the central portion of the effective pixel region in the image sensor 100. A cross section of the end of the effective pixel region will be described later. The imaging element 100 is configured by arranging a microlens 101, a color filter 102, a wiring layer 105, and a substrate 109 in order from the subject side.

  A plurality of photoelectric conversion units 108 are two-dimensionally arranged on the substrate 109. The photoelectric conversion unit 108 includes a photodiode that converts incident light into an electrical signal. An image signal converted by the photoelectric conversion unit 108, a control signal for controlling the photoelectric conversion unit 108, and the like are transmitted / received via the wiring 106 provided in the wiring layer 105.

  The wiring 106 also functions as an aperture mask that limits the incident light flux. A prescribed opening shape can be formed by the wiring 106, and the incident light beam can be limited by the opening shape to guide only a specific partial light beam to the photoelectric conversion unit 108. In the present embodiment, the wiring 106 extends in the X-axis direction so as to cover half of the photoelectric conversion unit 108 so as to correspond to the opening shape shown in FIG. The incident light beam is limited by the action of the wiring 106 functioning as an aperture mask, and parallax is generated. The end of the extended portion of the wiring 106 is formed in a tapered shape.

  On the other hand, the wiring 106 does not exist on the photoelectric conversion unit 108 that does not generate parallax. That is, the wiring 106 does not extend in the X-axis direction. In other words, it can be said that the wiring 106 functions as an aperture mask that does not limit the incident light beam incident on the corresponding photoelectric conversion unit 108, that is, passes the entire incident light beam. Note that the wiring 106 that forms the opening shape may be formed closest to the photoelectric conversion portion 108 in the wiring layer 105.

  The color filter 102 is provided on the wiring layer 105. The color filter 102 is a filter provided in a one-to-one correspondence with each of the photoelectric conversion units 108, which is colored so as to transmit a specific wavelength band to each photoelectric conversion unit 108. In order to output a color image, at least two types of color filters that are different from each other may be arranged, but in order to obtain a color image with higher image quality, it is preferable to arrange three or more types of color filters. For example, a red filter (R filter) that transmits the red wavelength band, a green filter (G filter) that transmits the green wavelength band, and a blue filter (B filter) that transmits the blue wavelength band may be arranged in a grid pattern. The color filter may be not only a combination of primary colors RGB but also a combination of YCM complementary color filters.

  The microlens 101 is provided on the color filter 102. The microlens 101 is a condensing lens that guides more incident light flux to the photoelectric conversion unit 108. The microlens 101 is provided in one-to-one correspondence with each of the photoelectric conversion units 108. In consideration of the relative positional relationship between the pupil center of the photographing lens 20 and the photoelectric conversion unit 108, the optical axis of the microlens 101 is shifted so that more incident light flux is guided to the photoelectric conversion unit 108. It is preferable.

  As described above, one unit of the color filter 102 and the microlens 101 provided in one-to-one correspondence with each photoelectric conversion unit 108 is referred to as a pixel. Note that in the case of an image sensor with good light collection efficiency and photoelectric conversion efficiency, the microlens 101 may not be provided. In the case of a backside illumination type image sensor, the wiring layer 105 is provided on the side opposite to the photoelectric conversion unit 108. Note that the color filter 102 is not provided when a monochrome image signal may be output.

  In the above description, the wiring 106 functions as an opening mask, but an opening mask may be formed separately. For example, an opening mask can be formed immediately below the color filter 102. The aperture mask may be arranged separately and independently corresponding to each photoelectric conversion unit 108, or may be formed collectively for a plurality of photoelectric conversion units 108 in the same manner as the manufacturing process of the color filter 102. Further, if a color component is given to the opening of the opening mask, the color filter 102 and the opening mask can be formed integrally.

The opening mask may be formed by a permeation blocking film provided over the photoelectric conversion unit 108. In this case, the opening mask is formed, for example, by sequentially laminating a SiN film and a SiO ₂ film to form a permeation blocking film and removing a region corresponding to the opening by etching. Furthermore, the region of the photoelectric conversion unit 108 itself may be formed to correspond to the opening.

  FIG. 8 is a diagram for explaining a change in pixel sensitivity and a light beam incident angle on a pixel depending on a wiring formation position. FIG. 8A is a diagram illustrating the configuration of a parallax Lt pixel as a comparative example. Here, the microlens 401, the wiring 406, and the photoelectric conversion unit 408 are illustrated. The wiring 406 is disposed in contact with the photoelectric conversion unit 408. The wiring 406 covers the right half of the photoelectric conversion unit 408. The focal position of the microlens 401 is set to the height of the light receiving surface of the photoelectric conversion unit 408. The middle diagram in FIG. 8A shows a state in which parallel light is incident on the microlens 401 vertically. In this state, a part of incident light reaches the photoelectric conversion unit 408. The left diagram in FIG. 8A shows a state in which the parallel light incident on the microlens 401 is tilted in the clockwise direction, that is, the minus side. In this state, all the incident light rays reach the photoelectric conversion unit 408. The right diagram in FIG. 8A shows a state in which the parallel light incident on the microlens 401 is tilted counterclockwise, that is, on the plus side. In this state, all the incident light rays do not reach the photoelectric conversion unit 408.

  FIG. 8B is a diagram illustrating the relationship between the light beam incident angle of the parallax Lt pixel and the pixel sensitivity as a comparative example. The horizontal axis indicates the light beam incident angle [θ] to the pixel, and the vertical axis indicates the pixel sensitivity. The broken line indicates the relationship between the light beam incident angle of the pixel without parallax and the pixel sensitivity. In the parallax Lt pixel as a comparative example, the wiring 406 is arranged in contact with the photoelectric conversion unit 408. Therefore, as shown by the bold line, although the sensitivity is obtained when the incident light angle is negative, In the case of, it has no sensitivity.

  FIG. 8C is a diagram illustrating the configuration of the parallax Lt pixel of the present embodiment. Here, the microlens 101, the wiring 106, and the photoelectric conversion unit 108 are illustrated. As already described, the wiring 106 is disposed at a position away from the photoelectric conversion unit 108. The wiring 106 covers the right half of the photoelectric conversion unit 108. Further, the focal position of the microlens 101 is set to the height of the light receiving surface of the photoelectric conversion unit 108. The center diagram in FIG. 8C shows a state in which parallel light is incident on the microlens 101 perpendicularly. In this state, a part of incident light reaches the photoelectric conversion unit 108. When compared with the parallax Lt pixel of the comparative example, the wiring 106 is arranged closer to the microlens 101 than the focal position of the microlens 101, but the range of the light flux to be limited is the same, so the photoelectric conversion unit 408 The amount of light that reaches is unchanged.

  The left figure of FIG.8 (c) has shown the state in which the parallel light which injects into the micro lens 101 inclines in the clockwise direction. In this state, a part of incident light reaches the photoelectric conversion unit 108, but the remaining part 111 is blocked by the wiring 106 and does not reach the photoelectric conversion unit 108. The right diagram in FIG. 8C shows a state in which the parallel light incident on the microlens 101 is tilted in the counterclockwise direction. In this state, not all of the incident light rays reach the photoelectric conversion unit 108, but a part 112 reaches the photoelectric conversion unit 108.

  FIG. 8D is a diagram illustrating the relationship between the light beam incident angle and the pixel sensitivity in the parallax Lt pixel of the present embodiment. The horizontal axis indicates the light beam incident angle [θ] to the pixel, and the vertical axis indicates the pixel sensitivity. The broken line indicates the relationship between the light beam incident angle of the pixel without parallax and the pixel sensitivity. In the parallax Lt pixel of this embodiment, since the wiring 106 is disposed at a position away from the photoelectric conversion unit 108, the pixel has sensitivity not only to the left half but also to the right half as illustrated by a thick line. Have. More specifically, as described in the left diagram of FIG. 8C, in this state, the pixel sensitivity is lowered by the amount that the remaining portion 111 does not reach the photoelectric conversion unit. The region 113 corresponds to the pixel sensitivity when the remaining part 111 reaches the photoelectric conversion unit 108. On the other hand, as described in the right diagram of FIG. 8C, in this state, a part 112 of the light beam reaches the photoelectric conversion unit 108. Therefore, the pixel has a region having sensitivity also in the right half. In the figure, a region 114 corresponds to pixel sensitivity due to a part 112 of the light beam reaching the photoelectric conversion unit 108.

  In the above description, the parallax Lt pixel is taken as an example, but the same applies to the parallax Rt pixel. Since the parallax Lt pixels and the parallax Rt pixels as described above are arranged, the distribution curve 1804 and the distribution curve 1805 shown in FIG. 5 can be easily overlapped. In general, the same can be said if the focal position of the microlens 101 is slightly deviated from the surface that limits the incident light beam. The surface which restrict | limits an incident light beam here corresponds to the wiring 106 in FIG.

  FIG. 9 is a diagram for explaining the incident angle (CRA) of the outermost ray among the subject luminous flux incident on the image sensor 100. More specifically, FIG. 9 shows the relationship between the incident angle of the outermost light beam and the exit distance of the photographic lens 20. The subject light flux incident on the image sensor 100 is, for example, a subject light flux incident on the effective pixel region. FIG. 9A shows a case where a wide-angle lens is attached as the interchangeable lens 300. FIG. 9B shows a case where a standard lens is attached as the interchangeable lens 300. FIG. 9C shows a case where a telephoto lens is attached as the interchangeable lens 300. In FIG. 9, it is assumed that the exit pupil distance becomes longer as the telephoto is increased.

  As shown in FIG. 9, the incident angle of the outermost light beam varies depending on the exit pupil distance of the photographing lens 20. Specifically, when the CRA when the standard lens is attached as the interchangeable lens 300 is used as a reference, as shown in FIG. 9A, the CRA when the wide-angle lens is attached becomes larger than the reference. On the other hand, as shown in FIG. 9C, the CRA when the telephoto lens is attached is smaller than the reference.

  By changing the incident angle of the outermost light beam, the incident angle of the incident light beam with respect to the pixels arranged at the end of the effective pixel region also changes. As will be described in detail later, in the image sensor 100 of the present embodiment, the wiring 106 is arranged so that the parallax pixels can receive the incident light beam regardless of whether a wide-angle lens or a telephoto lens is attached. The position and the opening of the wiring 106 are defined. As a result, shading between the left and right parallax images can be suppressed, and a difference between the left and right parallax images due to non-uniform pupil division can be suppressed.

  FIG. 10 is a diagram illustrating the configuration of the pixel 110 of the image sensor 100. It is the schematic showing the cross section of the pixel located in the edge part of an effective pixel area | region especially. Here, a right parallax pixel is exemplified as the pixel 110, and the microlens 101, the wiring 106, and the photoelectric conversion unit 108 are illustrated. In the wiring 106, a portion indicated by a broken line indicates an opening portion, and a portion indicated by hatching indicates a light shielding portion. The alternate long and short dash line indicates the incident light flux when the telephoto lens is attached, the alternate long and two short dashes line indicates the incident light flux when the standard lens is attached, and the three-dot chain line indicates the case where the wide-angle lens is attached The incident light flux is shown.

  In the present embodiment, the height of the aperture mask necessary for passing a preset amount of light is referred to as a target height h1. The height here is a distance from the photoelectric conversion unit 108. In the present embodiment, the preset light amount is ¼ of the light amount incident on the microlens 101. Note that the preset light quantity is not limited to this example.

  The wiring 106 is disposed at the target height h1. More specifically, the wiring 106 is disposed so that the surface of the wiring 106 that faces the photoelectric conversion unit 108 coincides with the target height h1. It can also be said that the opening 104 is disposed so as to be positioned between the microlens 101 and the target height h1. The target height h1 is a specific portion of the incident light flux on the right side of the paper surface in the first direction when the wide-angle lens is attached from the surface of the photoelectric conversion unit 108, and the telephoto lens is attached. In this case, the distance from the incident light flux to the intersection 123 with the light flux 122 of the specific portion on the left side in the drawing, which is the second direction. More specifically, the intersection 123 is an intersection of a peripheral ray 126 that is a light beam on the left edge of the light beam 121 in the light beam 121 and a peripheral light ray 127 that is a light beam on the right edge of the light beam 122 in the paper surface. Here, the luminous flux 121 is a luminous flux equivalent to ¼ of the total luminous flux incident on the microlens 101 when the wide-angle lens is attached, and the luminous flux 122 is obtained when the telephoto lens is attached. The luminous flux corresponding to ¼ of the total luminous flux incident on the microlens 101. In addition, the wiring 106 is arranged so that the light shielding portion is along the light beam 121. By arranging the wiring 106 as described above, the pixel 110 can receive at least ¼ of the amount of light incident on the microlens 101 regardless of which interchangeable lens is attached. That is, it can function as a parallax pixel. As described above, the opening of the wiring 106 is a parallax pixel arranged at the end of the effective pixel region, or the interchangeable lens light of the incident light flux for any of the interchangeable lenses that collect the subject image. It is positioned so as to pass at least a part of the predetermined partial light beam deviated from the axis.

  Note that the formation position of the wiring 106 is not limited to this example. The wiring 106 may be formed at a position higher than the formation position shown in FIG. 10 as long as it is a position away from the microlens 101. More specifically, the wiring 106 may be formed at a position where the surface of the wiring 106 facing the photoelectric conversion unit 108 is higher than the target height h1.

  The microlens 101 is disposed at a position where the optical axis 125 of the microlens 101 is shifted with respect to the pixel center line 124 indicating the center of the photoelectric conversion unit 108. Here, they are arranged at positions shifted to the right. When the microlens 101 is shifted, the peak angle of the angle dependency of the pixel sensitivity changes. Therefore, the microlens 101 may be shifted in accordance with the incident angle of the outermost ray of the subject light beam. For example, from the viewpoint of balance as a camera system, it may be shifted according to a standard lens.

  FIG. 11 is a diagram illustrating the configuration of the pixel 110 of the image sensor 100. It is the schematic showing the cross section of the pixel located in the edge part of an effective pixel area | region especially. FIG. 11 corresponds to FIG. 10 and shows a case where a wide-angle lens, a standard lens, and a telephoto lens are attached to each of the left parallax pixel and the right parallax pixel. Here, the microlens 101, the wiring 106, and the photoelectric conversion unit 108 are illustrated. In the wiring 106, a portion indicated by a broken line indicates an opening portion, and a portion indicated by hatching indicates a light shielding portion. A hatched area 131 in FIG. 13 indicates a portion of the incident light beam that reaches the photoelectric conversion unit 108. The one-dot chain line, the two-dot chain line, and the three-dot chain line are as described above.

  FIG. 11A shows the configuration of the left parallax pixel when a wide-angle lens is attached. In this case, approximately 3/4 of the incident light beam reaches the photoelectric conversion unit 108.

  FIG. 11B shows the configuration of the left parallax pixel when the standard lens is attached. In this case, approximately ½ of the incident light beam reaches the photoelectric conversion unit 108.

  FIG. 11C shows the configuration of the left parallax pixel when a telephoto lens is attached. In this case, approximately ¼ of the incident light beam reaches the photoelectric conversion unit 108.

  As described above, at least ¼ of the incident light beam reaches the photoelectric conversion unit 108 regardless of whether the wide-angle lens is mounted or the telephoto lens is mounted. Therefore, a left parallax image can be generated regardless of whether a wide-angle lens is attached or a telephoto lens is attached.

  FIG. 11D shows the configuration of the right parallax pixel when a wide-angle lens is attached. In this case, approximately ¼ of the incident light beam reaches the photoelectric conversion unit 108.

  FIG. 11E shows the configuration of the right parallax pixel when the standard lens is attached. In this case, approximately ½ of the incident light beam reaches the photoelectric conversion unit 108.

  FIG. 11F shows the configuration of the right parallax pixel when the telephoto lens is attached. In this case, approximately 3/4 of the incident light beam reaches the photoelectric conversion unit 108.

  As described above, at least ¼ of the incident light beam reaches the photoelectric conversion unit 108 regardless of whether the wide-angle lens is mounted or the telephoto lens is mounted. Therefore, a right parallax image can be generated regardless of whether a wide-angle lens is attached or a telephoto lens is attached.

  As shown in FIGS. 11B and 11E, when the standard lens is attached, the light amount of the incident light beam incident on the left parallax pixel is equal to the light amount of the incident light beam incident on the right parallax pixel. Become. That is, when the standard lens is attached, the pupil division ratio is equal between the left and right parallax pixels. As shown in FIGS. 11A and 11F, when a wide-angle lens is mounted, the amount of incident light flux incident on the left parallax pixel, and when the telephoto lens is mounted, the right parallax pixel The amount of incident incident light is approximately equal. When the wide-angle lens is mounted, the ratio of the partial light beam received by the left parallax pixel with respect to the total incident light beam to the microlens 101 (that is, the hatched area 131 in FIG. 11A) and the telephoto lens are mounted. In this case, it can be said that the ratio of the partial light beam received by the right parallax pixel to the total incident light beam to the microlens 101 (that is, the hatched region 131 in FIG. 11F) is substantially equal. Note that “substantially equal” includes a case where the difference is a slight difference and can be regarded as equal although it is not exactly equal due to manufacturing errors of a wide-angle lens, a telephoto lens, a left parallax pixel, and a right parallax pixel. Similarly, as shown in FIGS. 11C and 11D, when a telephoto lens is mounted, the amount of incident light beam incident on the left parallax pixel, and when a wide-angle lens is mounted, The amount of incident luminous flux incident on the parallax pixel is approximately equal. When the telephoto lens is mounted, the ratio of the partial light beam received by the left parallax pixel with respect to the total incident light beam to the microlens 101 (that is, the hatched area 131 in FIG. 11C) and the wide-angle lens are mounted. In this case, it can also be said that the ratio of the partial light beam received by the right parallax pixel to the total incident light beam to the microlens 101 (that is, the hatched region 131 in FIG. 11D) is substantially equal. As described above, when a wide-angle lens or a telephoto lens is attached, the pupil division ratio is not uniform between the left and right parallax pixels, but at least a predetermined amount of light is received. Therefore, it is possible to avoid a situation in which the pupil division ratio is extremely different between the left and right parallax pixels.

  In the above description, the wiring 106 is formed at the target height h1 that has already been described or at a position that is higher than the target height h1 and away from the microlens 101, but is lower than the target height h1. May be formed. In this case, there may be a case where more incident light beam reaches the photoelectric conversion unit 108 of the left parallax pixel, while the incident light beam does not reach the photoelectric conversion unit 108 of the right parallax pixel. Similarly, there may be a case where more incident light beam reaches the photoelectric conversion unit 108 of the right parallax pixel while the incident light beam does not reach the photoelectric conversion unit 108 of the left parallax pixel. Therefore, the opening width of the wiring 106 may be adjusted. Specifically, the opening width of the wiring 106 is made larger than the half opening.

  FIG. 12 is a diagram illustrating the configuration of the pixel 110 of the image sensor 100 according to the modification. It is the schematic showing the cross section of the pixel located in the edge part of an effective pixel area | region especially. Here, a right parallax pixel is exemplified as the pixel 110, and the microlens 101, the wiring 106, and the photoelectric conversion unit 108 are illustrated. In the wiring 106, a portion indicated by a broken line indicates an opening portion, and a portion indicated by hatching indicates a light shielding portion. The one-dot chain line, the two-dot chain line, and the three-dot chain line are as described above.

  As shown in the drawing, the wiring 106 is formed at a position lower than the target height h1, that is, closer to the photoelectric conversion unit 108 side. Here, in the present embodiment, the height of the aperture mask necessary for generating a parallax image is referred to as a limit height h2 regardless of the interchangeable lens. As already described, the wiring 106 is arranged so that the light shielding portion is along the light beam 121. However, even if the wiring 106 is arranged in this way, if the standard lens or the telephoto lens is mounted, if the standard lens or the telephoto lens is mounted, the total luminous flux is not shielded by the wiring 106. The photoelectric conversion unit 108 can be reached. In such a case, a parallax image cannot be generated. Therefore, the surface of the wiring 106 on the side facing the photoelectric conversion unit 108 includes a light beam 121 in a specific portion on the right side of the drawing, which is the first direction, of the incident light beam when a wide-angle lens is attached, and a telephoto lens. Is placed closer to the microlens 101 than the intersection 141 with the light beam 122 on the left side in the drawing, which is the second direction, of the incident light beam. More specifically, the intersection 141 is an intersection of a peripheral ray 126 that is a light beam on the left edge of the light beam 121 in the light beam 121 and a peripheral light beam 128 that is a light beam on the left edge of the light beam 122 in the paper surface. By arranging the wiring 106 as described above, not only when the standard lens is mounted, but also when the telephoto lens is mounted, a part of the incident light beam is shielded by the tapered surface of the wiring 106. Is done. Therefore, regardless of which interchangeable lens is attached, the pixel 110 can receive a part of the incident light beam, not the entire light beam. That is, it can function as a parallax pixel.

  Note that the arrangement position of the wiring 106 is not limited to this example. When the standard lens or the telephoto lens is attached, if it is allowed that all of the incident light beam reaches the photoelectric conversion unit 108, that is, the pixel 110 does not function as a parallax pixel, the photoelectric conversion is performed more than the intersection 141. It may be arranged on the conversion unit 108 side. In this case, the wiring 106 is arranged so that the light-shielding portion is along the light beam 121 and is not in contact with the photoelectric conversion unit 108. If at least a wide-angle lens is attached, the pixel 110 is parallaxed. It can function as a pixel.

  FIG. 13 is a diagram illustrating the configuration of the pixel 110 of the image sensor according to the modification. It is the schematic showing the cross section of the pixel located in the edge part of an effective pixel area | region especially. FIG. 13 corresponds to FIG. 12 and shows a case where a wide-angle lens, a standard lens, and a telephoto lens are attached to each of the left parallax pixel and the right parallax pixel. Here, the microlens 101, the wiring 106, and the photoelectric conversion unit 108 are illustrated. In the wiring 106, a portion indicated by a broken line indicates an opening portion, and a portion indicated by hatching indicates a light shielding portion. A hatching area 151 in FIG. 13 corresponds to the hatching area 131 shown in FIG. A hatched area 152 indicates a portion of the incident light flux that has passed through the wiring 106 by changing the position and opening width of the wiring 106. The one-dot chain line, the two-dot chain line, and the three-dot chain line are as described above.

  FIG. 13A shows the configuration of the left parallax pixel when a wide-angle lens is attached. FIG. 13B shows the configuration of the left parallax pixel when the standard lens is attached. FIG. 13C shows the configuration of the left parallax pixel when a telephoto lens is attached. As illustrated in FIGS. 13A, 13 </ b> B, and 13 </ b> C, the hatched area 151 reaches the photoelectric conversion unit 108. Further, as shown in FIGS. 13A and 13B, in addition to the light beam shown in the hatched area 151, the light beam shown in the hatched area 152 reaches the photoelectric conversion unit 108. Therefore, when a wide-angle lens or a standard lens is attached, the photoelectric conversion unit 108 can receive more light flux. Whether the wide-angle lens is attached or the telephoto lens is attached, the left parallax image can be generated.

  FIG. 13D shows the configuration of the right parallax pixel when a wide-angle lens is attached. FIG. 13E shows the configuration of the right parallax pixel when the standard lens is attached. FIG. 13F shows the configuration of the right parallax pixel when a telephoto lens is attached. As illustrated in FIGS. 13D, 13 </ b> E, and 13 </ b> F, the hatched area 151 reaches the photoelectric conversion unit 108. Further, as shown in FIGS. 13E and 13F, in addition to the light beam shown in the hatched area 151, the light beam shown in the hatched area 152 reaches the photoelectric conversion unit 108. Therefore, when a standard lens or a telephoto lens is attached, the photoelectric conversion unit 108 can receive more light flux. A right parallax image can be generated regardless of whether a wide-angle lens is attached or a telephoto lens is attached.

  As described above, even if the wiring 106 is formed at a position lower than the target height h1, by adjusting the opening width of the wiring 106, the photoelectric conversion unit 108 can be used regardless of the attached lens. At least a predetermined amount of light can be received. Therefore, shading between the left and right parallax images can be suppressed, and a difference between the left and right parallax images due to non-uniform pupil division can be suppressed.

  In the above description, the pixel is configured to include one photoelectric conversion unit in the unit pixel region, but may be configured to include two photoelectric conversion units in the unit pixel region. More specifically, the pixel includes, in the unit pixel region, a left photoelectric conversion unit that is displaced leftward with respect to the center of the unit pixel region, and a right photoelectric conversion unit that is displaced rightward. It may be. In this specification, such a structure may be referred to as 2PD. A pixel having a 2PD structure may be referred to as a divided pixel.

  FIG. 14 is a diagram illustrating the configuration of a pixel 500 of an image sensor that employs 2PD. In particular, it is a schematic diagram showing a cross section of an end portion of an effective pixel region. The pixel 500 includes a left photoelectric conversion unit 502, a right photoelectric conversion unit 503, a separation unit 504 that separates them, and a microlens 501. The left photoelectric conversion unit 502 and the right photoelectric conversion unit 503 are arranged symmetrically with respect to the center of the unit pixel region with the separation unit 504 interposed therebetween. The pixel 500 can function as either a left parallax pixel or a right parallax pixel depending on how the charge accumulated in the left photoelectric conversion unit 502 and the charge accumulated in the right photoelectric conversion unit 503 are read out. . Therefore, 2PD can be said to be a structure in which a left parallax pixel and a right parallax pixel coexist under one microlens 501. A hatching area 505 in FIG. 14 corresponds to the hatching area 131 shown in FIG. The one-dot chain line, the two-dot chain line, and the three-dot chain line are as described above. The condensing position of the microlens 501 is the same as the condensing position of the microlens 101 shown in FIG. That is, it is set to a position shifted in the positive direction of the Z axis from the position where the left photoelectric conversion unit 502 and the right photoelectric conversion unit 503 are formed.

  The left photoelectric conversion unit 502 and the right photoelectric conversion unit 503 are formed at positions where the wiring 106 shown in FIG. 10 is formed. That is, it is formed at a position higher than the target height h1. Note that the formation positions of the left photoelectric conversion unit 502 and the right photoelectric conversion unit 503 are not limited to this example. The left photoelectric conversion unit 502 and the right photoelectric conversion unit 503 may be formed at the target height h1 described above or at a position higher than the target height h1 and away from the microlens 501.

  FIG. 14A shows a pixel configuration when a wide-angle lens is attached. FIG. 14B shows a pixel configuration when a standard lens is attached. FIG. 14C shows a pixel configuration when a telephoto lens is attached. As shown in FIGS. 14A, 14 </ b> B, and 14 </ b> C, a part of the light beam indicated by the hatched region 505 reaches the separation unit 504, but most of the photoelectric conversion unit 502 on the left side. To reach. Therefore, if the pixel 500 is caused to function as a left parallax pixel, the same effect as that of the configuration shown in FIGS. 11A, 11B, and 11C can be obtained. That is, the photoelectric conversion unit 502 can receive at least a predetermined amount of light regardless of whether a wide-angle lens is attached or a telephoto lens is attached. As a result, a left parallax image can be generated.

  FIG. 14D shows a case where a wide-angle lens is attached. FIG. 14E shows a case where a standard lens is attached. FIG. 14F shows a case where a telephoto lens is attached. As shown in FIGS. 14D, 14 </ b> E, and 14 </ b> F, a part of the light beam indicated by the hatching region 505 reaches the separation unit 504, but most of the light conversion unit 503 on the right side. To reach. Therefore, if the pixel 500 is caused to function as a right parallax pixel, the same effect as in the case of the configuration shown in FIGS. 11D, 11E, and 11F can be obtained. That is, the photoelectric conversion unit 503 can receive at least a predetermined amount of light regardless of whether a wide-angle lens is attached or a telephoto lens is attached. As a result, a right parallax image can be generated.

  As described above, right and left parallax images can be generated regardless of the mounted lens even when an image sensor using 2PD is used. Further, as shown in FIGS. 14B and 14E, when the standard lens is mounted, the amount of incident light flux incident on the photoelectric conversion unit 502 and the light amount of incident light flux incident on the photoelectric conversion unit 503. And become equal. That is, when the standard lens is attached, the pupil division ratio is equal between the left and right parallax pixels. As shown in FIGS. 14A and 14F, when the wide-angle lens is mounted, the amount of incident light flux incident on the photoelectric conversion unit 502 and when the telephoto lens is mounted, the photoelectric conversion unit The amount of incident light flux incident on 503 is approximately equal. The ratio of the partial light beam received by the photoelectric conversion unit 502 to the total incident light beam on the micro lens 501 when the wide-angle lens is mounted and the total incident light beam on the micro lens 501 when the telephoto lens is mounted. It can also be said that the ratio of the partial light flux received by the photoelectric conversion unit 503 is substantially equal. Similarly, as shown in FIGS. 14C and 14D, when a telephoto lens is attached, when the amount of incident light flux incident on the photoelectric conversion unit 502 and a wide-angle lens are attached, The amount of the incident light beam incident on the photoelectric conversion unit 503 is approximately equal. The ratio of the partial light beam received by the photoelectric conversion unit 502 to the total incident light beam to the micro lens 501 when the telephoto lens is mounted and the total incident light beam to the micro lens 501 when the wide angle lens is mounted. It can also be said that the ratio of the partial light flux received by the photoelectric conversion unit 503 is substantially equal. As described above, when a wide-angle lens or a telephoto lens is attached, the pupil division ratio is not uniform between the left and right parallax pixels, but at least a predetermined amount of light is received. Therefore, it is possible to avoid a situation in which the pupil division ratio is extremely different between the left and right parallax pixels.

  In the case of using an image sensor that adopts 2PD, the camera body control unit 201 uses the optical axis of the interchangeable lens 300 out of the subject luminous flux incident from the interchangeable lens attached to any divided pixel. It is determined whether at least a part of the deviated predetermined partial light beam is incident on a specific photodiode among the plurality of photodiodes. And when it determines with not entering a specific photodiode, it notifies that to a user. The notification method to the user is as described above.

  In the above description, the left photoelectric conversion unit 502 and the right photoelectric conversion unit 503 are formed at the position where the wiring 106 shown in FIG. 10 or the like is formed, but the photoelectric conversion unit 108 shown in FIG. You may form in the formed position. In this case, the condensing position of the microlens 501 may be set to a position shifted in the positive direction of the Z axis from the position where the left photoelectric conversion unit 502 and the right photoelectric conversion unit 503 are formed.

  The concept of processing for generating 2D image data and parallax image data from captured image data output from the image sensor 100 will be described. FIG. 15 is a diagram illustrating an example of processing for generating 2D image data and parallax image data as reference image data. In FIG. 15, the basic lattice shown in FIG. 2 will be described as an example.

  As can be seen from the arrangement of parallax pixels and non-parallax pixels in the basic lattice, even if the output of the image sensor 100 is aligned with the pixel arrangement and arranged as it is, image data representing a specific image is not obtained. Only when the pixel outputs of the image sensor 100 are separated and collected for each pixel group characterized in the same manner, image data representing one image in accordance with the characteristics is formed. For example, when the left and right parallax pixels are gathered together, left and right parallax image data having parallax can be obtained. In this way, each piece of image data separated and collected for each identically characterized pixel group is referred to as plane data.

  The image processing unit 205 receives raw raw image data in which output values (pixel values) are arranged in the order of pixel arrangement of the image sensor 100, and executes plane separation processing for separating the raw image data into a plurality of plane data. The left column of the figure shows an example of processing for generating 2D-RGB plane data as 2D image data.

In generating the 2D-RGB plane data, the image processing unit 205 first removes the pixel values of the parallax pixels to form an empty grid. Then, the pixel value that becomes the empty grid is calculated by interpolation processing using the pixel values of the surrounding pixels. For example, the pixel values of the vacancy _{P 11} is the pixel value of the G filter pixels adjacent in an oblique _{direction, P -1-1, P 2-1, P} -12, averages calculates the pixel values of _{P 22} To calculate. Further, for example, the pixel value of the empty lattice P ₆₃ is calculated by averaging the pixel values of P ₄₃ , P ₆₁ , P ₈₃ , and P ₆₅ that are adjacent R filter pixel values by skipping one pixel vertically and horizontally. To do. Similarly, for example, the pixel value of the air grating _{P 76} is the pixel value of the adjacent B filter skipping one pixel vertically and _{horizontally,} and averaging operation of the pixel values of _{P 56, P 74, P 96} , P 78 calculate.

  Since the 2D-RGB plane data interpolated in this way is the same as the output of a normal imaging device having a Bayer array, various processes can be performed as 2D image data thereafter. That is, known Bayer interpolation is performed to generate color image data in which RGB data is aligned for each pixel. The image processing unit 205 performs image processing as a general 2D image according to a predetermined format such as JPEG when generating still image data and MPEG when generating moving image data.

In other words, the image processing unit 205 further separates the 2D-RGB plane data for each color and performs the interpolation processing as described above to generate each plane data as reference image data. That is, three of Gn ₀ plane data as green reference image plane data, Rn ₀ plane data as red reference image plane data, and Bn ₀ plane data as blue reference image plane data are generated. Since these plane data are generated based on non-parallax pixels having a high density with respect to the parallax pixels, the resolution is higher than the plane data of the parallax image data described below.

  The right column of the figure shows an example of generation processing of two G plane data, two R plane data, and two B plane data as plane data constituting the parallax pixel data. The two G plane data are GLt plane data as left parallax image data and GRt plane data as right parallax image data. The two R plane data are RLt plane data and right parallax image data as left parallax image data. The two B plane data are the BLt plane data as the left parallax image data and the BRt plane data as the right parallax image data.

In generating the GLt plane data, the image processing unit 205 removes pixel values other than the pixel values of the G (Lt) pixels from all output values of the image sensor 100 to form a vacant lattice. Then, two pixel values P ₁₁ and P ₅₅ remain in the basic lattice. Therefore, four equal portions primitive lattice vertically and horizontally, the 16 pixels of the top left is represented by an output value of the P _11, it is representative of the 16 pixels in the lower right in the output value of the P _55. Then, for the upper right 16 pixels and the lower left 16 pixels, average values of neighboring representative values adjacent in the vertical and horizontal directions are averaged and interpolated. That is, the GLt plane data has one value in units of 16 pixels. More preferably, linear interpolation according to the distance is introduced.

Similarly, when generating the GRt plane data, the image processing unit 205 removes pixel values other than the pixel value of the G (Rt) pixel from all the output values of the image sensor 100 to obtain an empty grid. As a result, two pixel values P ₅₁ and P ₁₅ remain in the basic lattice. Therefore, four equal portions primitive lattice vertically and horizontally, the 16 pixels in the upper right is represented by the output value of the P _51, to the 16 pixels in the lower left is represented by an output value of the P _15. The upper left 16 pixels and the lower right 16 pixels are interpolated by averaging the peripheral representative values adjacent vertically and horizontally. That is, the GRt plane data has one value in units of 16 pixels. More preferably, linear interpolation according to the distance is introduced. In this way, GLt plane data and GRt plane data having a resolution lower than that of the Gn ₀ plane data can be generated.

In generating the RLt plane data, the image processing unit 205 removes pixel values other than the pixel value of the R (Lt) pixel from all output values of the image sensor 100 to form a vacant lattice. Then, the primitive _lattice, the pixel values of _{P 27} remains. This pixel value is a representative value for 64 pixels of the basic grid. More preferably, linear interpolation according to the distance is introduced. Similarly, when generating the RRt plane data, the image processing unit 205 removes pixel values other than the pixel value of the R (Rt) pixel from all output values of the image sensor 100 to form a vacant lattice. Then, the pixel value P ₆₃ remains in the basic grid. This pixel value is a representative value for 64 pixels of the basic grid. More preferably, linear interpolation according to the distance is introduced. In this way, RLt plane data and RRt plane data having a resolution lower than that of the Rn ₀ plane data are generated. In this case, the resolution of the RLt plane data and the RRt plane data is lower than the resolution of the GLt plane data and the GRt plane data.

In generating the BLt plane data, the image processing unit 205 removes pixel values other than the pixel values of the B (Lt) pixels from all output values of the image sensor 100 to form a vacant lattice. Then, the primitive _lattice, the pixel values of _{P 32} remains. This pixel value is a representative value for 64 pixels of the basic grid. More preferably, linear interpolation according to the distance is introduced. Similarly, when generating the BRt plane data, the image processing unit 205 removes pixel values other than the pixel value of the B (Rt) pixel from all the output values of the image sensor 100 to obtain an empty grid. Then, the primitive _lattice, the pixel values of _{P 76} remains. This pixel value is a representative value for 64 pixels of the basic grid. More preferably, linear interpolation according to the distance is introduced. In this way, BLt plane data and BRt plane data having a resolution lower than that of the Bn ₀ plane data are generated. In this case, the resolution of the BLt plane data and the BRt plane data is lower than the resolution of the GLt plane data and the GRt plane data, and is equal to the resolution of the RLt plane data and the RRt plane data.

  In this embodiment, the image processing unit 205 uses these plane data to generate high-resolution left-viewpoint color image data and right-viewpoint color image data.

The red parallax plane (RLt ₀ plane data and RRt ₀ plane data) is generated using the pixel values of the Rn ₀ plane data and the pixel values of the RLt plane data and the RRt plane data. Specifically, for example, when calculating the pixel value RLt _0mn of the target pixel position (i _m , j _n ) of the RLt ₀ plane data, first, the image processing unit 205 first determines the same pixel position (i _{m of the} Rn ₀ plane data _). , J _n ) extract the pixel value Rn _0mn . Next, the image processing unit 205, the same pixel position of RLt plane data _(i m, _{j n)} pixel values _{RLt mn} from the same pixel position of the RRT plane data _(i m, _{j n)} pixel values _{RRT mn} from Extract. Then, the image processing unit 205 calculates the pixel value RLt _0mn by distributing the pixel value Rn _0mn by the ratio of the pixel values RLt _mn and RRt _mn . Specifically, it is calculated by the following equation (1).

RLt _0mn = 2Rn _0mn × RLt _mn / (RLt _mn + RRt _mn ) (1)
Similarly, when calculating the pixel value RRt _0mn of the target pixel position (i _m , j _n ) of the RRt ₀ plane data, the image processing unit 205 calculates the pixel value Rn _0mn as the pixel value RLt _mn and the pixel value RRt _mn . Calculate by distributing by ratio. Specifically, it is calculated by the following equation (2).

RRt _0mn = 2Rn _0mn × RRt _mn / (RLt _mn + RRt _mn ) (2)
The image processing unit 205 sequentially executes such processing from (1, 1) which is the pixel at the left end and the upper end to (i ₀ , j ₀ ) which is the coordinates at the right end and the lower end.

Then, upon generation processing RLt ₀ plane data and RRT ₀ plane data is high resolution red parallax plane is completed, then a high resolution green disparity plane GLt ₀ plane data and GRT ₀ generation processing plane data Execute. Specifically, the same pixel position of Rn ₀ plane data in the above description _(i m, _{j n)} instead of extracting the pixel values _{Rn 0mn} from the same pixel position of Gn ₀ plane data _{(i m, j} n) extracting pixel values _{Gn 0mn} from further _{GLt mn} instead of _{RLt mn,} extracts _{GRT mn} instead of _{RRT mn,} similarly processed. Furthermore, when the generation processing of GL ₀ t plane data and GRt ₀ plane data, which are high-resolution green parallax planes, is completed, generation of BLt ₀ plane data and BRt ₀ plane data, which are the next high-resolution blue parallax planes, is completed. Execute the process. Specifically, the same pixel position of Rn ₀ plane data in the above description _(i m, _{j n)} instead of extracting the pixel values _{Rn 0mn} from the same pixel position of Bn ₀ plane data _{(i m, j} n) extracting pixel values _{Bn 0mn} from further _{BLt mn} instead of _{RLt mn,} extracts _{BRt mn} instead of _{RRT mn,} similarly processed.

Through the above processing, high-resolution color image data (RLt ₀ plane data, GLt ₀ plane data, BLt ₀ plane data) for the left viewpoint and high-resolution color image data (RRt ₀ plane data, GRt _{0 for the} right viewpoint). Plane data, BRt ₀ plane data) is generated. That is, the left-viewpoint and right-viewpoint color image data is relatively simple as virtual outputs of the parallax Lt pixel and the parallax Rt pixel that do not actually exist as pixels of the image sensor 100 and have any of RGB color filters. Can be obtained by simple processing. Therefore, if these image data are reproduced by a 3D image compatible reproducing apparatus, the user can appreciate a high-resolution 3D video image as a color image. In particular, since the processing is simple, image data can be generated at high speed, and it can be applied to moving images.

  As described above, the process of generating the high-resolution parallax image data from the low-resolution parallax image data and the high-resolution 2D image data is performed by superimposing the parallax component of the low-resolution parallax image on the 2D image. Since the displacement process is realized, it can be called a parallax modulation process.

  In the above description, an interchangeable lens type digital camera is used as the digital camera, but a lens-integrated digital camera may be used. When a lens-integrated digital camera is equipped with a zoom lens, the zoom lens is located at the tele end, which is equivalent to the telephoto lens being attached. When the zoom lens is located at the wide end, a wide-angle lens is attached. This corresponds to the case where Therefore, by providing the image sensor described above, it is possible to generate left and right parallax images regardless of the position of the zoom lens. The above-described lens-integrated digital camera includes an image sensor including parallax pixels that are two-dimensionally and periodically arranged as effective pixels for outputting an image signal and each has an aperture mask, and a zoom for condensing a subject image. The aperture of the aperture mask is deviated from the optical axis of the zoom lens in the subject luminous flux for any focal length of the zoom lens, even for parallax pixels arranged at the end of the effective pixel region. It can be configured to be positioned so as to pass at least a part of the predetermined partial light flux. In addition, an image pickup device including divided pixels including a plurality of photodiodes that are two-dimensionally and periodically arrayed as effective pixels that output image signals, and each of which is displaced in the unit pixel region, and a subject image are collected. The zoom lens is divided from the optical axis of the zoom lens with respect to any focal length of the zoom lens, regardless of the focal length of the zoom lens. It is also possible to configure such that at least a part of the predetermined partial light flux is positioned so as to enter a specific photodiode among the plurality of photodiodes.

  In the above description, the camera body control unit 201 determines whether at least a part of the predetermined partial light flux passes through the opening of the wiring based on the identification information of the interchangeable lens 300. You may determine whether it passes through the opening of wiring. For example, it can be determined whether or not the output of the end of the effective pixel region passes through the opening of the wiring by determining whether the output is smaller than a predetermined threshold. When the output at the end of the effective pixel region is smaller than a predetermined threshold value, it is determined that it does not pass through the wiring opening. Further, depending on the set aperture value, whether it passes through the opening of the wiring changes. Therefore, it may be determined whether to notify the user based on the set aperture value. Further, it may be determined whether or not the outermost ray of the subject luminous flux that passes through the interchangeable lens 300 falls within a preset incident angle range with respect to the image sensor 100. In this case, it may be determined whether the outermost ray falls within an incident angle range set in advance with respect to the region to be imaged in the effective pixel region. For example, when the size of the image sensor 100 is equivalent to a full size of 35 mm and the APS-C size equivalent is selected as the imaging range, it may be determined whether it falls within the incident angle range with respect to the APS-C size equivalent. Good.

  In the above description, when the camera body control unit 201 determines that the subject light flux does not fall within the incident angle range, the camera body control unit 201 notifies the user to that effect, but other control may be executed. For example, even if the shooting mode is set to the parallax image shooting mode, shooting may be performed in the normal 2D shooting mode.

  The opening width of the wiring 106 may be set according to the pixel position in the image sensor 100. For example, you may set so large that it goes to the periphery of the image pick-up element 100. FIG. Thereby, when a wide-angle lens or a telephoto lens is attached, the parallax pixels can easily receive a predetermined amount of light. Further, a part of the pixel array may be designed according to the wide-angle lens, and the remaining part may be designed according to the telephoto lens. For example, half of the pixel arrangement may be designed according to the wide-angle lens, and the other half may be designed according to the telephoto lens.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 10 Digital camera, 20 Shooting lens, 21 Optical axis, 22 Aperture, 100 Image sensor, 101 Micro lens, 102 Color filter, 104 Aperture, 105 Wiring layer, 106 Wiring, 108 Photoelectric conversion part, 109 Substrate, 110 Pixel, 111 Remainder, 112 part, 113 region, 114 region, 121 beam, 122 beam, 123 intersection, 124 pixel center line, 125 optical axis, 126 marginal ray, 127 marginal ray, 128 marginal ray, 131 hatching region, 141 intersection, 151 Hatched area, 152 hatched area, 200 camera body, 201 camera body control unit, 202 A / D conversion circuit, 203 memory, 204 drive unit, 205 image processing unit, 207 memory card IF, 208 operation unit, 209 display unit, 210 Display control 213 Camera mount, 214 Camera memory, 220 Memory card, 300 Interchangeable lens, 301 Interchangeable lens control unit, 302 Lens memory, 303 Lens mount, 304 Lens drive unit, 322 Center line, 401 Micro lens, 406 Wiring, 408 Photoelectric conversion Part, 500 pixels, 501 micro lens, 502 photoelectric conversion part, 503 photoelectric conversion part, 504 separation part, 505 hatching area, 1801 distribution curve, 1802 distribution curve, 1803 distribution curve, 1804 distribution curve, 1805 distribution curve, 1806 composite distribution Curve, 1807 distribution curve, 1808 distribution curve, 1809 composite distribution curve

Claims (11)

An imaging device including parallax pixels that are two-dimensionally and periodically arranged as effective pixels that output image signals and each have an aperture mask;
A zoom lens that focuses the subject image and
With
The aperture of the aperture mask is determined in advance so that the parallax pixels arranged at the end of the effective pixel region are deviated from the optical axis of the zoom lens in the subject luminous flux with respect to a plurality of focal lengths of the zoom lens. Is positioned so as to pass at least part of the partial luminous flux,
The parallax pixel includes a photodiode and a microlens formed corresponding to the photodiode,
The opening mask is formed between the photodiode and the microlens;
The aperture mask has a predetermined ratio with respect to the total luminous flux when the outermost ray of the luminous flux that has passed through the zoom lens set relatively on the wide-angle side in the optical axis direction is incident. The peripheral light beam in the second direction opposite to the first direction of the partial light beam on the first direction side and the outermost light beam of the subject light beam that has passed through the zoom lens set relatively on the telephoto side are incident. The position of the intersection of the partial light beam on the second direction side having the predetermined ratio with respect to the total light flux of the subject and the peripheral ray on the first direction side , or between the intersection point and the microlens. Arranged imaging device . The imaging apparatus according to claim 1 , wherein each of the openings of the opening mask is a half opening. An imaging device including parallax pixels that are two-dimensionally and periodically arranged as effective pixels that output image signals and each have an aperture mask;
A zoom lens that focuses the subject image and
With
The aperture of the aperture mask is determined in advance so that the parallax pixels arranged at the end of the effective pixel region are deviated from the optical axis of the zoom lens in the subject luminous flux with respect to a plurality of focal lengths of the zoom lens. Is positioned so as to pass at least part of the partial luminous flux,
The parallax pixel includes a photodiode and a microlens formed corresponding to the photodiode,
The opening mask is formed between the photodiode and the microlens;
The aperture mask has a predetermined ratio with respect to the total luminous flux when the outermost ray of the luminous flux that has passed through the zoom lens set relatively on the wide-angle side in the optical axis direction is incident. The peripheral light beam in the second direction opposite to the first direction of the partial light beam on the first direction side and the outermost light beam of the subject light beam that has passed through the zoom lens set relatively on the telephoto side are incident. In the case where the partial light flux on the second direction side having the predetermined ratio with respect to the total light flux of the subject is closer to the photodiode side than the position of the intersection with the peripheral ray on the first direction side , and relatively the second of the first direction side of the partial light flux having a predetermined rate in the case where the outermost ray of the object light beam which has passed through the zoom lens set to the wide angle side is incident on the total subject light flux And marginal ray direction side, when the outermost ray of the object light beam which has passed through the zoom lens set to a relatively telephoto side is incident, the first having said predetermined percentage of the total subject light flux 2 direction side imaging device located on the microlens side of the intersection between the second direction side of the marginal ray of the partial light fluxes. The imaging apparatus according to claim 3 , wherein each opening of the opening mask is larger than a half opening. Whether for any of the parallax pixels, at least a part of a predetermined partial light beam deviated from the optical axis of the zoom lens out of the subject light beam incident from the zoom lens passes through the opening of the aperture mask A determination unit for determining
The imaging device according to any one of claims 1 to 4, further comprising: a notification unit that notifies the user when the determination unit determines not to pass through the opening. The determination unit determines whether an incident angle of an outermost ray in a subject light beam incident on an effective pixel region is within a preset angle range, whereby at least a part of the partial light beam is the aperture mask. The imaging device according to claim 5 , wherein it is determined whether or not it passes through the opening. The determination unit determines whether the incident angle of the outermost light ray is within a preset angle range among the subject light beams incident on the region to be imaged in the effective pixel region, thereby determining at least the partial light beam. The imaging apparatus according to claim 6 , wherein it is determined whether the part passes through an opening of the opening mask. An image pickup device including divided pixels including a plurality of photodiodes that are two-dimensionally and periodically arranged as effective pixels that output image signals, and each of which is displaced in a unit pixel region;
A zoom lens that focuses the subject image and
With
Even if the divided pixel is the divided pixel arranged at the end of the effective pixel region, the focal point of the zoom lens is previously deviated from the optical axis of the zoom lens with respect to any focal length of the zoom lens. It is positioned such that at least a part of the determined partial luminous flux is incident on a specific photodiode among the plurality of photodiodes,
The divided pixel includes a microlens corresponding to the plurality of photodiodes,
The divided pixels have a predetermined ratio with respect to the total subject luminous flux when the outermost ray of the subject luminous flux that has passed through the zoom lens set relatively on the wide-angle side in the optical axis direction is incident. The peripheral light beam in the second direction opposite to the first direction of the partial light beam on the first direction side and the outermost light beam of the subject light beam that has passed through the zoom lens set relatively on the telephoto side are incident. The position of the intersection of the partial light beam on the second direction side having the predetermined ratio with respect to the total light flux of the subject and the peripheral ray on the first direction side , or between the intersection point and the microlens. Arranged imaging device . For any of the divided pixels, at least some of the predetermined partial luminous fluxes deviated from the optical axis of the zoom lens of the subject light flux entering from the zoom lens is specific of the plurality of photodiodes A determination unit for determining whether the light is incident on the photodiode;
The imaging device according to claim 8, further comprising: a notifying unit that notifies a user when the determining unit determines that the light does not enter the specific photodiode. The determination unit determines whether the incident angle of the outermost ray in the subject light beam incident on the effective pixel region is within a preset angle range, so that at least a part of the partial light beam is the specific light beam. The imaging apparatus according to claim 9 , wherein it is determined whether the light enters the photodiode. The determination unit determines whether the incident angle of the outermost light ray is within a preset angle range among the subject light beams incident on the region to be imaged in the effective pixel region, thereby determining at least the partial light beam. The imaging apparatus according to claim 10 , wherein it is determined whether the part is incident on the specific photodiode.

Images