JP7099584B2 - Imaging device - Google Patents

Description

The present invention relates to an image pickup apparatus .

An image pickup device that corrects a focusing error based on data on axial chromatic aberration of an image pickup lens and adjusts the focus of the image pickup lens is known (Patent Document 1). However, in the prior art, there is a problem that the color shift of the captured image due to the axial chromatic aberration of the imaging lens cannot be corrected.

Japanese Patent Application Laid-Open No. 2003-143618

According to the first aspect, the image pickup apparatus photoelectrically converts the light transmitted through the first region of the optical system into a first photoelectric conversion unit to generate an electric charge, and the light transmitted through the second region of the optical system. A second photoelectric conversion unit that converts and generates an electric charge, a third photoelectric conversion unit that photoelectrically converts light transmitted through the first region of the optical system to generate an electric charge, and the second region of the optical system. Based on the fourth photoelectric conversion unit that photoelectrically converts the light transmitted through the light to generate an electric charge, the first signal based on the electric charge generated by the first photoelectric conversion unit, and the electric charge generated by the second photoelectric conversion unit. At least one of the deviation between the third signal based on the charge generated by the third photoelectric conversion unit and the fourth signal based on the charge generated by the fourth photoelectric conversion unit based on the amount of deviation from the second signal. A correction unit for correcting the amount is provided.

The main part block diagram of the image pickup apparatus which concerns on 1st Embodiment. The figure which shows the arrangement example of the pixel of the image pickup element which concerns on 1st Embodiment. The figure for demonstrating the light flux incident on the image pickup element which concerns on 1st Embodiment. The figure which shows the signal by the pixel of the image pickup element which concerns on 1st Embodiment. The figure which shows the signal of each color component standardized by the image pickup apparatus which concerns on 1st Embodiment, and the figure which compares the signal of each color component. The flowchart which shows the operation example of the image pickup apparatus which concerns on 1st Embodiment. The figure which shows the signal by the pixel of the image pickup element which concerns on 2nd Embodiment. The figure which shows the signal of each color component standardized by the image pickup apparatus which concerns on 2nd Embodiment, and the figure which compares the signal of each color component. The flowchart which shows the operation example of the image pickup apparatus which concerns on 2nd Embodiment.

(First Embodiment)
FIG. 1 is a configuration diagram of a main part of a digital camera 1 (hereinafter referred to as a camera 1) which is an image pickup apparatus according to the first embodiment. The camera 1 is composed of a camera body 2 and an interchangeable lens 3. The interchangeable lens 3 is attached to the camera body 2 via a mount portion (not shown). When the interchangeable lens 3 is attached to the camera body 2, the connection portion 202 on the camera body 2 side and the connection portion 302 on the interchangeable lens 3 side are connected, and communication between the camera body 2 and the interchangeable lens 3 becomes possible.

The interchangeable lens 3 includes an imaging optical system (imaging optical system) 31, a lens control unit 32, and a lens memory 33. The imaging optical system 31 is composed of a plurality of lenses including a focus adjusting lens (focus lens) and an aperture, and forms a subject image on the imaging surface of the imaging element 22 of the camera body 2. The lens control unit 32 moves the focus adjustment lens forward and backward in the optical axis L1 direction based on the signal output from the body control unit 21 of the camera body 2 to adjust the image formation position of the image formation optical system 31. The signal output from the body control unit 21 includes a signal indicating the movement amount, movement direction, movement speed, and the like of the focus adjustment lens. The lens memory 33 is composed of a non-volatile storage medium or the like, and stores information related to the interchangeable lens 3, for example, lens information such as information regarding the position of the exit pupil of the imaging optical system 31. The lens information stored in the lens memory 33 is read out by the lens control unit 32 and transmitted to the body control unit 21.

The camera body 2 includes a body control unit 21, an image sensor 22, a memory 23, a display unit 24, an operation unit 25, an electronic viewfinder (EVF) 26, and an eyepiece lens 27. The image pickup element 22 is an image sensor such as a CCD or CMOS, and a plurality of pixels are arranged two-dimensionally (row direction and column direction) on the image pickup element 22. The image sensor 22 receives the luminous flux that has passed through the exit pupil of the imaging optical system 31 by the two photoelectric conversion units of each pixel, and generates a pair of signals (two signals) according to the amount of light received. A pair of signals are output to the body control unit 21. The plurality of pixels of the image pickup device 22 have, for example, R (red), G (green), and B (blue) color filters, respectively. Each pixel captures a subject image through a color filter.

The memory 23 is a recording medium such as a memory card, and the body control unit 21 writes and reads image data, audio data, and the like. The display unit 24 displays an image corresponding to the image data generated by the body control unit 21. In addition, the display unit 24 displays various information (shutter speed, aperture value, ISO sensitivity, etc.) related to shooting conditions, a menu screen, and the like. The operation unit 25 includes a release button, a recording button, various setting switches, and the like, and outputs an operation signal corresponding to the operation of the operation unit 25 to the body control unit 21. The electronic viewfinder 26 displays an image corresponding to the image data generated by the body control unit 21. Further, the electronic viewfinder 26 displays various information related to the shooting conditions. The image and various information displayed on the electronic viewfinder 26 are observed by the user via the eyepiece 27.

The body control unit 21 is composed of a CPU, ROM, RAM, etc., and controls each unit of the camera 1 based on a control program. Further, the body control unit 21 performs various signal processing. For example, the body control unit 21 supplies a control signal to the image sensor 22 to control the operation of the image sensor 22. The body control unit 21 has a focus detection unit 28 and an image processing unit 29.

The focus detection unit 28 calculates the defocus amount by the pupil division type phase difference detection method using a pair of signals output from the image sensor 22, and transmits the defocus amount to the lens control unit 32. In other words, the focus detection unit 28 calculates the amount of deviation between the image forming surface of the image by the image pickup optical system 31 and the image pickup surface of the image pickup element 22 by using the pair of signals output from the image pickup element 22. The body control unit 21 generates information on the movement amount, the movement direction, and the like of the focus adjustment lens from the deviation amount, and transmits the information to the lens control unit 32 via the connection unit 202 and the connection unit 302. The lens control unit 32 drives a motor (not shown) based on the information transmitted from the body control unit 21 to form a focus adjustment lens at a position where an image formed by the image pickup optical system 31 is formed on the image pickup surface of the image pickup element 22. That is, move it to the in-focus position.

The image processing unit 29 generates image data based on a pair of signals output from the image sensor 22. The image processing unit 29 has an acquisition unit 29a and a processing unit 29b. Although details will be described later, the acquisition unit 29a acquires a pair of signals for each of a plurality of color components (plurality of color light) of light passing through different regions of the exit pupil of the imaging optical system 31 from the image pickup device 22. .. The processing unit 29b adds a pair of signals acquired by the acquisition unit 29a to generate an image signal. Further, the processing unit 29b performs various image processing on the image signal to generate image data.

Even after focusing the imaging optical system 31, the light of color components having different wavelengths such as R, G, and B according to the axial chromatic aberration has different focal positions (focusing positions). Therefore, in the image generated by imaging the subject image formed by the imaging optical system 31, color shift occurs due to the influence of the axial chromatic aberration by the imaging optical system 31. Therefore, in the present embodiment, the acquisition unit (input unit) 29a acquires a pair of signals for each of the RGB colors. The processing unit (correction unit) 29b calculates the amount of deviation between the pair of signals for each color, that is, the amount of deviation between the pair of images for each color according to the axial chromatic aberration. Further, the processing unit (correction unit, image generation unit) 29b generates an image signal corrected for axial chromatic aberration by synthesizing a pair of signals for each color based on the calculated deviation amount for each color. Therefore, it is possible to reduce the color shift that occurs in the image generated based on the image signal. This will be described in detail below.

FIG. 2 is a diagram showing an example of arranging pixels of the image pickup device 22 according to the first embodiment. In the image pickup device 22, the pixels 12 are arranged two-dimensionally (row direction and column direction). Each pixel 12 is provided with, for example, one of three color filters having different spectral sensitivities of R (red), G (green), and B (blue). The R color filter mainly transmits light in the red wavelength region, the G color filter mainly transmits light in the green wavelength region, and the B color filter mainly transmits light in the blue wavelength region. .. The color of the color filter arranged in the pixel 12 is schematically represented by notation as "R", "G" or "B". The pixel used as a dedicated pixel for focus detection may be provided with a filter (white filter) that transmits the entire wavelength range of the light incident on the pixel.

Each pixel 12 has different spectral sensitivity characteristics depending on the color filter arranged. In the image pickup element 22, the pixel group 401 in which the pixels 12 having the R and G color filters are alternately arranged, and the pixel 12 having the G and B color filters (hereinafter, the pixels having the R, G and B color filters). The pixel group 402 in which R pixels, G pixels, and B pixels are alternately arranged, respectively, is repeatedly arranged in a two-dimensional manner. In this way, the R pixel, the G pixel, and the B pixel are arranged according to the Bayer arrangement.

The pixel 12 has a microlens 40, and a photoelectric conversion unit 41 and a photoelectric conversion unit 42 arranged side by side in the horizontal direction (row direction). In FIG. 2, the photoelectric conversion unit 41 and the photoelectric conversion unit 42 are arranged side by side in the row direction (X-axis direction shown in FIG. 2), that is, in the horizontal direction, but in the column direction (Y-axis direction shown in FIG. 2). That is, they may be arranged side by side in the vertical direction.

FIG. 3 is a diagram for explaining a light flux incident on the image pickup device according to the first embodiment. In the example shown in FIG. 3, for simplification of the explanation, only three pixels are shown in the pixel 12. As described above, the pixel 12 includes a microlens 40, a color filter 43, and a photoelectric conversion unit 41 and a photoelectric conversion unit 42 that receive a light beam transmitted through the microlens 40 and the color filter 43. Each pixel 12 is provided with a microlens 40, a color filter 41, a photoelectric conversion unit 41, and a photoelectric conversion unit 42 in the plus direction of the Z axis.

Light that has passed through different regions of the exit pupil 60 of the imaging optical system 31 is incident on the photoelectric conversion unit 41 and the photoelectric conversion unit 42 of each pixel 12. The first luminous flux that has passed through the first pupil region 61 is incident on the photoelectric conversion unit 42 via the microlens 40, and the second luminous flux that has passed through the second pupil region 62 is photoelectrically converted via the microlens 40. It is incident on the portion 41. The photoelectric conversion unit 41 receives light that has passed through the second pupil region 62, and generates an electric charge according to the amount of light received. The photoelectric conversion unit 42 receives light that has passed through the first pupil region 61, and generates an electric charge according to the amount of light received. Each pixel 12 outputs as a pair of signals a signal based on the electric charge generated by the photoelectric conversion unit 41 and a signal based on the electric charge generated by the photoelectric conversion unit 42. In the following description, the signal based on the charge generated by the photoelectric conversion unit 41 is referred to as a signal A, and the signal based on the charge generated by the photoelectric conversion unit 42 is referred to as a signal B. As described above, each pixel 12 transmits a pair of signals of the light-based signal B passing through the first pupil region 61 and the light-based signal A passing through the second pupil region 62 to the body control unit 21. Output.

Further, since each pixel 12 has a color filter of any of R, G, and B, the pair of signals output from the pixel 12 is a signal corresponding to any of the color components of R, G, or B. Hereinafter, the signal A and the signal B output from the R pixel will be referred to as a signal Ra and a signal Rb, respectively. Similarly, the signal A and the signal B output from the G pixel are referred to as a signal Ga and a signal Gb, respectively, and the signal A and the signal B output from the B pixel are referred to as a signal Ba and a signal Bb, respectively.

The focus detection unit 28 performs a correlation calculation based on, for example, a pair of signals of the signal Ga and the signal Gb output from each G pixel. By this correlation calculation, the focus detection unit 28 determines the amount of deviation (phase difference information) between the image of the first luminous flux passing through the first pupil region 61 and the image of the second luminous flux passing through the second pupil region 62. ) Is calculated, and the defocus amount is calculated based on this image shift amount.

FIG. 4 is a diagram showing a signal by the pixels of the image pickup device according to the first embodiment. The horizontal axis indicates the arrangement direction of the plurality of pixels 12, and the vertical axis indicates the signal level for each color component by each pixel 12. FIGS. 4A to 4C show the distribution of signals from each pixel 12 in a direction orthogonal to the white line (for example, the horizontal direction) when a subject in which one white line is arranged on a black background is imaged. show. Further, FIGS. 4A to 4C show signals from each pixel 12 after focusing using the phase difference information by a pair of signals from the G pixel. FIG. 4A shows the distribution of B component signals (signals Ba and Bb) by each B pixel, and FIG. 4B shows the distribution of G component signals (signals Ga and Gb) by each G pixel and FIG. 4 (c). ) Indicates the distribution of the signal (signal Ra, Rb) of the R component by each R pixel.

Since axial chromatic aberration occurs in the image pickup optical system 31, the focal positions of the peaks of the signal A and the signal B differ according to the light of each color component of RGB. Therefore, as shown in FIGS. 4A to 4C, the amount of deviation (phase difference) between the signal A and the signal B of each color component is different. After the acquisition unit 29a acquires a pair of signals for each RGB component of light that has passed through different pupil regions, the processing unit 29b determines the amount of deviation of the pair of signals for each RGB component, that is, the amount of deviation between the pair of images. Is calculated. Since the subject is focused with the G component, the amount of deviation of the pair of signals (signals Ga and Gb) of the G component shown in FIG. 4B is substantially zero, and the pair of images of the G component among the RGB components The amount of deviation is the smallest. With respect to the B component and the R component shown in FIGS. 4A and 4C, a phase shift occurs between the pair of signals due to axial chromatic aberration.

The processing unit 29b performs correction processing for correcting a pair of signals of other color components different from the reference color component based on the deviation amount of the reference color component that minimizes the deviation amount of the image. For example, the G component that minimizes the deviation amount is used as the reference color component, and the signal Ba and the signal Bb are moved (shifted) in parallel so that the deviation amount of the B component matches the deviation amount of the G component. Similarly, the signal Ra and the signal Rb are shifted so that the deviation amount of the R component matches the deviation amount of the G component. Further, the signals of the B component and the R component are shifted so that the peak positions of the pair of signals due to the color components match. The processing unit 29b shifts the signals Ba and Bb so that the amount of deviation between the signal Ba and the signal Bb is reduced based on the amount of deviation between the signal Ga and the signal Gb, and the amount of deviation between the signal Ra and the signal Rb is increased. A correction process is performed to shift the signals Ra and Rb so as to decrease.

4 (d) to 4 (f) show the signal distribution of each color component after the correction process. 4 (d) shows the distribution of the signal of the B component, FIG. 4 (e) shows the distribution of the signal of the G component, and FIG. 4 (f) shows the distribution of the signal of the R component. The signal is not corrected for the G component that minimizes the deviation amount. The processing unit 29b generates a signal obtained by synthesizing the signal A and the signal B of each color component after the correction processing. Specifically, the processing unit 29b generates additional signals Ba + b, Ga + b, and Ra + b, which are the sum of the signals A and B of each color component, as image signals, as shown in FIGS. 4 (d) to 4 (f), respectively. .. Further, for comparison, FIGS. 4A to 4C show an addition signal obtained by adding the signal A and the signal B of each color component before the correction processing. The addition signal Ba + b is a signal obtained by adding the signal Ba and the signal Bb, the addition signal Ga + b is a signal obtained by adding the signal Ga and the signal Gb, and the addition signal Ra + b is a signal obtained by adding the signal Ra and the signal Rb. .. In this way, the processing unit 29b shifts the pair of signals of the R component and the B color component different from the G component, and then adds the pair of signals of each color component to generate an image signal. Corrects color shift due to top chromatic aberration.

FIG. 5 is a diagram showing signals of each color component standardized by the image pickup apparatus according to the first embodiment and a diagram comparing signals of each color component. FIGS. 5A and 5B are diagrams showing the results of normalization (normalization) so that the peak value of the signal of each color component becomes a predetermined value (for example, 1). 5 (a) is the result of normalization of the addition signal before the correction process shown in FIGS. 4 (a) to 4 (c), and FIG. 5 (b) is a result of FIGS. 4 (d) to 4 (f). This is the result of standardization of the added signal after the correction processing shown in.

5 (c) and 5 (d) are diagrams for comparing the addition signals of each standardized color component. 5 (c) is a diagram comparing the addition signals of each color component before the correction process shown in FIG. 5 (a), and FIG. 5 (d) is a diagram showing each color component after the correction process shown in FIG. 5 (b). It is a figure which compares the addition signal of. B / G-1 and R / G-1 shown in FIGS. 5 (c) and 5 (d) are values obtained by subtracting 1 from the ratio of the addition signal Ba + b of the B component and the addition signal Ga + b of the G component, respectively, and the R component. It is a value obtained by subtracting 1 from the ratio of the addition signal Ra + b of the above and the addition signal Ga + b of the G component. As shown in FIG. 5, in the region where the signal level of each color component changes significantly, that is, in the boundary region between the white line and the black background, which is the subject, B / G-1 and R / G-1 are affected by the influence of axial chromatic aberration. The absolute value is large, and color shift occurs in the image generated by the signal of each color component.

Therefore, in the present embodiment, as described above, the processing unit 29b performs correction processing so that the shapes of the waveforms due to the signals of the respective color components match. The processing unit 29b shifts and adds a pair of signals so that the difference between the color components becomes small in the boundary region. As a result, after the correction process shown in FIG. 5 (d), the difference between the signals of each color component becomes smaller than that before the correction process shown in FIG. 5 (c). In particular, the value of B / G-1 at the pixel position P has changed significantly, and the value after the correction process shown in FIG. 5 (d) is about the same as the value before the correction process shown in FIG. 5 (c). It is reduced by 15%. In this way, the processing unit 29b generates an image signal corrected for axial chromatic aberration by synthesizing a pair of signals for each color based on the calculated deviation amount for each color, and uses the image signal to relate to the subject image. Generate image data. Therefore, it is possible to suppress color shift caused by axial chromatic aberration. An image based on an image signal with axial chromatic aberration corrected may be displayed as a through image (live view image) displayed on the display unit 24, and the image corrected with axial chromatic aberration may be displayed as a through image. Is displayed.

FIG. 6 is a flowchart showing an operation example of the image pickup apparatus according to the first embodiment. The process shown in FIG. 6 is executed, for example, when the operation unit 25 is operated by the user and shooting is started.

In step S100, the body control unit 21 of the camera body 2 acquires a pair of signals of each color component output from the image sensor 22 as a focus detection signal. In step S110, the body control unit 21 calculates the defocus amount for each color by using the focus detection signal for each color. In step S120, the body control unit 21 calculates, for example, a range on the image pickup surface of the image pickup device 22 in which the defocus amount is equal to or less than a predetermined threshold value, and determines it as a correction range.

In step S130, the body control unit 21 detects the reference color component having the minimum deviation amount based on the deviation amount of the pair of signals of each color component in the correction range. Further, the body control unit 21 calculates the shift amount of a pair of signals of other color components different from the reference color component based on the deviation amount of the reference color component. In step S140, the body control unit 21 performs the above-mentioned correction processing on the signal from each pixel 12 corresponding to the calculated correction range to generate an image signal. That is, the body control unit 21 performs a process of shifting the pair of signals based on the determined shift amount and then adding the correction range on the imaging surface. The body control unit 21 performs various image processing on the generated image signal to generate image data, and ends the processing shown in FIG.

According to the above-described embodiment, the following effects can be obtained.
(1) The image processing apparatus (image processing unit 29) has an acquisition unit 29a for acquiring a pair of signals for each of a plurality of color components of light passing through different regions of the exit pupil 60 of the imaging optical system 31 and an acquisition unit 29a. A processing unit 29b that generates an image signal corrected for axial chromatic aberration caused by the imaging optical system 31 based on a pair of signals acquired by the unit 29a is provided. Since this is done, it is possible to reduce the color shift that occurs in the image due to the influence of axial chromatic aberration.
(2) The processing unit 29b calculates the amount of deviation of the image formed by the imaging optical system 31 using the pair of signals, and synthesizes the pair of signals based on the amount of deviation to generate an image signal. Since this is done, it is possible to calculate the amount of deviation between the pair of images for each color according to the axial chromatic aberration. Further, by synthesizing a pair of signals for each color based on the calculated amount of deviation for each color, it is possible to generate an image signal in which the color deviation due to axial chromatic aberration is corrected.

(3) The processing unit 29b is different from the reference color component based on the deviation amount of the reference color component that minimizes the deviation amount of the image formed by the imaging optical system 31 among the plurality of color components. A pair of color component signals are shifted and combined to generate an image signal. In the present embodiment, a pair of signals of other color components different from the reference color component are shifted in the most focused direction, and correction is made so that the amount of deviation of the other color components is reduced. As a result, it is possible to reduce the color shift that occurs in the image.
(4) The processing unit 29b shifts and synthesizes a pair of signals of the other color components so that the deviation amount of the other color components matches the deviation amount of the reference color component. Since this is done, the correction process can be performed so that the shapes of the waveforms due to the signals of the respective color components match, and the color shift can be reduced. In addition, it is possible to obtain an image in which each color component is in focus.

(Second embodiment)
The image pickup apparatus according to the second embodiment will be described with reference to FIGS. 7 to 9. In the figure, the same or corresponding parts as those of the first embodiment are designated by the same reference numbers, and the differences from the image pickup apparatus according to the first embodiment will be mainly described. In the first embodiment, an example in which a pair of signals of other color components different from the reference color component is corrected to generate an image signal based on the deviation amount of the reference color component that minimizes the deviation amount of the image. Explained. In the second embodiment, an example in which a pair of signals of other color components different from the reference color component is corrected to generate an image signal based on the deviation amount of the reference color component that maximizes the deviation amount of the image. Will be explained.

FIG. 7 is a diagram showing a signal by the pixels of the image pickup device according to the second embodiment. The imaging conditions are the same as in the case of the first embodiment, FIGS. 7 (a) to 7 (c) show the signal distribution of each color component before correction, and FIGS. 7 (d) to 7 (f) show the signal distribution after correction. The distribution of the signal of each color component of is shown.

In the example shown in FIG. 7, the processing unit 29b uses the B component as a reference color component, and shifts the signal Ra and the signal Rb so that the deviation amount of the R component matches the deviation amount of the B component. Further, the processing unit 29b shifts the signal Ga and the signal Gb so that the deviation amount of the G component matches the deviation amount of the B component. For a pair of G component signals, a digital low-pass filter is applied to the shifted signal Ga and signal Gb so that the shape of the waveform of the signal of the G component and the shape of the waveform of the signal of the B component more closely match. ing. FIG. 7E shows the signal Ga and the signal Gb after being filtered by the digital low-pass filter. The signal Ga + b shown in FIG. 7 (e) is a signal obtained by adding the filtered signal Ga and the signal Gb. In this way, the processing unit 29b ensures that the shape of the signal waveform due to the reference color component that maximizes the image shift amount matches the shape of the signal waveform due to other color components different from the reference color component. Perform correction processing. It should be noted that the filter processing may also be performed on the pair of signals of the R component.

FIG. 8 is a diagram showing signals of each color component standardized by the image pickup apparatus according to the second embodiment and a diagram comparing signals of each color component. 8 (a) is the result of standardizing the addition signal before the correction process shown in FIGS. 7 (a) to 7 (c), and FIG. 8 (b) is a result of FIGS. 7 (d) to 7 (f). This is the result of standardization of the added signal after the correction processing shown in. 8 (c) is a diagram comparing the addition signals of each color component before the correction process shown in FIG. 8 (a), and FIG. 8 (d) is a diagram after the correction process shown in FIG. 8 (b). It is a figure which compares the addition signal of each color component.

By performing the correction process described above, after the correction process shown in FIG. 8 (d), the difference between the signals of each color component is smaller than that before the correction process shown in FIG. 8 (c). The value of R / G-1 at each pixel position shown in FIG. 8D is close to zero. As a result, it is possible to suppress the color shift of the image caused by the axial chromatic aberration.

For example, in the correction range in which the defocus amount is equal to or less than a predetermined threshold value, for example, in the image region of the main subject, the body control unit 21 performs the same correction processing as in the case of the first embodiment to perform color due to axial chromatic aberration. Reduce the deviation. Further, in a correction range in which the defocus amount is larger than a predetermined threshold value, for example, in an image region of a subject different from the main subject, the correction processing according to the second embodiment is performed to reduce the color shift due to axial chromatic aberration.

FIG. 9 is a flowchart showing an operation example of the image pickup apparatus according to the second embodiment. The process shown in FIG. 9 is executed, for example, when the operation unit 25 is operated by the user and shooting is started.

In step S200, the body control unit 21 of the camera body 2 acquires a pair of signals of each color component output from the image sensor 22 as a focus detection signal. In step S210, the body control unit 21 calculates the defocus amount for each color component. In step S220, the body control unit 21 calculates, for example, a range on the imaging surface where the defocus amount is equal to or less than a predetermined threshold value as the first correction range, and the defocus amount is larger than the predetermined threshold value on the imaging surface. The range is calculated as the second correction range.

In step S230, the body control unit 21 detects the reference color component having the maximum deviation amount based on the deviation amount of the pair of signals of each color component in the second correction range. Further, the body control unit 21 determines the shift amount of a pair of signals of other color components different from the reference color component based on the deviation amount of the reference color component that maximizes the deviation amount. Regarding the first correction range, as in the case of the first embodiment, the body control unit 21 has a pair of color components of each color component based on the deviation amount of the color component that minimizes the deviation amount in the first correction range. Determine the amount of signal shift.

In step S240, the body control unit 21 determines a filter to be used in the correction process. For example, the body control unit 21 performs correction processing on the signal by each pixel 12 corresponding to the first correction range without using a filter. Further, the body control unit 21 performs correction processing using a filter on the signal by each pixel 12 corresponding to the second correction range. For example, the body control unit 21 selects a filter to be used for the correction process according to the amount of blurring of the image.

In step S250, the body control unit 21 corrects the signal from each pixel 12 corresponding to the second correction range based on the amount of deviation of the color component that maximizes the amount of deviation of the image. .. Further, the body control unit 21 performs signal correction processing on the signal from each pixel 12 corresponding to the first correction range based on the deviation amount of the color component that minimizes the deviation amount of the image. That is, the body control unit 21 shifts the pair of signals based on the determined shift amount for the first correction range and then adds them, and shifts the pair of signals for the second correction range based on the determined shift amount. After that, filter processing is performed and then addition is performed to generate an image signal. The body control unit 21 performs various image processing on the generated image signal to generate image data, and ends the processing shown in FIG.

According to the above-described embodiment, in addition to the same action and effect as in the first embodiment, the following action and effect can be obtained.
(5) The processing unit 29b is different from the reference color component based on the deviation amount of the reference color component that maximizes the deviation amount of the image formed by the imaging optical system 31 among the plurality of color components. A pair of color component signals are shifted and combined to generate an image signal. In the present embodiment, a pair of signals of other color components different from the reference color component are shifted in the most blurred direction, and correction is made so that the deviation amounts of the respective color components match. As a result, it is possible to reduce the color shift that occurs in the image. In addition, it is possible to obtain an image with a bokeh effect.

(6) The processing unit 29b outputs a pair of signals based on the amount of deviation of the color component that minimizes the amount of deviation of the image formed by the imaging optical system 31 among the plurality of color components in the image region of the main subject. In the image area of a subject different from the main subject, a pair of signals are combined based on the deviation amount of the color component having the maximum deviation amount among the plurality of color components. In the present embodiment, for example, in the image region of the main subject, the image of the reference color component that is in the most focused state and the image of another color component different from the reference color component are corrected so as to match. Further, for example, in an image region other than the main subject, the image due to the reference color component that is in the most blurred state and the image due to another color component different from the reference color component are corrected so as to match. As a result, it is possible to reduce color shift due to axial chromatic aberration in the image area of the main subject and the image area other than the main subject. Further, for the image area of the main subject, it is possible to obtain an image in which each color component is in focus, and for the image area other than the main subject, it is possible to obtain an image to which the bokeh effect is imparted.

The following modifications are also within the scope of the present invention, and one or more of the modifications can be combined with the above-described embodiment.

(Modification 1)
In the above-described embodiment, a 2PD configuration having two photoelectric conversion units in one pixel has been described as an example, but the configuration of each pixel is not limited to this. For example, a 4PD configuration having four photoelectric conversion units per pixel may be used.

(Modification 2)
In the above-described embodiment, an example of determining the correction range based on the defocus amount has been described. However, the correction range may be determined based on the Depth Map, the MTF (Modulation Transfer Function) of the lens, or the like, and the content of the correction processing for the pair of signals of each color component for the correction range may be determined. ..

(Modification 3)
In the above-described embodiment, an example of correcting the color shift due to axial chromatic aberration by correcting the shift amount of each color component to match based on the shift amount of the reference color component has been described. However, in the region where the amount of blurring is desired to be large, the correction process based on the amount of deviation of the reference color component may not be performed. In this case, the correction process may not be performed individually for each color. For example, after applying a blur filter according to the amount of blurring to the pair of signals of each color component, the pair of signals of each color component are combined. As a result, it is possible to obtain an image in which the blur is emphasized in the region where the amount of blur is desired to be increased.

(Modification example 4)
In the above-described embodiment, a 2PD configuration having two photoelectric conversion units in one pixel has been described as an example, but the configuration of each pixel is not limited to this. For example, a 4PD configuration having four photoelectric conversion units per pixel may be used.

(Modification 5)
In the above-described embodiments and modifications, an example in which the present invention is applied to a digital camera as an image processing device has been described. However, the present invention describes, for example, a smartphone, a tablet, a personal computer, a camera built in a PC, an in-vehicle camera, or the like. It can be applied to other devices.

(Modification 6)
In the above-described embodiments and modifications, the example in which the image processing device is provided in the image pickup device has been described, but the image processing device may be realized by a computer. In this case, the image processing apparatus is configured by causing a computer (or a CPU or the like) to execute a program that performs processing based on the flowcharts illustrated in FIGS. 6 and 9. The program can be supplied as various forms of computer program products, such as those provided via storage media and communication lines.

The present invention also includes the following camera bodies.
(1) A plurality of first light receiving units having a first photoelectric conversion unit and a second photoelectric conversion unit to receive the first color light, and a third photoelectric conversion unit and a fourth photoelectric conversion unit are provided. The plurality of second photoelectric conversion units received light with respect to the positions of the plurality of second light receiving units that receive the second color light and the image of the subject received by the plurality of first photoelectric conversion units. Correction to shift the position of the image of the subject is performed on the signals output from the plurality of first light receiving units, and the image of the subject received by the plurality of first photoelectric conversion units and the second photoelectric conversion are performed. Reduce the difference between the image of the subject received by the unit and the image of the subject received by the plurality of third photoelectric conversion units and the image of the subject received by the fourth photoelectric conversion unit. An image pickup apparatus having a correction unit, an image generation unit that generates an image signal by a signal from the plurality of second light receiving units, and a corrected signal from the plurality of first light receiving units.
(2) In the image pickup apparatus as described in (1), the difference between the image of the subject received by the plurality of first photoelectric conversion units and the image of the subject received by the plurality of second photoelectric conversion units is the plurality of deviations. It is larger than the deviation between the image of the subject received by the third photoelectric conversion unit and the image of the subject received by the plurality of fourth photoelectric conversion units.
(3) In an image pickup apparatus such as (2), the correction unit receives light from the position of the image of the subject received by the plurality of first photoelectric conversion units by the plurality of second photoelectric conversion units. The position of the image of the subject is corrected to shift, and the deviation between the image of the subject received by the plurality of first photoelectric conversion units and the image of the subject received by the plurality of second photoelectric conversion units is reduced.
(4) In the image pickup apparatus as described in (1), the difference between the image of the subject received by the plurality of first photoelectric conversion units and the image of the subject received by the plurality of second photoelectric conversion units is the plurality of deviations. It is smaller than the deviation between the image of the subject received by the third photoelectric conversion unit and the image of the subject received by the plurality of fourth photoelectric conversion units.
(5) In an image pickup apparatus such as (4), the correction unit receives light from the position of the image of the subject received by the plurality of first photoelectric conversion units by the plurality of second photoelectric conversion units. The position of the image of the subject is corrected to shift, and the deviation between the image of the subject received by the plurality of first photoelectric conversion units and the image of the subject received by the plurality of second photoelectric conversion units is increased.
(6) In the image pickup apparatus as in (5), the correction unit applies a low-pass filter to the corrected signals from the plurality of first light receiving units.
(7) In the imaging apparatus such as (1) to (6), the correction unit receives signals from the plurality of second photoelectric conversion units with respect to signals from the plurality of first photoelectric conversion units. Make corrections to shift.
(8) In an image pickup apparatus such as (1), the correction unit receives light from a main subject among the plurality of first light receiving units and the plurality of second light receiving units. In the light receiving unit and the second light receiving unit, the color having the larger deviation of the image of the subject is corrected as the first color, and the plurality of first light receiving units and the plurality of second light receiving units are corrected. Of the above, the first light receiving unit and the second light receiving unit that receive light from a subject other than the main subject perform correction with the color having the smaller image deviation of the subject as the second color. ..
(9) A plurality of first light receiving units having a first photoelectric conversion unit and a second photoelectric conversion unit to receive the first color light, and a third photoelectric conversion unit and a fourth photoelectric conversion unit are provided. Positions of an input unit for inputting an image signal captured by an image pickup unit having a plurality of second light receiving units for receiving the second color light, and an image of a subject received by the plurality of first photoelectric conversion units. On the other hand, the correction for shifting the position of the image of the subject received by the plurality of second photoelectric conversion units is performed on the signals output from the plurality of first light receiving units, and the plurality of first photoelectric conversion units are converted. The difference between the image of the subject received by the unit and the image of the subject received by the plurality of second photoelectric conversion units, the image of the subject received by the plurality of third photoelectric conversion units, and the plurality of fourth photoelectrics. An image is formed by a correction unit that reduces the difference between the image of the subject received by the conversion unit and the deviation of the image of the subject, the signals from the plurality of second light receiving units, and the corrected signals from the plurality of first light receiving units. An image processing device having an image generation unit that generates a signal.
(10) In an image processing apparatus such as (9), the difference between the image of the subject received by the plurality of first photoelectric conversion units and the image of the subject received by the plurality of second photoelectric conversion units is the above. It is larger than the deviation between the image of the subject received by the plurality of third photoelectric conversion units and the image of the subject received by the plurality of fourth photoelectric conversion units.
(11) In an image processing apparatus such as (10), the correction unit is a plurality of second photoelectric conversion units with respect to the position of an image of a subject received by the plurality of first photoelectric conversion units. Correction is performed to shift the position of the image of the received subject, and the deviation between the image of the subject received by the plurality of first photoelectric conversion units and the image of the subject received by the plurality of second photoelectric conversion units is reduced.
(12) In an image processing apparatus such as (9), the difference between the image of the subject received by the plurality of first photoelectric conversion units and the image of the subject received by the plurality of second photoelectric conversion units is the above. It is smaller than the deviation between the image of the subject received by the plurality of third photoelectric conversion units and the image of the subject received by the plurality of fourth photoelectric conversion units.
(13) In an image processing apparatus such as (12), the correction unit is a plurality of second photoelectric conversion units with respect to the position of an image of a subject received by the plurality of first photoelectric conversion units. Correction is performed to shift the position of the image of the received subject, and the deviation between the image of the subject received by the plurality of first photoelectric conversion units and the image of the subject received by the plurality of second photoelectric conversion units is increased.
(14) In an image processing apparatus such as (13), the correction unit applies a low-pass filter to the corrected signals from the plurality of first light receiving units.
(15) In the image processing apparatus as described in (9) to (14), the correction unit may be used with respect to the signals from the plurality of first photoelectric conversion units from the plurality of second photoelectric conversion units. Make corrections to shift the signal.
(16) In an image processing apparatus such as (9), the correction unit receives light from a main subject among the plurality of first light receiving units and the plurality of second light receiving units. In the light receiving unit and the second light receiving unit, the color having the larger deviation of the image of the subject is corrected as the first color, and the plurality of first light receiving units and the plurality of second light receiving units are corrected. In the first light receiving section and the second light receiving section that receive light from a subject other than the main subject, the color with the smaller image deviation of the subject is used as the second color for correction. conduct.

2 ... Camera body, 3 ... Interchangeable lens, 21 ... Body control unit, 22 ... Image sensor, 29 ... Image processing unit, 29a ... Acquisition unit, 29b ... Processing unit, 31 ... Imaging optical system

Claims (9)

A first photoelectric conversion unit that photoelectrically converts light transmitted through the first region of an optical system to generate an electric charge, and a first photoelectric conversion unit.
A second photoelectric conversion unit that photoelectrically converts light transmitted through the second region of the optical system to generate an electric charge, and a second photoelectric conversion unit.
A third photoelectric conversion unit that photoelectrically converts light transmitted through the first region of the optical system to generate an electric charge, and a third photoelectric conversion unit.
A fourth photoelectric conversion unit that photoelectrically converts light transmitted through the second region of the optical system to generate an electric charge, and a fourth photoelectric conversion unit.
It is generated by the third photoelectric conversion unit based on the amount of deviation between the first signal based on the charge generated by the first photoelectric conversion unit and the second signal based on the charge generated by the second photoelectric conversion unit. A correction unit that corrects the amount of deviation of at least one of the third signal based on the electric charge and the fourth signal based on the electric charge generated by the fourth photoelectric conversion unit.
An image pickup device equipped with. In the image pickup apparatus according to claim 1 ,
The correction unit sets the third signal and the fourth signal so that the amount of deviation between the third signal and the fourth signal matches the amount of deviation between the first signal and the second signal. Image pickup device for correction. In the image pickup apparatus according to claim 1 or 2 .
The correction unit is an image pickup device that corrects at least one of the third signal and the fourth signal based on the minimum deviation amount between the first signal and the second signal. In the image pickup apparatus according to claim 1 or 2 .
The correction unit is an image pickup device that corrects at least one of the third signal and the fourth signal based on the maximum deviation amount between the first signal and the second signal. In the image pickup apparatus according to any one of claims 1 to 4 .
An imaging device including a detection unit that detects the amount of deviation between the first signal and the second signal and the amount of deviation between the third signal and the fourth signal. In the image pickup apparatus according to any one of claims 1 to 5 ,
The first filter that transmits light of the first wavelength and
A second filter that transmits light of a second wavelength is provided.
The first photoelectric conversion unit and the second photoelectric conversion unit photoelectrically convert the light transmitted through the first filter.
The third photoelectric conversion unit and the fourth photoelectric conversion unit are image pickup devices that photoelectrically convert the light transmitted through the second filter. In the image pickup apparatus according to any one of claims 1 to 6 .
With the first microlens,
With a second microlens,
The first photoelectric conversion unit and the second photoelectric conversion unit photoelectrically convert the light transmitted through the first microlens.
The third photoelectric conversion unit and the fourth photoelectric conversion unit are image pickup devices that photoelectrically convert light transmitted through the second microlens. In the image pickup apparatus according to any one of claims 1 to 7 .
An image pickup apparatus including a generation unit that generates image data by adding the third signal and the fourth signal corrected by the correction unit. The image pickup apparatus according to any one of claims 1 to 8 .
An image pickup apparatus including a focus detection unit that detects the focus of the optical system based on the third signal and the fourth signal corrected by the correction unit.

Images