JP2019049616A - Image processing device, imaging device and method for controlling image processing device - Google Patents

Abstract

To provide an image processing device capable of calculating an influence of resampling on an image signal as a confidence.SOLUTION: An image processing device includes: a confidence generation unit 403 for allocating a confidence to a target pixel in accordance with the frequency of a signal outputted from an acquisition unit 401; a decimation unit 402 and an image addition processing unit 405 for resampling the signal outputted from the acquisition unit 401; a detection unit 406 for calculating distribution information corresponding to a distance on the basis of the signal outputted from the image addition processing unit 405, and calculating a confidence of the distribution information; and a combining unit 407 for combining the confidence outputted by the confidence generation unit 403 and the confidence outputted by the detection unit 406.SELECTED DRAWING: Figure 4

Description

  The present invention relates to an image processing apparatus, an imaging apparatus, and a control method for the image processing apparatus.

  In an image pickup apparatus such as a digital camera, there are known techniques for increasing the number of shots in continuous shooting, performing high-speed moving image shooting, and suppressing camera batteries when reading signals. Japanese Patent Application Laid-Open No. 2004-228688 discloses a configuration that can perform readout (resampling) with pixel thinning driving and pixel signal addition, and can capture images with low resolution.

JP 2008-278453 A JP 2008-015754 A

  However, in Patent Document 1, the image signal after the addition thinning causes “moire” in which a part of the image is altered due to an alias signal (folding component) having a false frequency, and the quality of the image is deteriorated. . Patent Document 2 discloses an image processing apparatus that detects a defocus amount distribution representing distance information from a relative displacement amount of a plurality of subject images in which light beams coming from different regions of a pupil of a photographing optical system are generated. Is disclosed. When the defocus distribution in Patent Document 2 is detected using an image signal including a aliasing component, an erroneous detection of the defocus amount occurs due to a false resolution subject.

  SUMMARY OF THE INVENTION An object of the present invention is to provide an image processing apparatus that can calculate an influence on an image signal due to resampling as reliability.

  In order to solve the above-described problem, an image processing apparatus according to the present invention assigns a reliability to a pixel of interest according to an extraction unit that extracts a predetermined frequency band from a signal output from an imaging unit and an output of the extraction unit. First reliability generation means.

  ADVANTAGE OF THE INVENTION According to this invention, the image processing apparatus which enabled calculation of the influence on the image signal by resampling as reliability can be provided.

It is a block diagram which shows the structure of an imaging device. It is a figure which shows arrangement | positioning of the pixel of an imaging part. It is a figure explaining the relationship between the light beam which emerges from an exit pupil, and the unit pixel of an imaging part. It is the block diagram which showed the function structure of the imaging part and the image process part. It is a flowchart explaining operation | movement of an imaging part and an image process part. It is a figure explaining a horizontal pixel addition thinning-out process. It is a figure explaining the filter coefficient of an addition process. It is a figure explaining the frequency characteristic of the filter coefficient of luminance addition. It is a figure explaining the to-be-photographed object of the band whose normalized frequency is 0.08. It is a figure explaining the to-be-photographed object of the band whose normalization frequency is 0.26. It is a figure explaining the frequency characteristic of a high pass filter. It is a figure explaining the frequency characteristic of a band pass filter. It is a figure explaining the frequency characteristic of a low-pass filter. It is a figure explaining the extraction space | interval of a return | return reliability map. It is a figure explaining an image addition process. It is the block diagram which showed the function structure of the imaging part and the image process part.

(First embodiment)
FIG. 1 is a block diagram illustrating a functional configuration of the imaging apparatus 100. The imaging device 100 is a digital camera, for example. The imaging apparatus 100 includes a system control unit 101, a ROM 102, a RAM 103, an optical system 104, an imaging unit 105, an image processing unit 106, a recording medium 107, and a bus 108. The set of blocks excluding the optical system 104 may be realized as an image processing apparatus.

  The system control unit 101 reads out an operation program for each block included in the imaging apparatus 100 from the ROM 102, develops the program in the RAM 103, and executes the program, thereby controlling the operation of each block included in the imaging apparatus 100. The system control unit 101 is, for example, a CPU (Central Processing Unit). A ROM (Read only memory) 102 is a rewritable nonvolatile memory, and stores parameters necessary for the operation of each block in addition to the operation program of each block included in the imaging apparatus 100. The exit pupil distance is also stored in the ROM 102 as lens information necessary for focus detection and the like. A RAM (Random Access Memory) 103 is a rewritable volatile memory, and is used as a temporary storage area for data output in the operation of each block included in the imaging apparatus 100.

  The optical system 104 forms a subject image on the imaging unit 105. The optical system 104 also includes a stop, and adjusts the light amount at the time of photographing by adjusting the aperture diameter of the optical system. The imaging unit 105 is an imaging device, photoelectrically converts an optical image formed on the imaging device by the optical system 104, applies A / D conversion processing to the obtained analog image signal, and applies the digital image signal to the RAM 103. Output and store. The imaging unit 105 is, for example, a CCD sensor or a CMOS sensor. Further, the imaging unit 105 of the present embodiment is configured to be able to perform readout with pixel thinning driving and addition of pixel signals, which is disclosed in Japanese Patent Application Laid-Open No. 2008-278453. Furthermore, the imaging unit 105 can perform filter processing such as a high-pass filter and a band-pass filter on the digital image signal after the A / D conversion processing.

  The image processing unit 106 performs various processes such as white balance adjustment, color interpolation, reduction / enlargement, and filtering on the image data stored in the RAM 103 and various image processes such as creation of a defocus map. The recording medium 107 is a detachable memory card or the like, and an image stored in the RAM 103 and processed by the image processing unit 106, an image A / D converted by the imaging unit 105, and the like are recorded as a recorded image. The bus 108 is used for exchanging signals between the blocks.

  FIG. 2 is a diagram illustrating an arrangement of pixels of the imaging unit 105. The imaging device of the imaging unit 105 includes a plurality of microlenses and a plurality of pixels corresponding to the plurality of microlenses. As shown in FIG. 2, in the imaging unit 105, unit pixels 200 (one pixel) are two-dimensionally arranged in a matrix. On each pixel, color filters of R (Red) / G (Green) / B (Blue) are arranged in a Bayer array. In addition, in the unit pixel 200, a sub-pixel a that is a first photoelectric conversion unit and a sub-pixel b that is a second photoelectric conversion unit are arranged. An FD (photodiode) 201a is disposed in the sub-pixel a, and an FD 201b is disposed in the sub-pixel b. The output signals of the sub-pixel a and the sub-pixel b are used for defocus amount detection. Further, a signal (a / b addition signal) obtained by adding the output signals of the sub-pixel a and the sub-pixel b is used for generating an image for photographing.

  FIG. 3 is a schematic diagram showing the relationship between the light beam emerging from the exit pupil of the optical system 104 and the unit pixel 200. A color filter 301 and a micro lens 302 are formed on the unit pixel 200. The optical axis 304 indicates the center of the light beam emitted from the exit pupil 303 with respect to the pixel having the microlens 302 (unit pixel 200). The light beam that has passed through the exit pupil 303 enters the unit pixel 200 with the optical axis 304 as the center.

  The pupil region 305 and the pupil region 306 are different pupil regions (partial regions) of the exit pupil. The light beam passing through the pupil region 305 is received by the sub-pixel a (FD 201a) through the micro lens 302. On the other hand, the light beam passing through the pupil region 306 is received by the sub-pixel b (FD 201b) through the micro lens 302. Thus, each of the sub-pixel a and the sub-pixel b receives light from different areas of the exit pupil of the photographing lens. Therefore, by comparing the output signal of the sub-pixel a and the output signal of the sub-pixel b, it becomes possible to detect the defocus amount by the phase difference method.

  FIG. 4 is a block diagram for explaining the processing contents of the imaging unit 105 and the image processing unit 106. The imaging unit 105 includes an A image / B image acquisition unit (hereinafter referred to as an acquisition unit) 401, a horizontal pixel addition thinning unit (hereinafter referred to as a thinning unit) 402, a aliasing component reliability generation unit (hereinafter referred to as a reliability generation unit). 403 and a reliability map extraction unit (hereinafter referred to as an extraction unit) 404. The image processing unit 106 includes an image addition processing unit 405, a defocus amount detection unit (hereinafter referred to as a detection unit) 406, and a reliability map integration unit (hereinafter referred to as an integration unit) 407. FIG. 5 is a flowchart for explaining the operations of the imaging unit 105 and the image processing unit 106. Hereinafter, description will be made with reference to FIGS. 4 and 5.

  In step S501, the acquisition unit 401 acquires an A image Bayer signal and a B image Bayer signal that are output signals of the image sensor. The A image Bayer signal is an image signal obtained from the sub-pixel a, and the B image Bayer signal is an image signal obtained from the sub-pixel b. For the following description, the number of pixels of the signal acquired by the acquisition unit 401 is referred to as a full size here. In this embodiment, the number of horizontal pixels is 6000 pixels.

  In step S502, the thinning unit 402 performs horizontal pixel addition thinning processing on each of the full-size A image Bayer signal and B image Bayer signal acquired in step S501 to generate a pixel signal. The horizontal pixel addition thinning process is an example of resampling. In this embodiment, the number of horizontal pixels of the full-size Bayer signal is 6000 pixels, but the number of horizontal pixels of the pixel signal becomes 2000 pixels by the horizontal pixel addition thinning process.

  FIG. 6 is a schematic diagram for explaining the horizontal pixel addition thinning process. An RG row 601 indicates an RG row in which R pixels are arranged, and a GB row 602 indicates a GB row in which B pixels are arranged. In the horizontal pixel addition thinning process, pixel values are added at black pixel positions, and the number of pixels after the addition thinning becomes 1/3. The number of horizontal pixels when the 1/3 addition is thinned is 6000 × 1/3 = 2000 pixels. A filter coefficient 607 is a coefficient of a three-pixel addition filter in the R pixel 603 in the RG row 601 and the G pixel 604 in the GB row 602. The filter coefficient 608 is a coefficient of a three-pixel addition filter in the G pixel 605 in the RG row 601 and the B pixel 606 in the GB row 602. For example, pixel values in the R pixel 603 are added according to the filter coefficient 607.

  In step S503, the reliability generation unit 403 generates reliability for the aliasing component as an aliasing reliability map. Specifically, each of the full-size A image Bayer signal and B image Bayer signal (output signal) is subjected to filter processing to generate an A image reliability map and a B image reliability map to generate two reliability maps. Are combined to generate a folding reliability map.

  First, the folding component will be described. The aliasing component changes according to the number of thinned pixels by addition. In the present embodiment, pixel thinning by addition is performed twice in total by the thinning unit 402 and the image addition processing unit 405. The image addition processing unit 405 generates a luminance signal by adding pixel signals in a Bayer unit of 2 × 2 pixels. By this processing, the number of pixels is reduced by 1/2 in the horizontal direction and 1/2 in the vertical direction. That is, the full-size Bayer signal acquired in step S501 is added by the thinning unit 402, and becomes a pixel signal having 2000 horizontal pixels. Further, the pixel signal having 2000 horizontal pixels is added by the image addition processing unit 405, so that the luminance signal (Y) having 1000 horizontal pixels is obtained.

FIG. 7 is a diagram for explaining the filter coefficient of the addition process. The filter coefficient 701 is a luminance addition filter coefficient. A filter coefficient 701 is a filter coefficient indicating the addition process (step S505) in the image addition processing unit 405. In the addition process for generating the luminance signal (Y), the pixels subjected to the addition thinning process in the thinning unit 402 are further added. For example, in the RG row 601 in the horizontal direction, the image addition processing unit 405 adds the R pixel 603 and the G pixel 605 after the addition thinning process.
FIG. 8 is a diagram illustrating the frequency characteristics of the filter coefficient 701. The horizontal axis is a normalized frequency normalized by the sampling frequency, and 0.5 is the Nyquist frequency. The vertical axis represents the normalized amplitude normalized by the amplitude of normalized frequency = 0. Since the number of horizontal pixels of the luminance signal for detecting the defocus amount is reduced to 1/6 with respect to the number of horizontal pixels of the full size, a signal having a normalized frequency higher than 1/6 = 0.166, that is, The gray area in FIG. 8 is the folding component. With the aliasing component, a part of the image is altered and moire is generated, thereby degrading the quality of the image.

Next, the influence of the aliasing component on the defocus amount detection will be described. FIG. 9 illustrates the case of a frequency band that does not become a folded component, and FIG. 10 illustrates the case of a frequency band of a folded component.
FIG. 9A is a diagram showing a line profile of a sine wave subject having a band with a normalized frequency of 0.08 in FIG. The band with the normalized frequency of 0.08 is in the white region in FIG. 8, that is, the region that is not the aliasing component. The horizontal axis represents horizontal coordinates, and the vertical axis represents signal values. A solid line 901 represents the output signal of the subpixel a, and a broken line 902 represents the output signal of the subpixel b. From the solid line 901 and the broken line 902, it can be seen that the subpixel a and the subpixel b are out of phase by one pixel in the horizontal direction.

  FIG. 9B is a diagram illustrating a luminance signal obtained by performing addition thinning processing on the output signal illustrated in FIG. A solid line 903 represents a luminance signal generated from the output signal of the subpixel a, and a broken line 904 represents a luminance signal generated from the output signal of the subpixel b. Since the number of pixels is reduced to 1/6 of the full size by the addition thinning process, the phase shift amount is also degenerated. That is, when a phase shift is detected in units of subpixels, the shift amount is 1/6 pixel. As shown in FIG. 9 (B), the phase shift amount of the sub-pixel can be accurately extracted even if the addition thinning process is performed in the frequency band that does not become the aliasing component. Therefore, the correct defocus amount can be detected. As a method for calculating the defocus amount from the phase shift amount of the subpixel, for example, a method disclosed in Japanese Patent Application Laid-Open No. 2008-15754 may be used.

  On the other hand, FIG. 10A is a diagram showing a line profile of a sine wave subject having a band with a normalized frequency of 0.26 in FIG. A band having a normalized frequency of 0.26 is a gray region in FIG. 8, that is, a region of aliasing components. The horizontal axis represents horizontal coordinates, and the vertical axis represents signal values. A solid line 1001 represents the output signal of the subpixel a, and a broken line 1002 represents the output signal of the subpixel b. From the solid line 1001 and the broken line 1002, it can be seen that the sub-pixel a and the sub-pixel b are out of phase by one pixel in the horizontal direction.

  FIG. 10B is a diagram illustrating a luminance signal obtained by performing addition thinning processing on the output signal illustrated in FIG. A solid line 1003 represents a luminance signal generated from the output signal of the subpixel a, and a broken line 1004 represents a luminance signal generated from the output signal of the subpixel b. As shown in FIG. 10B, the luminance signal obtained by performing the addition thinning process on the output signal in the frequency band serving as the aliasing component changes in phase due to aliasing, and the 1/6 pixel whose phase deviation is the correct deviation amount is is not.

  That is, when the normalized frequency band in the full size state is higher than 0.166, the correct defocus amount cannot be detected by the aliasing component, while when it is lower than 0.166, the correct defocus amount is detected. can do. Therefore, in order to determine whether or not the correct defocus amount can be detected after the addition thinning process, it is only necessary to detect whether the normalized frequency band in the full size state is higher or lower than 0.166.

  Therefore, the aliasing reliability map is generated based on whether the normalized frequency band in the full size state is higher or lower than 0.166. Specifically, a high-pass filter (HPF) is applied to each of the full-size A image Bayer signal and B image Bayer signal, and if the output value is lower than a preset threshold value, high reliability is obtained. If is high, a low reliability is assigned and a reliability map is generated. Note that the high reliability region corresponds to the white region in FIG. 8, and the low reliability region corresponds to the gray region in FIG. 8 showing the aliasing component. Then, the reliability map of the A image and the reliability map of the B image are integrated to generate a folding reliability map.

  FIG. 11 is a diagram illustrating the frequency characteristics of the high-pass filter. In the present embodiment, a high-pass filter having a frequency characteristic 1101 that attenuates frequencies in a high reliability region whose frequency is lower than 0.166 and passes frequencies in a low reliability region whose frequency is higher than 0.166 is used. The high-pass filter having the frequency characteristic 1101 is, for example, a high-pass filter having a filter coefficient of [−1 2 −1], and the threshold value is set to 40 for the output signals in FIGS. 9A and 10A. By doing so, the reliability can be assigned.

  A bandpass filter (BPF) may be used instead of the highpass filter to generate the aliasing reliability map. By using the bandpass filter, the scale of computation increases, but more reliable reliability can be assigned. FIG. 12 is a diagram illustrating the frequency characteristics of the bandpass filter. In the present embodiment, a band-pass filter having a frequency characteristic 1201 that cuts off a frequency in a high reliability region whose frequency is lower than 0.166 and passes a frequency in a low reliability region whose frequency is higher than 0.166 is used. By providing 50 taps of filter coefficients, it is possible to configure a band-pass filter having a frequency characteristic 1201 having a steep filter characteristic. When generating a loopback reliability map using a bandpass filter, for the output value subjected to the bandpass filter, if the output value is lower than the preset threshold value, the high reliability is obtained. If is high, assign low reliability.

  A low pass filter (LPF) may be used instead of the high pass filter to generate the aliasing reliability map. FIG. 13 is a diagram for explaining the frequency characteristics of the low-pass filter. In the present embodiment, a low-pass filter having a frequency characteristic 1301 that passes a frequency in a high reliability region whose frequency is lower than 0.166 and cuts off a frequency in a low reliability region whose frequency is higher than 0.166 is used. When generating the aliasing reliability map using a low-pass filter, the output value subjected to the low-pass filter has a high reliability for an output value higher than a preset threshold value, and an output value lower than a preset threshold value. Assign low confidence.

  In this embodiment, a high-pass filter (or a band-pass filter or a low-pass filter) is applied to each of the full-size A image Bayer signal and B image Bayer signal to generate a folding reliability map for the A image and a folding reliability map for the B image. To do. Further, a folded reliability map is output by integrating the two generated reliability maps with a logical sum. That is, in the integrated aliasing reliability map, high reliability is output only when the reliability in both the A and B images is high, and the other combinations are output as low reliability. The gradation of the reliability map in the present embodiment is a binary value of high reliability or low reliability, but is not limited to this, and the multi-level reliability is determined according to the output value of the filter. Also good. In this embodiment, the case of adding and thinning to 1/6 has been described. However, the present invention is not limited to this. What is necessary is just to change suitably the extraction band of the filter which extracts reliability according to an addition thinning number.

  In step S504, the extraction unit 404 extracts values at predetermined intervals from the aliasing reliability map generated in step S503. The predetermined interval is set according to a defocus map (or a defocus map reliability map) described in step S506. FIG. 14 is a diagram for explaining the extraction interval of the return reliability map. A pixel 1401 represents one pixel in the aliasing reliability map generated in step S503. The extraction unit 404 extracts information on the folding reliability map at intervals of the black pixel of interest 1402 in accordance with the position of the defocus amount detected by the detection unit 406 in step S506. In the present embodiment, for example, information on the folding reliability map is extracted every 120 pixels. If the number of horizontal pixels in the aliasing reliability map generated in step S503 is 6000 pixels, the number of horizontal pixels in the aliasing reliability map after extraction is 6000/120 = 50 pixels. Note that the position of the target pixel for calculating the reliability of the aliasing component in step S503 may be calculated discretely as the same coordinates as when the defocus amount is calculated, and the extraction unit 404 may be configured to do nothing.

  In step S505, the image addition processing unit 405 adds the pixel signals subjected to the horizontal pixel addition thinning process by the thinning unit 402 in step S502 to generate a luminance signal. FIG. 15 is a diagram illustrating the image addition process. As shown in FIG. 15, the image addition processing unit 405 generates a luminance signal by adding pixel signals in units of 2 × 2 pixels as a Bayer. For example, the image addition processing unit 405 adds the signals of Ra, Ga, Ga, and Ba pixels surrounded by a thick line to generate a Ya luminance signal. By this processing, the number of pixels is reduced by 1/2 in the horizontal direction and 1/2 in the vertical direction. That is, the full-size Bayer signal acquired in step S501 is added by the thinning unit 402 (step S502), and further added by the image addition processing unit 405 (step S505), thereby obtaining a horizontal 1000 pixel luminance signal. Become. In this embodiment, the example in which the addition thinning process is performed in step S502 and step S505 has been described. However, the present invention is not limited to this, and pixels may be reduced by resampling by an interpolation process.

  In step S506, the detection unit 406 generates a defocus map and a reliability map of the defocus map based on the luminance signal of the A image and the luminance signal of the B image generated in step S505. Then, the detection unit 406 outputs the generated defocus map to the RAM 103. The defocus map is distribution information corresponding to the distance. The position for calculating the defocus amount and its reliability is discrete, and in this embodiment, the defocus amount is detected every 20 pixels. Therefore, the number of horizontal pixels of the generated defocus map and the reliability map of the defocus map is 1000/20 = 50 pixels.

  For example, the detection unit 406 calculates a defocus amount from the correlation amount of pixel signals using a method disclosed in Japanese Patent Application Laid-Open No. 2008-15754, and generates a defocus map. The detection unit 406 evaluates the contrast between the A image and the B image, and assigns a low reliability to the defocus amount when the contrast is low, and assigns a high reliability to the defocus amount when the contrast is high. Generate a confidence map. Note that the reliability map generation method is not limited to this. For example, it is determined whether the minimum value when the correlation calculation between the A image and the B image is performed is greater than or equal to a predetermined value. A reliability may be assigned, and if the reliability is less than a predetermined value, a high reliability may be assigned.

  In step S507, the integration unit 407 integrates the folding reliability map generated in step S504 and the defocus map reliability map generated in step S506 by logical sum. The logical sum operation is the same as in step S503. When the reliability is multivalued, for example, integration is performed using a method such as a weighted average. The integration unit 407 outputs the integrated reliability map to the RAM 103.

  The defocus map and the integrated reliability map output to the RAM 103 are used for image processing such as autofocus and background blur addition for a captured image. If the pixel of interest is low-reliability, refer to the defocus amount of surrounding high-reliability pixels to avoid using the defocus amount that is erroneously detected by addition decimation. High-precision image processing can be performed. In the present embodiment, the case where the reliability map of the aliasing component is generated as the reliability map corresponding to the defocus map generated with reference to the image signal after the addition thinning is described, but the present invention is not limited thereto. is not. The reliability map of the aliasing component may be applied to evaluate whether the motion vector is accurate or to determine whether the quality of the image quality is deteriorated due to “moire”.

  As described above, according to the present embodiment, the imaging unit can calculate whether or not the aliasing component is included after the addition thinning from the full-size signal before the addition thinning as the reliability for the aliasing component. For this reason, it is possible to detect with high accuracy a region where erroneous detection occurs due to the influence of the aliasing component due to the addition decimation with respect to the defocus amount detected from the image signal after the addition decimation.

(Second Embodiment)
The imaging unit 105 according to the first embodiment includes a thinning unit 402, a reliability generation unit 403, and an extraction unit 404, and calculates the reliability of pixel addition thinning and aliasing components. In the present embodiment, a case where these processes are performed by the image processing unit 106 will be described. Note that the configuration of the imaging apparatus 100 in the present embodiment is the same as each configuration described using FIG. 1 in the first embodiment. In addition, the pixel arrangement of the imaging unit 105 is the same as the pixel arrangement described with reference to FIG. 2 in the first embodiment. The image processing unit 106 may be realized as an image processing device.

FIG. 16 is a block diagram for explaining processing contents of the imaging unit 105 and the image processing unit 106. In this embodiment, in addition to the generation of the defocus map and the reliability map, the image signal AB Bayer image is also output to the RAM 103 for 4K recording.
The imaging unit 105 includes an acquisition unit 401. In the acquisition unit 401, the imaging unit 105 acquires a full-size A image Bayer signal and a B image Bayer signal having a horizontal pixel count of 6000 pixels. In the first embodiment, the imaging unit 105 has a configuration that can perform pixel addition thinning or filter processing to calculate reliability, but in the present embodiment, the number of these is reduced and the configuration of the imaging unit is reduced. It is simplified.

  The image processing unit 106 includes a thinning unit 402, a reliability generation unit 403, an extraction unit 404, an image addition processing unit 405, a detection unit 406, an integration unit 407, an AB image generation unit 1601, and an adaptive interpolation reduction unit 1602. The operations from the horizontal pixel addition thinning unit 402 to the integration unit 407 in the image processing unit 106 are the same as those in the first embodiment. Also in this case, when detecting the defocus amount, it is possible to detect a region where erroneous detection occurs due to the influence of the aliasing component due to addition thinning as low reliability. Therefore, when applied by determining a subject region to be tracked according to the defocus amount, it is possible to suppress erroneous detection of the defocus amount and to determine the region of the tracking subject with higher accuracy.

  Next, the 4K recording operation of the image processing unit 106 will be described. The AB image generation unit 1601 adds the A image Bayer signal and the B image Bayer signal to generate an AB image Bayer signal. The number of horizontal pixels of the generated AB image Bayer signal is 6000 pixels. The adaptive interpolation reduction unit 1602 reduces the AB image Bayer signal and generates a 4K size AB image Bayer signal. The adaptive interpolation reduction unit 1602 uses a method for obtaining a reduced image with better quality, instead of reduction by simple pixel thinning like the horizontal pixel addition thinning unit 402. For example, as disclosed in JP-A-2006-103797, the direction in which the signal correlation is high is determined, and the method of interpolating the signal of the pixel of interest is changed based on the determination result. A method that can obtain a good reduced image is used. In the present embodiment, the adaptive interpolation reduction unit 1602 reduces the AB image Bayer signal having a horizontal pixel number of 6000 pixels by interpolation processing, and generates a 4K size AB image Bayer signal having a horizontal pixel number of 4000 pixels.

  In the present embodiment, the case has been described in which the image processing unit performs from addition thinning-out processing to generation of the reliability map of the aliasing component as the reliability map corresponding to the defocus map. However, the present invention is not limited to this. Absent. For example, as disclosed in Japanese Patent Application Laid-Open No. 2006-58963, a digital front end unit may be provided, and the digital front end unit may generate a reliability map by addition thinning or filtering.

  As described above, according to the present embodiment, the reliability of the aliasing component can be calculated in the image processing unit. For this reason, when the image processing unit detects the defocus amount of the phase difference method from the image signal after the thinning-out addition, it is possible to determine a region where erroneous detection occurs due to the thinning-out thinning as low reliability.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

DESCRIPTION OF SYMBOLS 100 Imaging device 101 System control part 105 Imaging part 106 Image processing part

Claims (11)

Extraction means for extracting a predetermined frequency band from a signal output from the imaging means;
An image processing apparatus comprising: a first reliability generation unit that assigns a reliability to the pixel of interest according to an output of the extraction unit. Sampling means for re-sampling the signal output from the imaging means;
Defocusing means for calculating distribution information corresponding to a distance based on a signal output from the sampling means;
Second reliability generation means for calculating the reliability of the distribution information;
The image processing according to claim 1, further comprising: an integration unit that integrates the reliability output by the first reliability generation unit and the reliability output by the second reliability generation unit. apparatus.   The image processing apparatus according to claim 2, wherein the re-sampling is addition thinning-out processing or interpolation processing.   The image according to any one of claims 1 to 3, wherein the first reliability generation unit assigns a low reliability when the frequency of the signal output from the imaging unit is higher than a threshold value. Processing equipment. The extraction means is a high pass filter or a band pass filter,
The first reliability generation unit compares the output of the extraction unit with a threshold value, and assigns a low reliability level when the output is larger than the threshold value. The image processing apparatus according to item. The extraction means is a low-pass filter;
The first reliability generation means compares the output of the extraction means with a threshold value, and assigns a low reliability when the output is smaller than the threshold value. The image processing apparatus according to item. An imaging means having a photoelectric conversion unit;
An image processing apparatus comprising: the image processing apparatus according to claim 1.   8. The imaging device according to claim 7, wherein the imaging unit includes a plurality of pixels corresponding to each of the plurality of microlenses, and each pixel includes a first photoelectric conversion unit and a second photoelectric conversion unit. Imaging device.   The imaging apparatus according to claim 7 or 8, wherein the imaging unit adds and outputs signals output from the photoelectric conversion unit. A control method for an image processing apparatus, comprising:
An extraction step of extracting a predetermined frequency band from the signal output from the imaging means;
And a first reliability generation step for assigning reliability to the pixel of interest according to the output of the extraction step. A sampling step of re-sampling the signal output from the imaging means;
A defocusing step of calculating distribution information corresponding to the distance based on the signal output in the sampling step;
A second reliability generation step of calculating the reliability of the distribution information;
The control method according to claim 10, further comprising: an integration step of integrating the reliability output in the first reliability generation step and the reliability output in the second reliability generation step. .

Images