JP2017188837A - Image processing apparatus, control method thereof, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing technology that can perform appropriate flicker correction on an image even when a driving method of an imaging device changes within a frame.SOLUTION: An image processing device 100 comprises flicker detection means 105 that detects a flicker component of an image read from an imaging device 102 in which a driving method in acquiring the image changes in a frame, correction value calculation means 106 that generates a flicker correction value based on the flicker component detected by the flicker detection means 105, and pixel-based information read from an imaging device 102 according to a change of the driving method, and flicker correction means 108 that removes the flicker component of the image read from the imaging device 102 using the flicker correction value.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus including an imaging apparatus such as a digital camera, and more particularly to an image processing technique for performing an appropriate flicker correction on an image when a driving method of an imaging element changes within a frame.

  In imaging devices such as digital cameras, multi-functionalization using image sensors such as CMOS sensors has progressed, and technologies that can detect the focus using the pupil division method and periodic signal level fluctuation (hereinafter referred to as flicker) components in images are corrected. There is technology to do.

  For example, in a pixel row used for focus detection processing of an image sensor, a signal is read out independently for each photodiode (hereinafter referred to as PD) of each pixel, and in a pixel row not used for focus detection processing, a PD signal is stored in the pixel. A technique has been proposed in which signals for image generation are read out by adding. In this proposal, an increase in signal readout time from the image sensor can be suppressed (Patent Document 1).

  In addition, a technique for predicting a correction value of the next frame from the current frame and correcting the flicker by preliminarily storing the storage time of the image sensor and the degree of modulation of the luminance signal by the blinking light source has been proposed ( Patent Document 2).

JP 2006-021052 A JP 2013-17238 A

  However, in the conventional technique disclosed in Patent Document 1, when pixel signals of a pixel row used for focus detection and a pixel row that is not so are read by a rolling shutter, the time required for reading differs from each other. For this reason, the flicker cycle changes within the frame depending on whether or not the focus detection pixel signal is read out, and thus cannot be applied to the conventional technique disclosed in Patent Document 2.

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing technique that enables appropriate flicker correction on an image even when the driving method of an image sensor changes within a frame.

  In order to achieve the above object, an image processing apparatus according to the present invention includes a flicker detection unit that detects a flicker component of an image read from an image sensor whose driving method for acquiring an image changes in a frame, and the flicker Correction value calculation means for generating a flicker correction value based on the flicker component detected by the detection means and information based on the pixels read from the image sensor in accordance with a change in the driving method; and the flicker correction value And flicker correction means for removing a flicker component of an image read from the image sensor using the image sensor.

  According to the present invention, it is possible to appropriately perform flicker correction on an image even when the driving method of the image sensor changes within a frame.

1 is a control block diagram illustrating a system configuration of a digital camera which is a first embodiment of an image processing apparatus of the present invention. It is a circuit block diagram of the unit pixel of an image sensor. (A) is a graph showing the luminance variation in the time axis direction of the line of interest, and (b) is a graph showing the relationship between the luminance variation due to the blinking light source and the phase with respect to the image sensor. It is a figure which shows typically the relationship between the image imaged with the image pick-up element, and flicker. It is a figure explaining the flicker component which arises when the drive system of an image sensor changes within a frame. It is a block diagram which shows the specific structure of an extended flicker correction value calculation part. It is a timing chart explaining operation | movement of an extended flicker correction value calculation part. It is a figure explaining the relationship between the signal read from an image sensor, and a flicker correction value signal. It is a flowchart explaining the operation | movement of the flicker correction of a camera based on control of a system control part. In the digital camera which is 2nd Embodiment of the image processing apparatus of this invention, it is a graph explaining the case where the ratio of the reading time for every line of an image signal and a focus detection signal is 1: 2. It is a block diagram explaining the specific structure of an extended flicker correction value calculation part. It is a graph explaining the relationship between the signal read from an image sensor, and a flicker correction value signal.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 is a control block diagram showing a system configuration of a digital camera which is a first embodiment of an image processing apparatus of the present invention. In the present embodiment, a digital camera that is an example of an imaging apparatus is illustrated as an image processing apparatus of the present invention, but the present invention is not limited to this.

  As shown in FIG. 1, a digital camera 100 (hereinafter referred to as a camera 100) of the present embodiment includes an imaging optical system 101, an imaging element 102, an A / D conversion unit 103, a capture unit 104, a flicker detection unit 105, and An extended flicker correction value calculation unit 106 is provided. The camera 100 also includes a flicker correction unit 108, a digital signal processing unit 109, a system control unit 110, an AF evaluation value calculation unit 111, and a memory 112.

  The image sensor 102 photoelectrically converts the subject image formed through the imaging optical system 101 including the focus lens and outputs the subject image to the A / D converter 103. The imaging element 102 is provided with a microlens array (not shown) as pupil dividing means on the front side (subject side), has a two-dimensional array structure of unit pixel cells corresponding to each microlens, and the imaging optical system 101. The amount of exposure is controlled by a shutter that constitutes.

  Then, the image sensor 102 photoelectrically converts the image of the formed subject image, captures it, and sequentially converts the charges accumulated in the PD constituting the unit pixel to a voltage at the time of readout control, thereby converting the A / D converter 103. Output to. The single pixel cell is a pixel arrayed in the row direction and divided into two pupils (hereinafter referred to as a divided pixel (first photoelectric conversion unit) a and a divided pixel (second photoelectric conversion unit) b). Have. Focus detection is performed based on the phase difference between the divided pixels a and b, and a captured image is generated by adding the divided pixels a and b. Note that the transfer and reading of the charges accumulated in the image sensor 102 are executed for each line.

  The A / D conversion unit 103 converts an analog signal output from the image sensor 102 via an analog signal processing unit that amplifies ISO sensitivity or a CDS circuit (not shown) into a digital signal and outputs the digital signal to the capture unit 104.

  The capture unit 104 determines the effective scanning period and type of the pixel data output from the A / D conversion unit 103, and uses the data obtained by adding the divided pixels a and b as an image signal to the flicker detection unit 105 and the flicker correction unit 108. Output. Further, the capture unit 104 outputs the data of the divided pixels a and b as a focus detection signal and outputs an image signal to the AF evaluation value calculation unit 111. Further, the capture unit 104 outputs a control signal for the effective scanning period of the image signal and the focus detection signal to the system control unit 110.

  The flicker detection unit 105 detects the flicker component (frequency amplitude / phase) of the image signal output from the capture unit 104 and outputs the detected flicker component to the extended flicker correction value calculation unit 106. The flicker component may be detected as a flicker peak and bottom.

  The extended flicker correction value calculation unit 106 calculates the vertical count number based on the flicker component from the flicker detection unit 105 and the effective scanning period of the image signal and focus detection signal output from the system control unit 110, and the flicker correction value. Is generated. Details of the flicker correction value calculation method will be described later.

  The flicker correction unit 108 is flicker correction means for performing flicker correction by outputting an image signal in which the influence of flicker is reduced. Specifically, the flicker correction unit 108 performs digital signal processing on the image signal from which the flicker component has been removed by subtraction processing between the image signal output from the capture unit 104 and the flicker correction value generated by the extended flicker correction value calculation unit 106. Output to the unit 109.

  The digital signal processing unit 109 performs digital signal processing typified by synchronization processing, gamma processing, and noise reduction processing on the image signal output in a Bayer array from the flicker correction unit 108, and passes through the system control unit 110. Output to the memory 112. Since digital signal processing such as synchronization processing, gamma processing, and noise reduction is well known, detailed description thereof is omitted.

  The system control unit 110 includes a CPU, an MPU, and the like, and controls the entire camera 100.

  The AF evaluation value calculation unit 111 calculates an autofocus evaluation value component based on the focus detection signal and the image signal output from the capture unit 104, and outputs the calculation result to the memory 112. Specifically, the AF evaluation value calculation unit 111 calculates the evaluation value by the phase difference detection method based on the focus detection signal and the other focus detection signal generated by subtracting the image signal and the focus detection signal. Then, the calculated evaluation value is output to the memory 112. In addition, since the calculation method of an evaluation value is well-known, the detailed description is abbreviate | omitted.

  The memory 112 holds the captured image output from the digital signal processing unit 109 and the evaluation value output from the AF evaluation value calculation unit 111. The memory 112 can also be used as a work area for the system control unit 110. The non-volatile memory 113 is configured by a flash ROM or the like, and program codes executed by the system control unit 110 and various data are written therein.

  Next, a method for reading the output signal of the image sensor 102 will be described with reference to FIG. Here, a part that greatly affects the time required for reading the output signal of the image sensor 102 will be described.

  As shown in FIG. 2, the optical signals incident on the PDs 201a and 201b of the divided pixels a and b described above are photoelectrically converted, and charges corresponding to the exposure amount are accumulated. When the signals txa and txb applied to the gates of the transfer gates 202a and 202b are set to high level, the charges accumulated in the PDs 201a and 201b are transferred to the FD (floating diffusion) unit 203.

  The FD unit 203 is connected to the gate of the FD amplifier 204, and the charge amount transferred from the PD 201 a and the PD 201 b is converted into a voltage amount by the FD amplifier 204. By setting the signal sel applied to the gate of the pixel selection switch 206 to a high level, the pixel signal converted into a voltage by the FD amplifier 204 is output to the output unit vout of the unit pixel.

  Before the pixel signal is read, the signal res applied to the gate of the FD reset switch 205 is set to a high level in order to reset the FD unit 203. Further, when charge accumulation is started, the signal res and the signals txa and txb are simultaneously set to the high level in order to reset the charges of the PDs 201a and 201b. As a result, both the transfer gates 202a and 202b and the FD reset switch 205 are turned on, and the PD 201a and PD 201b are reset via the FD unit 203.

  When reading the region to be imaged, the signals txa and txb are simultaneously set to the high level so that the charges accumulated in the PD 201a and the PD 201b are combined and transferred to the FD unit 203, and the combined signal of the divided pixels a and b Is output.

  On the other hand, when reading a region other than an image target, for example, a region designated as a focus detection target (a focus detection target region), first, the signal txa is set to a high level and the signal txb is set to a low level. Thereby, only the electric charge accumulated in the PD 201a is transferred to the FD unit 203, and the pixel signal of the divided pixel a is output. Next, the signal txb is also set to a high level, whereby the charge accumulated in the PD 201b is synthesized with the charge accumulated in the PD 201a in the FD unit 203, and a synthesized signal of the divided pixels a and b is output.

  Since the focus detection target area sequentially outputs the pixel signal of the divided pixel a and the combined pixel signal of the divided pixels a and b in units of lines, the time required to read out the signal is twice that of the image target area. Become.

  Here, the case where the readout time of the focus detection target region is twice as long as that of the image target region is illustrated, but it is not necessarily double. For example, the increase in the readout time can be suppressed by reducing the readout amount from the image sensor 102, for example, by adding and outputting the pixel signal of the divided pixel a with the surrounding pixels. Further, by narrowing the focus detection target region in the horizontal direction, it is possible to suppress a difference in readout time between a line including the focus detection target region and a line not including the focus detection target region.

  In addition, here, the image pickup device 102 including two PDs 201a and 201b for one pixel is illustrated, but the same applies to the case where reading from the image pickup device is performed by a method in which the time required to read out the signal for each region is different. Therefore, the present invention can be applied. For example, the present invention can also be applied to a case where pixel signals are added to the surroundings for each region to switch whether to read, to change the number of pixels to be added, or to read by thinning instead of addition.

  Also, when changing the number of bits of pixel data for each area, or when reading only a part of the area multiple times for an image sensor capable of non-destructive readout that can read out signals with the PD held in charge. Also, the present invention can be applied.

  Next, a flicker detection method will be described with reference to FIG. FIG. 3A is a graph illustrating the luminance variation in the time axis direction of the line of interest 501. In FIG. 3A, the vertical axis represents the signal level of the luminance of the image, and the horizontal axis represents the time for each frame. In addition, changes in luminance of the target line 501, the target line 504, and the target line 505 for each of the times α, α + T, and α + 2T are indicated as luminance fluctuations 502.

  FIG. 3B is a graph showing the relationship between the luminance variation and phase due to the blinking light source (flicker light source) for the image sensor 102. FIG. 3B shows the flicker component 503 in the frame of the image. Further, the above-described luminance fluctuation 502 shows the flicker component 503 as a waveform sampled at the time interval T of the vertical synchronization signal.

  One cycle of the luminance fluctuation 502 can be obtained by sampling the luminance change of the target line in a plurality of frames. Thus, the amplitude of the flicker component 503 can be obtained based on the amount of change from one period of the luminance fluctuation 502 to the maximum luminance value. For example, when the phase of the line of interest 504 is obtained, it can be calculated from the relationship between the amplitude of the flicker component 503 and the signal level of the luminance of the line of interest 504. The phase of the target line 505 can be calculated in the same manner as the target line 504.

  The flicker detection method is not limited to this, and when the driving method of the image sensor 102 changes within the frame, a flicker detection frame is provided in an area where the driving method does not change within the frame, and the flicker component is detected within the detection frame. May be detected. For example, the flicker component may be detected by sampling a part of one or more periods of the image signal in the vertical direction.

  Next, a flicker correction method when only addition reading of the image sensor 102 in which the divided pixels a and b are arranged will be described with reference to FIG. FIG. 4 is a diagram schematically illustrating a relationship between an image captured by the image sensor 102 and flicker.

  FIG. 4A shows a region (hereinafter referred to as an image target region) 305 corresponding to an image to be output by only performing addition reading of the divided pixels a and b in the single pixel cell of the image sensor 102. . In FIG. 4A, the horizontal direction (horizontal direction) indicates the horizontal size of the image, and the vertical direction (vertical direction) indicates the vertical readout line of the image signal and the focus detection signal of the image sensor 102. . Further, in the enlarged image 306 obtained by enlarging a predetermined area of the image target area 305, the pixel value of the divided pixel a is A, and the pixel value of the divided pixel b is B. In this image target region 305, a flicker component 303 is generated according to the relationship between the readout cycle of the image sensor 102 and the cycle of the point light source.

  FIG. 4B shows a case where the image signal 301 is read from the image sensor 102 with a predetermined accumulation time. In FIG. 4B, flicker generated by modulating the signal level by the blinking light source appears like an image signal 301 in the form of light and dark in the vertical direction.

  In FIG. 4C, a flicker component 303 whose level varies periodically in the image signal 301 appears depending on the light emission period of the blinking light source. In FIG. 4C, the vertical direction corresponds to the number of vertical lines, and the horizontal direction corresponds to the signal level of the flicker component of the image signal.

  FIG. 4D shows a flicker correction value 302 for removing the flicker component 303 of the image signal 301. In FIG. 4D, the vertical axis corresponds to the number of vertical lines of the image signal 301, and the horizontal axis corresponds to the signal level of the flicker correction value 302. By subtracting the flicker correction value 302 having a uniform period from the flicker component 303, the flicker correction is performed in the image target area 305.

  Here, an example of calculating the flicker correction value will be described. The flicker correction value Yn can be obtained using the following formula (1) which is a well-known formula.

  In the above formula (1), N is a sampling parameter, Yn (n = 0, 1,... N−1) is sampled data, and θk is a phase difference of flicker components.

  In this embodiment, the extended flicker correction value calculation unit 106 stores the accumulation time of the image sensor 102, the modulation degree of the luminance signal by the blinking light source, and the frequency characteristics of flicker measured in advance as a table. The unit 110 may learn and store the information.

  Next, a flicker component generated when the driving method of the image sensor 102 changes within a frame will be described with reference to FIG.

  In FIG. 5A, the horizontal direction indicates the horizontal size of the image, and the vertical direction indicates the vertical readout line of the image signal of the image sensor 102 and the focus detection signal. In FIG. 5A, in the single pixel cell of the image sensor 102, the image target region 601 performs only addition reading of the divided pixels a and b, and the focus detection target region 602 alternately in units of vertical lines. Addition readout of the divided pixels a and b and readout of the divided pixel a are performed.

  Specifically, in the area 601 to be imaged, as shown in the enlarged image 607, addition reading of the divided pixels a and b is performed for 1, 2,. On the other hand, in the focus detection target area 602, as shown in the enlarged image 608, readout of the divided pixels a and addition readout of the divided pixels a and b are alternately performed on m, m + 1,.

  FIG. 5A shows a flicker component 603 generated according to the relationship between the readout cycle of the image sensor 102 and the cycle of the blinking light source. From FIG. 5A, it can be seen that a flicker component 603 having a constant frequency appears in the signal regardless of reading of the image signal and the focus detection signal.

  In FIG. 5B, only an image signal obtained by extracting the A + B line of the image target area 601 and the focus detection target area 602 is shown as an image 605. FIG. 5B shows a vertical line of the image signal in the focus detection target region 602 and a corresponding image 609 between the image signal and the readout line. In the focus detection target region 602, the initial value of the line count increment value is +1 with respect to the number of read lines with respect to the number of vertical lines of the image signal, and thereafter, the line count increment value is doubled.

  FIG. 5C shows a flicker component 604 whose level periodically varies in the image signal 605 depending on the light emission period of the blinking light source. The vertical axis corresponds to the number of vertical lines of the image signal 605, and the horizontal axis corresponds to the flicker signal level of the image signal. The flicker component 604 has a period of the flicker component 604 doubled in the focus detection target region 602 due to the effect that the sampling period is doubled with respect to the readout signal of the image sensor 102.

  FIG. 5D shows a flicker correction value 606 for removing the flicker component 604 of the image signal 605. In FIG. 5D, the vertical axis corresponds to the number of vertical lines of the image signal 605, and the horizontal axis corresponds to the signal level of the flicker correction value 606. As described above, when the driving method of the image sensor 102 does not change within the frame, the flicker correction value 606 having a uniform period can be used.

  By the way, even when the driving method of the image sensor 102 changes within a frame, the image target region 601 can be corrected with the flicker correction value 606. However, since the period of the flicker component 604 is doubled in the focus detection target area 602, flicker correction value 606 cannot correct flicker correctly. That is, even if the flicker correction value 606 is subtracted from the flicker component 604, the influence of flicker cannot be reduced effectively.

  Therefore, in this embodiment, the extended flicker correction value calculation unit 106 can appropriately correct the focus detection target region 602 even when the driving method of the image sensor 102 changes in the frame.

  FIG. 6 is a block diagram showing a specific configuration of the extended flicker correction value calculation unit 106. As shown in FIG. 6, the extended flicker correction value calculation unit 106 includes an image signal line counter 701, an adder 702, a flicker correction table 703, and a focus detection signal line counter 704.

  The image signal line counter 701 counts the number of vertical lines of the image signal based on the image control signal from the system control unit 110, and outputs the image line count signal 709 to the adder 702.

  The focus detection signal line counter 704 counts the number of vertical lines of the focus detection signal based on the focus detection control signal from the system control unit 110, and outputs a focus detection line count signal 710 to the adder 702.

  The adder 702 adds the count signal 709 for the image line and the count signal 710 for the focus detection line, and outputs the total line count signal 711 to the flicker correction table 703.

  The flicker correction table 703 generates a flicker correction value based on the amplitude and phase information output from the flicker detection unit 105 and the total line count signal 711, and outputs the flicker correction value signal 712 to the flicker correction unit 108.

  Here, the operation of the extended flicker correction value calculation unit 106 will be described with reference to FIGS. 7 and 8. FIG. 7 is a timing chart for explaining the operation of the extended flicker correction value calculation unit 106.

  The vertical direction in FIG. 7 indicates a vertical synchronization signal, an input signal of the capture unit 104, an output image signal of the capture unit 104, an output focus detection signal of the capture unit 104, an image line count signal 709, a focus detection line count signal 710, and a total line. A count signal 711 is shown. 7 indicates the flicker correction value signal 712, the input signal of the flicker correction unit 108, and the flicker correction image signal 713.

  The horizontal direction in FIG. 7 indicates time, the signal AB indicates the added readout signal of the divided pixels a and b, the signal A indicates the readout signal of the divided pixel a, and the signal X indicates the signal in an indefinite state. m is a start line in the focus detection target region. In this embodiment, 20 lines of focus detection signals are read out in the focus detection target region. Y indicates a flicker correction value signal 712 that is an output of the flicker correction table 703 for the vertical line.

  In the focus detection target region, the flicker correction value is associated with the signal AB of the output image signal of the capture unit 104 for each line. The flicker correction unit 108 subtracts the flicker correction value signal 712 from the output image signal of the capture unit 104 and outputs a flicker correction image signal 713.

  FIG. 8 is a diagram for explaining the relationship between the signal read from the image sensor 102 and the flicker correction value signal 712.

  In FIG. 8, the vertical axis represents the number of lines of the signal read from the image sensor 102, and the horizontal axis represents the signal level of the flicker correction value Y discretized from the start line m to m + 39. Further, the output image signal of the capture unit 104 and the flicker correction value signal 712 are made to correspond to the total line count signal 711. 8 and 7 that the flicker correction value Y for the output image signal of the capture unit 104 is generated for each line.

  FIG. 9 is a flowchart for explaining the flicker correction operation of the camera 100 based on the control of the system control unit 110. Each process shown in FIG. 9 is executed by the CPU or the like of the system control unit 110 after the program stored in the storage unit such as the nonvolatile memory 113 is expanded in the memory 112.

  In FIG. 9, in step S1001, the system control unit 110 performs flicker detection processing by the flicker detection unit 105 based on the image signal output from the capture unit 104, and proceeds to step S1002.

  In step S1002, the system control unit 110 refers to the counter number of the focus detection signal line counter 704 (see FIG. 6) of the extended flicker correction value calculation unit 106, and determines whether the line is a focus detection signal line. If the line is not the focus detection signal, the system control unit 110 proceeds to step S1004. If the line is the focus detection signal, the system control unit 110 proceeds to step S1003.

  In step S1003, the system control unit 110 increments the counter value of the focus detection signal line counter 704, and proceeds to step S1004.

  In step S1004, the system control unit 110 refers to the counter number of the image signal line counter 701 (see FIG. 6) of the extended flicker correction value calculation unit 106 to determine whether the line is an image signal line. If it is not an image signal line, the system control unit 110 proceeds to step S1006. If it is an image signal line, the system control unit 110 proceeds to step S1005.

  In step S1005, the system control unit 110 increments the counter value of the image signal line counter 701, and proceeds to step S1006.

  In step S1006, the system control unit 110 obtains the total line count by adding the counter value of the image signal and the counter value of the focus detection signal by the adder 702, and proceeds to step S1007.

  In step S1007, the system control unit 110 generates a flicker correction value from the flicker correction table 703 based on the amplitude and phase information output from the flicker detection unit 105 and the total line count obtained in step S1006, and the process proceeds to step S1008. .

  In step S1008, the system control unit 110 corrects the flicker component of the captured image by subtracting the flicker correction value generated in step S1007 from the image signal acquired from the flicker correction unit 108, and the process proceeds to step S1009.

  In step S1009, the system control unit 110 determines whether or not the flicker correction process is completed. If not completed, the system control unit 110 returns to step S1001, and if completed, ends the process.

  As described above, in the present embodiment, flicker correction can be appropriately performed on a captured image even when the driving method of the image sensor 102 changes within a frame.

  Further, in this embodiment, the extended flicker correction value calculation unit 106 shown in FIG. 6 is configured only by adding the focus detection signal line counter 704 and the adder 702 to the existing flicker correction separation calculation unit. Therefore, even when the driving method of the image sensor 102 changes within a frame, flicker correction can be appropriately performed on a captured image with a simple circuit configuration.

(Second Embodiment)
Next, a digital camera that is a second embodiment of the image processing apparatus of the present invention will be described with reference to FIGS. In the present embodiment, with respect to the parts that overlap or correspond to those in the first embodiment, only the parts that are different will be described using the drawings and the reference numerals.

  In the present embodiment, it is possible to cope with a case where the ratio of the readout time for each line of the image signal and the focus detection signal is different.

  First, the case where the ratio of the readout time for each line of the image signal and the focus detection signal is 1: 2 will be described with reference to FIG.

  In FIG. 10, the horizontal direction (lateral direction) indicates the horizontal size of the image, and the vertical direction indicates the vertical readout line of the image signal and the focus detection signal from the image sensor 102.

  In the image target area 401, as shown in the enlarged image 402, the divided pixels “a” and “b” are added and read out by 1, 2... Lines. On the other hand, in the focus detection target area 403, as shown in the enlarged image 404, readout of the divided pixels a and b and addition readout of the divided pixels a and b are performed on m, m + 1.

  FIG. 11 is a block diagram illustrating a specific configuration of the extended flicker correction value calculation unit 106A.

  In the present embodiment, the extended flicker correction value calculation unit 106A includes a gain coefficient unit 1101 and a multiplier 1102 added to the extended flicker correction value calculation unit 106 of the first embodiment (see FIG. 6). Is different.

  The gain coefficient unit 1101 is set by the system control unit 110 according to the ratio of the readout time of the image signal and the focus detection signal, and outputs the gain value to the multiplier 1102. Here, the value set by the system control unit 110 is “2” from the readout of the divided pixels a and b.

  The multiplier 1102 multiplies the focus detection line count signal 710 and the gain value from the gain coefficient unit 1101, and outputs the focus detection signal gain line signal 1103 to the adder 702.

  The adder 702 adds the count signal 709 of the image signal line and the gain line signal 1103 of the focus detection signal, and outputs the sum as a total line count signal 711 to the flicker correction table 703.

  Then, the system control unit 110 generates a flicker correction value based on the total line count signal 711 in the flicker correction table 703 and outputs the flicker correction value signal 712 to the flicker correction unit 108 as in the first embodiment. To do.

  FIG. 12 is a graph illustrating the relationship between the signal read from the image sensor 102 and the flicker correction value signal 712.

  In FIG. 12, the vertical axis represents the number of lines of the signal read from the image sensor 102, and the horizontal axis represents the signal level of the flicker correction value Y discretized from the start line m to m + 39. Further, the output image signal of the capture unit 104 and the flicker correction value signal 712 are made to correspond to the total line count signal 711. From FIG. 12, it can be seen that the flicker correction value Y for the output image signal of the capture unit 104 is generated every two lines.

  As described above, in this embodiment, even when the driving method of the image sensor 102 changes within a frame and the ratio of the readout time of the image signal and the focus detection signal is different, the captured image is appropriately selected. Flicker correction can be performed. Other configurations and operational effects are the same as those in the first embodiment.

  In addition, this invention is not limited to what was illustrated by said each embodiment, In the range which does not deviate from the summary of this invention, it can change suitably.

  For example, in each of the above embodiments, the flicker correction in the case where the driving method of the image sensor 102 changes within the frame has been described, but the same flicker correction can be performed even when the driving method does not change.

  In each of the above-described embodiments, the case where there is only one region where the driving method of the image sensor 102 changes within the frame has been described, but flicker correction can be similarly performed even when there are a plurality of regions.

  Furthermore, one or more of the functional blocks shown in FIG. 1 may be realized by hardware such as an ASIC or a programmable logic array (PLA), or may be realized by executing software by a programmable processor such as a CPU or MPU. May be. Further, it may be realized by a combination of software and hardware. Therefore, in the above description, even when different functional blocks are described as the operation subject, the same hardware can be realized as the subject.

  Furthermore, the present invention is also realized by executing the following processing. That is, the present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read the program. It can also be realized by processing to be executed.

DESCRIPTION OF SYMBOLS 100 Camera 101 Image pick-up optical system 102 Image pick-up element 103 A / D conversion part 104 Capture part 105 Flicker detection part 106 Extended flicker correction value calculation part 108 Flicker correction part 109 Digital signal processing part 110 System control part 111 AF evaluation value calculation part 112 Memory

Claims (9)

Flicker detection means for detecting a flicker component of an image read from an image sensor whose driving method for acquiring an image changes within a frame;
Correction value calculation means for generating a flicker correction value based on the flicker component detected by the flicker detection means and information based on pixels read out from the image sensor in accordance with a change in the driving method;
An image processing apparatus comprising: flicker correction means for removing a flicker component of an image read from the image sensor using the flicker correction value.   The information based on the pixels read from the image sensor in accordance with the change in the driving method includes signals output from the image target area and the non-image target area of the image sensor. The image processing apparatus according to claim 1. A drive unit that reads out signals from the imaging element in which pixels including a first photoelectric conversion unit and a second photoelectric conversion unit are arranged in a row direction;
The region to be imaged is a region for reading out only the signal of the first photoelectric conversion unit and the signal of the second photoelectric conversion unit,
3. The non-image area is an area for reading out the signal of the first photoelectric conversion unit together with the signal of the first photoelectric conversion unit and the signal of the second photoelectric conversion unit. An image processing apparatus according to 1.   The image processing apparatus according to claim 3, wherein the driving unit reads out charges accumulated in each pixel in units of lines of the image sensor.   Information based on the pixels read from the image sensor in response to the change in the driving method is counted based on pixel signals output from the image target area and the non-image target area of the image sensor. 5. The image processing apparatus according to claim 2, wherein the number of lines is a sum of the number of lines in the image target area and the number of lines in the non-image target area.   The correction value calculating unit is configured to set a coefficient according to a ratio of a readout time of the pixel signal output from the image target region of the image sensor and the pixel signal output from the non-image target region. 6. The image processing apparatus according to claim 5, further comprising: means.   The image processing apparatus according to claim 2, wherein the non-image target area is a focus detection target area. A flicker detection step of detecting a flicker component of an image read from an image sensor whose driving method for acquiring an image changes within a frame;
A correction value calculation step for generating a flicker correction value based on the flicker component detected in the flicker detection step and information based on the pixel read from the image sensor in accordance with the change in the driving method;
And a flicker correction step for removing a flicker component of the image read from the image sensor using the flicker correction value.   A program for causing a computer to execute each step of the control method for an image processing apparatus according to claim 8.