JP2008113237A - Flicker detecting method and device in imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a flicker detecting method and device in an imaging apparatus capable of performing accurate flicker correction by using an imaging apparatus having an image pickup element for storing electric charges in respective pixels arrayed in a matrix as in a CMOS (complementary metal oxide semiconductor) and reading a frame by scanning in a pixel row unit and making it possible to judge whether a detected flicker frequency is correct under any photographing condition. <P>SOLUTION: New flicker detection processing is executed during executing flicker correction processing, when flicker is detected by the new flicker detection processing, a flicker frequency detected during flicker detection processing before executing the new flicker detection processing is judged as erroneous detection, and the frequency judged as erroneous detection is changed to continue the flicker correction processing. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a flicker detection method and apparatus in an image pickup apparatus, and in particular, has an image pickup apparatus that stores and reads out charges as image data in individual image pickup elements by scanning in units of pixel rows, such as a fluorescent lamp. In addition, flicker correction is performed by detecting flicker that occurs when an object illuminated by a light source whose brightness changes due to the power supply frequency is captured, and flicker correction is performed, and whether the flicker detection result is correct is determined. The present invention relates to a flicker detection method and apparatus in an image pickup apparatus that can be corrected.

  Conventionally, CCDs (Charge Coupled Devices) and CMOSs (Complementary Metal Oxide Semiconductors) have been used as imaging elements that constitute imaging devices of video cameras and digital cameras. Of these, CMOS-based image sensors can be used in the same manufacturing process as LSIs compared to CCD-based image sensors, making them more mass-productive and providing various functions for each pixel, including amplifiers. The size of the entire circuit can be reduced, and the cost is advantageous. In addition, a single power source with a low voltage is sufficient, and there is an advantage that smear, which is a problem in the CCD, is so small that it can be ignored, and that data can be read at high speed.

  However, the image pickup device using CMOS has a lot of noise, and the image pickup data from the image pickup device constituting each pixel arranged in a matrix is taken out by scanning in units of pixel rows, unlike the CCD. At this time, the timing of performing the charge exposure is different for each line, and the image data is read in time series, so that the accumulation time is shifted by the time required for scanning for each scanning line. Due to this drawback, in an imaging device using CMOS, when a subject moving at high speed is imaged, an image bent in the scanning direction is output, or the illumination of the subject is caused by the power supply frequency such as a fluorescent lamp. When a light source whose brightness changes is used, a bright and dark striped pattern is generated in an image called flicker as exaggerated in FIG.

  This flicker occurs when the fluorescent lamp repeats blinking 100 times per second (in the case of 50 Hz) or 120 times (in the case of 60 Hz) depending on the power supply frequency. For example, the frame rate of the imaging device is set to 15 times / second. Then, this blinking occurs in one frame about 6.7 times at 50 Hz, and 8 times at 60 Hz. If the charge accumulation time in each image sensor is not an integral multiple of the blinking period of this light source, The stripe pattern as shown in FIG. 7 is generated in the image.

  On the other hand, in the CCD, since the charge accumulated at the same time by light irradiation is conveyed to the amplifier using the transfer function as it is, the reading of the entire data is slower than the CMOS, but unevenness of light and dark in the screen. It does not occur, the sensitivity of the CCD itself is higher than that of CMOS, and there is little noise.

  However, for example, in a single-lens reflex digital camera, the size of the sensor can be increased, so that the sensitivity and noise can be easily improved. Furthermore, an image sensor using CMOS can easily realize the continuous shooting function required for a single-lens reflex digital camera or the like due to its high speed, and this is combined with the advantages of the image sensor using CMOS described above. Many recent single-lens reflex digital cameras are equipped with an image sensor using this CMOS. The above-mentioned flicker problem can also be prevented by setting the shutter time, that is, the charge accumulation time of each image pickup device in the CMOS, to a multiple of the interval of light and dark, that is, the blinking cycle of the light source depending on the power supply frequency.

  However, as described above, it is necessary to accurately detect the blinking cycle of the light source caused by the power supply frequency in order to eliminate or make the stripe pattern due to flicker disappear or inconspicuous. With regard to such technology, for example, Patent Document 1 discloses a circuit in which a photodiode and a 100 Hz band-pass filter and a full-wave rectifier circuit and a comparator are connected in series, a 120 Hz band-pass filter and a full-wave filter in the same photodiode. A circuit in which a rectifier circuit and a circuit in which a comparator are connected in series is prepared and flickering is prevented by detecting the blinking cycle of a fluorescent lamp is shown.

  In Patent Document 2, in an imaging apparatus using a CCD that can capture both a moving image and a still image, an imaging signal is input to a flicker detection unit provided with a microcomputer, and whether or not it fluctuates periodically. A method for determining whether flicker is observed (see paragraph 0077) and a sensor for detecting a periodic change in illumination light are separately provided, and whether a flicker is generated by taking a signal from the sensor into a microcomputer. Is shown (see paragraph 0082).

  Further, Patent Document 3 discloses that a plurality of integrating means 1 for integrating the pixel levels of the video signal for each line in order to accurately detect flicker even when the luminance level of the video signal is fluctuated due to the movement of the subject, etc. An averaging unit 3 that averages the output of the integrating unit 1 with respect to lines at the same image position in a frame or a field; a stationary part extracting unit 4 that extracts a static part of the image using the output of the integrating unit 1; Dividing means 5 for dividing the integration result for each line of the integrating means 1 by the averaging result for each line of the averaging means 3 with respect to the static part extracted by the static part extracting means 4, and the division of the dividing means 5 A flicker detection / correction method including a flicker determination means 6 for determining the presence / absence of flicker by frequency analysis of the result is shown.

  In Patent Document 4, a plurality of flicker detection frames are set by dividing an image in the vertical scanning direction, luminance data is detected for each flicker detection frame, and the front and rear 2 frames are detected for each flicker detection frame. A method of calculating a flicker frequency by extracting a flicker component by taking a difference from luminance data of one frame is shown. In the flicker detection method disclosed in Patent Document 4, adjustment is made so that the frame rate of the imaging signal and the flicker frequency are not synchronized so that the flicker component does not disappear when the difference between the previous and subsequent frames is taken. .

JP-A-64-81580 JP-A-9-51481 JP 2001-119708 A JP 2003-189129 A

  However, in the flicker circuit disclosed in Patent Document 1, a photodiode, 100 Hz and 120 Hz band pass filters, a full-wave rectifier circuit, a comparator, and the like must be prepared only for flicker detection, and the circuit becomes complicated. It is uneconomical. The image pickup apparatus disclosed in Patent Document 2 is an image pickup apparatus using a CCD. Unlike a CMOS, the image pickup apparatus shown in FIG. 7 does not show a stripe pattern as shown in FIG. 7 due to flicker. The technology cannot be used as it is.

  On the other hand, the methods disclosed in Patent Document 3 and Patent Document 4 take the difference between the current frame image and the previous frame image, extract the flicker component, and then extract the bandpass filter (BPF), Fourier transform, and flicker component. Counting peaks and valleys and detecting flicker frequency components are effective in detecting flicker frequency in an imaging apparatus using CMOS.

  However, even in the methods disclosed in Patent Document 3 and Patent Document 4, for example, a moving object is included in the image data, the difference from the image data of the previous frame is large due to camera shake, and the shutter speed is further increased. If the (exposure time) becomes longer and the change in the amount of light due to flicker becomes smaller and the detection accuracy becomes worse, the flicker component cannot be extracted correctly and erroneous determination may occur.

  That is, if a moving object is included in the image data, or if the difference between the previous frame image data is large due to camera shake or the like, the image moves when the difference between the current frame image and the previous frame image is taken. For example, 50 Hz is erroneously determined to be 60 Hz, and vice versa, 60 Hz is erroneously determined to be 50 Hz, or flicker is not accurately extracted. It may happen that the presence / absence of the error itself is erroneously determined.

  Therefore, as disclosed in Patent Document 3, the static part extraction means 4 for extracting the static part of the image, and the integration result for each line of the integration means 1 for the static part extracted by the static part extraction means 4 is averaged. Although the dividing means 5 etc. for dividing by the averaged result for each line of the converting means 3 can be provided, thereby improving the accuracy of the detection method, it requires very complicated processing and is still like a hand shake. If the part does not occur, the misjudgment cannot be completely eliminated.

  For this reason, in the present invention, an imaging device having an imaging element that performs charge accumulation and frame reading in each pixel array arranged in a matrix like a CMOS and scanning in units of pixel rows is used, and under any imaging conditions. It is also an object to provide a flicker detection method and apparatus in an image pickup apparatus that can determine whether or not the detected flicker frequency is correct and can perform accurate flicker correction.

In order to solve the above problems, a flicker detection method in an imaging apparatus according to the present invention is as follows.
A flicker detection method in an imaging apparatus that performs flicker detection processing that detects flicker presence and absence and flicker frequency in a captured image generated under a blinking light source, and performs flicker correction processing based on the detection result,
A new flicker detection process is performed during the flicker correction process, and when flicker is detected by the new flicker detection process, the flicker correction process is made to correspond to a frequency different from the flicker frequency. .

In addition, the flicker detection apparatus in the imaging apparatus that implements this method is:
Each pixel is arranged in a matrix, and has an imaging device having an imaging device that performs charge accumulation and frame readout by scanning in units of pixel rows, and under a blinking light source included in image data from the imaging device In flicker detection for detecting presence / absence and frequency of flicker that occurs, a flicker detection / correction unit that performs a flicker correction process by taking a difference in accumulated charge amount between a plurality of temporally continuous frames and detecting a flicker component In the flicker detection device of the imaging device
The flicker detection / correction unit performs a new flicker detection process in the flicker correction process execution state, and if the flicker generation state is present when the new flicker detection process is executed, the flicker correction process is performed at a frequency different from the flicker frequency. It has the function to respond | correspond to.

  In this way, when a new flicker detection process is performed during the flicker correction process and flicker is detected as a result, the flicker frequency detected during the first flicker detection process is an error. The simple flicker correction is performed by simply changing the flicker frequency, so that accurate and correct flicker correction can be performed easily.

  The image pickup device includes an image sensor that performs charge accumulation and frame reading in each pixel arranged in a matrix by scanning in units of pixel rows, and the flicker detection processing includes a stored charge amount between a plurality of temporally continuous frames. By taking this difference, data reflecting flicker due to blinking of the light source can be obtained, and flicker can be detected with high accuracy.

  Further, the flicker detection process is continuously performed, and when the detection result does not change a predetermined number of times, the flicker detection process is stopped, so that an accurate flicker correction is performed without making a wrong determination again. be able to.

  Further, when the brightness of the subject changes by a predetermined amount or more in a state where the flicker detection process is stopped, the flicker detection process is resumed, for example, the lighting changes from a fluorescent lamp to an incandescent lamp or moves to the outdoors. In such a case, flicker is not generated, and flicker correction processing is not necessary, and unnecessary flicker correction is not required.

  Furthermore, when the brightness of the subject exceeds a predetermined value, the flicker detection / correction process is stopped, so that, for example, image data from the imaging apparatus or a separately provided photometric sensor is used as the brightness detection means. When the brightness of the subject at the output, that is, the illuminance is the illuminance of a general fluorescent lamp (about 300 Lux to 700 Lux), flicker detection processing is performed, and when the illuminance is clearly larger than this, If flicker correction is performed, stopping the detection can further prevent erroneous detection.

  According to the present invention, even if an image pickup apparatus having an image pickup element that performs charge accumulation in each pixel and frame readout by scanning in units of pixel rows, as in a CMOS, with a simple configuration and method, the detected flicker is detected. It is possible to provide a flicker detection method and apparatus in an imaging apparatus that can correct the frequency and always perform accurate flicker correction.

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. However, the configuration described in this embodiment is merely an illustrative example, and is not intended to limit the scope of the present invention.

  FIG. 1 is a block diagram showing a schematic configuration of an embodiment of an imaging apparatus having a flicker detection apparatus according to the present invention. In the figure, reference numeral 10 denotes an image sensor such as a CMOS in which charge storage elements for imaging a subject are arranged in a matrix and charges are stored in each pixel and a frame is read out by scanning in units of pixel rows. An image pickup unit that has a gain amplifier that amplifies an output signal from the image pickup device, an A / D converter that converts an analog image signal into a digital image signal, and the like, and 11 outputs an image signal. 12, a camera signal processing unit having a luminance signal processing unit 13, an image memory unit 14, a flicker detection / correction unit 15, and the like.

  The image processing unit 12 in the camera signal processing unit 11 performs image processing such as color interpolation, noise removal, contour enhancement, and image format change on the image signal transmitted from the imaging unit 10, and the luminance signal processing unit 13. Then, signal processing necessary for autofocus (AF), automatic exposure (AE), white balance correction (AWB), and the like is performed, and the processed image data is stored in the image memory unit 14. The flicker detection / correction unit 15 extracts and detects flicker components from the image data stored in the image memory unit 14, and determines whether the detection result is correct. Based on the flicker correction.

  As will be described later, the flicker detection / correction unit 15 divides an image signal of one frame into a plurality of frames in the vertical direction in order to determine the flicker frequency by taking the difference of the accumulated charge amount between a plurality of consecutive frames, for example. And calculating means for calculating the average brightness for each frame and calculating the difference between the average brightness of the corresponding frames in consecutive frames, and a storage device for the calculation result and the like. In addition to detecting flicker, a luminance signal in an image signal sent from the image pickup unit 10 is used, or a photometric sensor provided separately is used to detect the brightness of a subject and the brightness, that is, the illuminance is generally When it is judged that the illumination intensity of a typical fluorescent lamp is clearly larger than the illuminance (about 300 Lux to 700 Lux), it is judged to be outside and has a function of preventing flicker detection.

  In the normal state in which flicker detection is not performed, the image pickup apparatus having the flicker detection / correction unit 15 according to the present invention amplifies the image signal from the image pickup device by the gain amplifier in the image pickup unit 10 and then converts the analog image signal to A The digital signal is converted by the / D converter, the image processing unit 12 of the camera signal processing unit 11 performs image processing such as color interpolation, noise removal, contour enhancement, and image format change, and the luminance signal processing unit 13 performs autofocus ( AF), automatic exposure (AE), white balance correction (AWB), and the like are performed, and the processed image data is stored in the image memory unit 14 and displayed on a display device (not shown). The shutter (not shown) is opened and closed to pick up an image, and this is also recorded in an external memory (not shown).

  In the imaging apparatus configured as described above, when the subject is illuminated with a light source having a light amount change caused by a power supply frequency or the like, the brightness of the subject falls below a predetermined value as described above, thereby causing flicker. When it is determined that there is a possibility, the presence / absence of flicker and the flicker frequency are determined. The present invention determines whether or not the flicker frequency is correct. Prior to the description of the present invention, a normal flicker determination routine will be briefly described.

  FIG. 4 is a diagram for explaining the outline of flicker detection in the flicker detection / correction unit 15. FIG. 4A is a diagram showing a state in which image data for one frame is divided when flicker detection is performed. (B) is a diagram showing the average value of the luminance signal in each frame divided as in (A), (C) is a diagram showing a band-pass filter (BPF) for detecting the flicker frequency, and FIG. FIG. 4 is a diagram for explaining that a stripe pattern caused by flicker appears clearly on an image by shortening the charge accumulation time when detecting flicker in an image pickup device using CMOS.

  First, one frame of image data sent from the imaging unit 10 of FIG. 1 and subjected to the above-described processing and stored in the image memory unit 14 is converted into a flicker detection / correction unit 15 in FIG. As shown, detection frame [0], detection frame [1],..., And detection frame [15] are equally divided in the vertical direction. The number of divisions of image data for one frame is set so that the total time required for scanning within each frame is shorter than the light source blinking cycle to be detected. That is, for example, if the frame rate of the imaging unit 10 is 15 times / second, this blinking occurs approximately 6.7 times in the case of 50 Hz and 8 times in the case of 60 Hz in one frame. In order to make the maximum luminance part a separate frame, it is necessary to have at least 16 divisions. A larger number of frame divisions is advantageous for frequency detection accuracy, but the calculation processing load increases, and is determined in consideration of this point. In the following description, a case where image data for one frame is divided into 16 as an example will be described as an example. However, this is not limited to 16 divisions. It is obvious that it is good.

  Then, as shown by 33 in FIG. 5, the charge accumulation time of the image pickup device using the CMOS constituting the image pickup unit 10 is set to a value that clearly shows the light and darkness due to flicker, and the amount of charge accumulated thereby An instruction to compensate for this decrease by increasing the gain of the amplifier of the imaging unit 10 is sent to the imaging unit 10 through the signal line 16 as a signal accompanying flicker correction. In FIG. 5, reference numeral 31 denotes a vertical synchronizing signal, 32 denotes a change in brightness of the light source depending on the power supply frequency, and 33 denotes a black line corresponding to each horizontal line of the image sensor using CMOS. The length of charge accumulation time 34 indicates the light and darkness caused by flicker generated by the captured image data.

  As can be seen from FIG. 5, in the flicker detection process, as the charge accumulation time 33 of the image sensor using the CMOS constituting the image capturing unit 10 is shortened, the relative ratio of the light amount integral values between the horizontal elements increases. In this case, as shown at 34, data in which the brightness change due to flicker appears remarkably is obtained, and therefore the accuracy of flicker detection is increased. In addition, since the charge accumulation time 33 is shortened, the influence of the image data on the subject blur and camera shake is reduced, and flicker detection can be performed more effectively.

  On the other hand, as shown in FIG. 6, when the charge accumulation time of the image pickup device using the CMOS constituting the image pickup unit 10 is made longer than the light source brightness change period 32 by 51, each device becomes a subject. Since the image data obtained by integrating the brightness changed due to the flicker in the image is obtained, the average value of the brightness change of the light source that illuminates the subject is recorded as the obtained image data. The effect on the image due to is relatively small. Therefore, in this case, flicker detection becomes difficult.

  Then, as shown in FIG. 4A, the image data for the frames sent from the imaging unit 10 to the camera signal processing unit 11 and stored in the image memory unit 14 are detected frames [0] and [1]. ],... Equally divided in the vertical direction with the detection frame [15], and the average value of the luminance signal in each frame is shown in FIG. 4B as Y [0], Y [1],. Calculate and store as shown in [15].

  Next, the flicker detection / correction unit 15 similarly divides the frame stored in the image memory unit 14 into detection frames [0], detection frames [1],..., And detection frames [15]. The average value of the luminance signal in the frame of is calculated as indicated by Y [0], Y [1],..., Y [15] in FIG. The difference from the average value of the luminance signal of the frame is calculated.

  This difference calculation process is for erasing the subject data included in the image data and extracting only the light and dark stripe pattern (flicker component) due to flicker. At this time, in order to leave only the light and dark due to the flicker, the positions where the flicker appears in the two frames that are different are made different. Otherwise, the frame rate and the flicker frequency (the power supply frequency of the fluorescent lamp) are synchronized, so the position where the flicker occurs is fixed on the screen, and the flicker component disappears when the difference is taken. Because.

  The frame rate synchronized with the flicker frequency is a value obtained by dividing the power supply frequency of the fluorescent lamp by a positive integer or a value obtained by multiplying the positive integer by a positive integer. For example, when the power supply frequency is 50 Hz, 100 frames / second, 50 frames / second 25 frames / second,..., And in the case of 60 Hz, 120 frames / second, 60 frames / second, 30 frames / second,. Therefore, in order to avoid this, the horizontal synchronization period is set to be longer or shorter so that the frame rate becomes a value slower or faster than the value synchronized with the flicker frequency so that the flicker component rises. To do.

  When the flicker component is extracted in this way, the luminance signal average values indicated by Y [0], Y [1],..., Y [15] in FIG. The luminance signal average value input terminal 21 in the bandpass filter shown in FIG. The bandpass filters 22 and 23 are for detecting the blinking frequency of the light source that illuminates the subject that causes flicker. Since the brightness of the light source changes in a cycle that is twice the power frequency, 22 is set to 100 Hz. Correspondingly, 23 is provided corresponding to 120 Hz.

  Therefore, since the band-pass filters 22 and 23 output a wave having a frequency corresponding to the frequency of the input luminance signal average value, the output from the output terminals 24 and 25 determines the presence / absence of flicker and the frequency. I do. If no flicker is detected, the charge accumulation time of the image sensor is reset to the value before the setting for flicker detection. If there is flicker, the image sensor using the CMOS in the imaging unit 10 is based on the detected flicker frequency. The flicker correction signal 16 in FIG. 1 is sent to the imaging unit 10 so as to obtain an image without flicker so that the charge accumulation time (shutter time) is set to a flicker-corresponding accumulation time that is an integral multiple of the flicker interval.

  The determination of the presence / absence of flicker and the frequency may be performed based on the average luminance signal value shown in FIG. In other words, flicker can be determined to be flicker when there is a difference in the luminance signal average value of each frame shown in FIG. 4B, so that one luminance is placed next to one frame. Next, it is compared with the next to the luminance signal average value of all the frames, and it is determined that there is flicker when a difference in brightness occurs at a constant interval. Further, the flicker frequency is determined by the interval. calculate. In other words, as described above, if the frame rate of the imaging unit is 15 times / second, this blinking will occur approximately 6.7 times in the case of 50 Hz and 8 times in the case of 60 Hz in one frame. When 16 is set to 16, a dark portion or a bright portion comes every two or three in the case of 50 Hz, and every two in the case of 60 Hz.

  In this way, an image without flicker can be obtained by detecting flicker and adjusting the shutter time. As described above, a moving object is included in the image data as described above. If the difference from the image data of the previous frame is large due to camera shake, etc., or if the shutter speed (exposure time) is long and the change in the amount of light due to flicker is small and the detection accuracy is poor, the flicker component is correctly extracted. For example, it is determined that 60 Hz flicker is generated from a subject image illuminated with a fluorescent lamp in a 50 Hz region, or conversely, 50 Hz from a subject image illuminated with a fluorescent lamp in a 60 Hz region. An erroneous determination may be made in which it is determined that flicker has occurred. Therefore, in the present invention, the determination process of the flicker detection result based on the flowchart shown in FIG. 2 is performed so that the correction can be made when the flicker frequency is erroneously determined due to erroneous detection.

  First, when the process starts in step S20, it is first determined in step S21 whether or not the flicker mode is confirmed. The final state of the flicker mode means the following state.

  That is, the normal flicker is detected by the flicker detection / correction unit 15 as described above using the luminance signal in the image signal sent from the imaging unit 10 or a photometric sensor provided separately. Then, it is performed when it is determined that the brightness, that is, the illuminance is the illuminance of a general fluorescent lamp (about 300 Lux to 700 Lux) (however, this is an example, and it may be always performed). However, even under lighting conditions where flicker detection must be performed, flicker detection and a process for determining whether the detection result is correct are performed a plurality of times according to the flowchart shown in FIG. 2 (shown in step S25 in FIG. 2). If the flicker frequency is determined to be correct multiple times in succession, the subsequent flicker detection is not performed until the illumination condition changes, and the detected flicker frequency is detected. The state where the flicker correction is continuously performed is referred to as a confirmed flicker mode.

  For this reason, if it is determined in step S21 that the flicker mode is in the finalized state, the process proceeds to step S33 and the process ends. If not, the process proceeds to step S22. In step S22, the flicker detection process described above is performed. In the next step S23, it is determined whether or not flicker is present. If there is further flicker, the flicker frequency is determined. If it is determined that there is no flicker, the process proceeds to step S24 where the flicker detection processing count value n is (n + 1), and it is further determined in step S25 whether the value of n is equal to or greater than the flicker stable continuous detection count x. If Yes, the flicker mode is determined, and if No, the process proceeds to step S33 and ends. When the process proceeds from step S25 to step S33, and the illuminance of the subject changes, it is determined whether or not to continue the process based on the flow of FIG. 2 according to the flowchart of FIG.

  On the other hand, if flicker is detected in step S23 and the flicker is determined to be flicker based on 50 Hz, the process proceeds to step S27. If the flicker is determined to be 60 Hz, the process proceeds to step S28. In either case, in step S27 or step S28, it is determined whether flicker correction is currently being performed based on 50 Hz (in the case of step S27) or 60 Hz (in the case of step S28).

  This is a process for determining whether or not flicker detection has been performed before this time. The flicker detection process in step S22 described above determines that there is flicker in step S23. This indicates that the flicker is still present even though the flicker frequency detected previously is wrong and correction processing is performed so as not to generate flicker.

  Therefore, if it is determined in step S23 that the flicker is based on 50 Hz, for example, and if it is determined in step S27 that the flicker is further corrected based on 50 Hz, the flicker correction based on the past 50 Hz and the current determination (for example, the flicker frequency is determined to be 50 Hz). ) Is an error, the process proceeds to step S31, flicker correction based on 60 Hz is executed, and the mode is changed to the 60 Hz flicker correction mode in step S32. In the same manner, if it is determined in step S23 that the flicker is based on 60 Hz, for example, and if it is determined in step S28 that the flicker is further corrected based on 60 Hz, the flicker correction based on the past 60 Hz and the current determination are incorrect. Therefore, the process proceeds to step S29, and flicker correction based on 50 Hz is executed. In step S30, the mode is set to the 50 Hz flicker correction mode.

  If it is determined in step S23 that the flicker is based on 50 Hz, for example, and if it is determined in step S27 that the flicker is not currently being corrected based on 50 Hz (that is, the flicker is being corrected based on 60 Hz), the process proceeds to step S29 to 50 Hz. Based on the flicker correction, the mode is changed to the 50 Hz flicker correction mode in step S30. Similarly, if it is determined in step S23 that the flicker is based on 60 Hz, and if it is determined in step S28 that the flicker is not being corrected based on 60 Hz (that is, the flicker is being corrected based on 50 Hz), the process proceeds to step S31. The flicker correction based on the above is executed, and the mode is changed to the 60 Hz flicker correction mode in step S32.

  In this way, if the flicker frequency is erroneously detected for some reason as described above, flicker remains when flicker correction is performed based on the erroneous detection result, so that correct flicker correction can be performed by detecting it. It is possible to solve the problem that flicker occurs in the photographed image.

  The above is the flicker detection result correctness determination method according to the present invention. As described above, the flicker detection result correctness determination shown in FIG. 2 is performed under the illumination conditions in which flicker is considered to exist. The flicker mode is determined as described above, and this is determined by the flowchart shown in FIG.

  The processing shown in FIG. 3 is basically performed every time an image is captured. This is because the light source that illuminates the subject changes from a light source that requires flicker detection to a light source that does not require flicker detection, and vice versa. This is because there is a possibility of changing from a light source that is not required to a light source that is required, and it is unnecessary to detect flicker by detecting the brightness of the subject. This is to prevent erroneous detection from being performed. However, as described above, the flicker detection may be performed regularly instead of determining whether or not the brightness of the subject is detected in this way.

  First, when the processing starts in step S40, the image capturing unit shown in FIG. 1 in FIG. 1 captures an image in step S41, and the image processing unit 12 and the luminance signal processing unit 13 perform processing on the captured image, and the image memory unit. 14 is stored. Then, the brightness (illuminance) of the stored image is detected, and whether or not the photometric value is not more than a predetermined value that clearly indicates that it is outdoor light in step S43, or the subject illuminance is that of a general fluorescent lamp as described above. It is determined whether the illuminance is about (approximately 300 Lux to 700 Lux).

  If it is determined that the brightness of the subject is greater than the predetermined threshold value, the process proceeds to step S44 to determine whether or not the flicker correction is currently being performed. If the flicker correction is not currently performed, the process proceeds to step S52. In step S45, the flicker correction is stopped, and if the flicker mode is determined in step S46, the flicker mode is cleared. In step S52, the previously measured photometric value is replaced with the currently measured photometric value. To finish the process.

  On the other hand, if it is determined in step S43 that the brightness of the subject is smaller than the predetermined threshold value, the process proceeds to step S47 to determine whether or not this photometry is the first time, and in the case of the first time, the process proceeds to step S52. If the process is performed and the process is not the first time, the process proceeds to step S48. In step S48, the absolute value of the difference between the current photometric value and the previous photometric value is calculated as the luminance change amount. In step S49, it is determined whether the luminance change amount is greater than or equal to a predetermined threshold value. This determination is made when, for example, the light source having flicker under the fluorescent lamp is moved to a light source without flicker under the incandescent lamp. If the determination result is No, the process proceeds to step S51. If the process proceeds to S50 and the flicker mode is being determined, it is cleared.

  Then, the process proceeds to step S51, and flicker determination is performed according to the flowchart of FIG. 2, and when that is done, the process proceeds to step S52, where the above-described process is performed, and the process ends in step S53. In the flow chart of FIG. 3, even when the flicker mode is confirmed under a fluorescent lamp, for example, even when flicker correction becomes unnecessary by moving to an incandescent lamp with the same illuminance or outdoors with the same illuminance. In step S21 of step 2, it is determined that the flicker mode is confirmed, so the flicker mode is determined to continue. For example, by clearing the flicker mode at intervals of 5 to 10 seconds and performing flicker detection, the flicker mode is determined. Can deal with other problems.

  In this way, any imaging condition using an imaging device that performs charge accumulation and frame readout by pixel row unit scanning in each pixel arranged in a matrix like a CMOS can be obtained. Even if it is below, it can be determined whether or not the detected flicker frequency is correct, and accurate flicker correction can be performed with a simple configuration.

  According to the present invention, even an image pickup apparatus having an image pickup element using CMOS can obtain an image in which flicker is always corrected correctly, and the image pickup apparatus can be configured with high accuracy.

It is a block diagram which shows schematic structure of one Embodiment in the imaging device which has a flicker detection apparatus which becomes this invention. It is a flowchart of the flicker detection result correctness determination method according to the present invention. It is a flowchart for determining whether or not to perform flicker detection processing in the flicker detection result correctness determination method according to the present invention. 4A and 4B are diagrams for explaining an outline of flicker detection in the flicker detection / correction unit 15. FIG. 4A is a diagram illustrating a state in which image data for one frame is divided when flicker detection is performed, and FIG. FIG. 6A is a diagram showing an average value of luminance signals in each frame divided as in FIG. 5A, and FIG. 6C is a diagram showing a band-pass filter (BPF) for detecting the flicker frequency. It is a figure for demonstrating that the stripe pattern produced by a flicker appears clearly on an image by shortening charge accumulation time in the case of flicker detection in an imaging device using CMOS. It is a figure showing the light quantity change which arises by flicker when not shortening charge accumulation time and raising a frame rate. It is a figure which exaggerated and showed the flicker which arises by using CMOS as an image pick-up element, and using the light source from which brightness changes according to a power supply frequency for illumination of a to-be-photographed object.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image pick-up part 11 Camera signal processing part 12 Image processing part 13 Luminance signal processing part 14 Image memory part 15 Flicker detection / correction part 20 One frame of image signal 21 Luminance signal average value input terminal 22 Band pass filter 23 corresponding to 100 Hz 120 Hz Bandpass filters 24 and 25 corresponding to the output terminal of the detection output

Claims (7)

A flicker detection method in an imaging apparatus that performs flicker detection processing that detects flicker presence and absence and flicker frequency in a captured image generated under a blinking light source, and performs flicker correction processing based on the detection result,
A new flicker detection process is performed during the flicker correction process, and when flicker is detected by the new flicker detection process, the flicker correction process is made to correspond to a frequency different from the flicker frequency. Flicker detection method in imaging apparatus.   An image pickup device that performs charge accumulation and frame readout in each pixel arranged in a matrix by scanning in units of pixel rows, and the flicker detection process includes a difference in accumulated charge amount between a plurality of temporally continuous frames. The flicker detection method in the imaging apparatus according to claim 1, wherein the flicker detection method is performed.   The flicker detection method according to claim 1 or 2, wherein the flicker detection process is continuously performed, and the flicker detection process is stopped when a detection result does not change a predetermined number of times.   4. The flicker detection method for an imaging apparatus according to claim 3, wherein the flicker detection process is resumed when the brightness of the subject changes by a predetermined amount or more in a state where the flicker detection process is stopped.   5. The flicker detection method for an image pickup apparatus according to claim 1, wherein the flicker detection / correction process is stopped when the brightness of the subject exceeds a predetermined value. Each pixel is arranged in a matrix, and has an imaging device having an imaging device that performs charge accumulation and frame readout by scanning in units of pixel rows, and under a blinking light source included in image data from the imaging device In flicker detection for detecting presence / absence and frequency of flicker that occurs, a flicker detection / correction unit that performs a flicker correction process by taking a difference in accumulated charge amount between a plurality of temporally continuous frames and detecting a flicker component In the flicker detection device of the imaging device
The flicker detection / correction unit performs a new flicker detection process in the flicker correction process execution state, and if the flicker generation state is present when the new flicker detection process is executed, the flicker correction process is performed at a frequency different from the flicker frequency. A flicker detection apparatus in an imaging apparatus, characterized by having a function corresponding to   The flicker detection / correction unit has a function of detecting the brightness of a subject from captured image data, and has a function of stopping the flicker correction process when the brightness of the subject exceeds a predetermined value. The flicker detection apparatus in the imaging apparatus according to claim 6.

Images