JP2009213076A - Imaging device and control method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To adaptively perform a countermeasure against a flicker in response to a change of a light source. <P>SOLUTION: The change of the light source is detected in a live view mode irrespective of whether a line flicker is generated in an image displayed in a display unit. When the change of the light source is detected, a flicker detection operation is started, and it is detected whether the line flicker is generated in the image displayed in the display unit. A flicker deletion mode is adaptively started, therefore, in response to the change of the light source. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an imaging apparatus that captures a subject using an imaging element and a control method thereof.

  2. Description of the Related Art Conventionally, an imaging apparatus having a so-called live view mode in which an image signal based on an imaging signal read out from an imaging element at a predetermined cycle is sequentially output to a display device such as a liquid crystal display provided in the apparatus has been widely used. By using the live view mode, the photographer can observe the subject image in real time without looking into the viewfinder. The live view mode is installed not only in digital video cameras that mainly shoot moving images but also in digital cameras that mainly shoot still images.

  If display or recording in the live view mode is performed under a light source such as a fluorescent lamp that repeatedly blinks according to the frequency of the commercial power supply, flicker may occur in the recorded image or the display image in the live view mode. This flicker occurs due to the relationship between the charge accumulation time in the image sensor, the frame frequency based on the period when the charge is read from the image sensor, and the emission frequency of the fluorescent lamp.

  The occurrence of flicker under fluorescent lamp illumination will be schematically described with reference to FIG. In FIG. 1, the fluorescent lamp is assumed to be a type using a choke coil. Fig.1 (a) shows the single phase sine wave of a commercial power source, and FIG.1 (b) shows the time change of an example of the voltage applied to a fluorescent lamp with this commercial power source. FIG.1 (c) shows the time change of an example of the light quantity (luminance) which generate | occur | produces from the fluorescent lamp to which the voltage was applied like FIG.1 (b). In this example, since the fluorescent lamp emits light twice in one cycle of the commercial power supply, the emission frequency of the fluorescent lamp is twice the power supply frequency. That is, the light emission frequency of the fluorescent lamp is 100 Hz in an area where the power supply frequency is 50 Hz, and the light emission frequency is 120 Hz in an area where the power supply frequency is 60 Hz.

Under a light source in which the amount of light periodically changes, even if the charge accumulation time t in the image sensor is constant, the amount of charge accumulated differs depending on the timing of the start and end of the charge accumulation operation, and the exposure amount differs. End up. This state is shown in FIG. 1 (d) and FIG. 1 (e). Since the exposure amount is a value obtained by integrating the light amount with time, even when the exposure time t is constant, the exposure amount (area S ₁ ) during the period when the light amount of the fluorescent lamp is large and the exposure amount (area during the period when the light amount of the fluorescent lamp is small. Comparing with (S ₂ ), it can be seen that S ₁ ≠ S ₂ , and an equal exposure amount cannot be obtained.

  The difference in the exposure amount depending on the start and end timing of the charge accumulation operation under a light source whose light quantity changes periodically appears as flicker on the recorded image and the display image in the live view mode.

  By the way, in an image pickup apparatus using a CCD (Charge Coupled Device) as an image pickup element, a global electronic shutter in which the start and end of charge accumulation time are the same for all pixels is used. Therefore, the entire screen is a surface flicker that is affected by the flicker due to the frame frequency and the light emission frequency of the fluorescent lamp. In the surface flicker, if the entire screen is corrected, the influence of the flicker can be removed and has already been put into practical use.

  On the other hand, in recent years, an imaging device using an imaging element (for example, a CMOS image sensor) is widely used that sequentially resets pixels for each line, accumulates charges, and then sequentially reads the charges from the pixels for each line. . CMOS is an abbreviation for Complementary Metal-Oxide Semiconductor.

  In the CMOS image sensor, the screen is read line by line by a rolling electronic shutter. For this reason, horizontal stripe-like line flicker may occur in the imaging screen due to the relationship between the charge accumulation time, the frame frequency (the number of screens per second), and the emission frequency of the fluorescent lamp. Also, depending on the frame frequency of the moving image, the horizontal stripes due to the line flicker appear to move in the vertical direction, so that the screen becomes extremely uncomfortable during viewing. In line flicker, the effect of flicker can be eliminated by setting the charge accumulation time to an integral multiple of the light emission period of the fluorescent lamp and aligning the exposure amount for each line.

  Thus, conventionally, a technique for suppressing a flicker component in a video signal is known. However, in order to remove either surface flicker or line flicker, dedicated control is required. Therefore, a general method is to detect the presence or absence of flicker at some timing and control flicker countermeasures according to the detection result. Hereinafter, an operation mode in which flicker countermeasure control is performed is referred to as a flicker erasing mode, and an operation mode in which flicker countermeasure control is not performed is referred to as a normal mode.

  As a timing for performing flicker detection, a predetermined period immediately after the start of the live view mode is conventionally known.

Further, Patent Document 1 discloses a technique for taking a countermeasure against flicker in a camera using an image sensor having photoelectric characteristics in a linear characteristic region and a logarithmic characteristic region and having a non-integral pixel configuration. According to the description in Patent Document 1, since such an image sensor has different influences from flicker in each photoelectric conversion characteristic region, when the photoelectric conversion characteristic to be used is switched, the flicker detection unit is set again within a predetermined period. Drive.
JP 2006-13593 A

  However, these conventional configurations have a problem in that they cannot cope with changes in the presence or absence of flicker due to changes in the light source in the environment where the subject exists.

  For example, consider a case where the live view mode is activated outdoors where there is sunshine and flicker does not occur, and then moved indoors under fluorescent lighting. In this case, since it was outdoor when the live view mode was activated, flicker is not detected immediately after the start of the live view mode, and the live view is executed in the normal mode in which no flicker countermeasure is taken. Even if it moves to the indoors under the illumination of the fluorescent lamp in this state, the live view continues to be executed in the normal mode, so the live view display image is affected by flicker caused by the fluorescent lamp.

  Also, for example, conversely to the above, consider a case where the live view mode is activated indoors under fluorescent lamp illumination, and the vehicle is moved outdoors without flicker. In this case, when the live view mode is activated, since it is indoors under fluorescent lamp illumination, flicker is detected, and the live view is executed in the flicker erasure mode in which flicker countermeasures are taken. When the user moves outdoors in this state, the live view live view is continuously executed in the flicker erasure mode.

  In the flicker erasing mode, for example, in the case of a CMOS image sensor, as described above, the charge accumulation time is controlled to be an integral multiple of the light emission period of the fluorescent lamp. Therefore, the charge accumulation time, that is, the exposure time can be set only discretely. Therefore, in order to obtain proper exposure, it is necessary to use both optical aperture adjustment and adjustment of ISO sensitivity of the image sensor.

  At this time, when the aperture is reduced, the depth of field becomes deep, and it is difficult to confirm the focus of the subject on the display image by live view. Further, when the diaphragm mechanism is driven to adjust the diaphragm, a mechanical driving sound is generated, and this diaphragm driving sound is recorded together with the recorded moving image during moving image recording.

  Further, it is conceivable that the flicker detection is always performed, and whether the live view is executed in the normal mode or the flicker erasure mode is always switched according to the detection result. However, in this case, calculation processing for flicker detection is always performed, and there is a problem that calculation load increases.

  Furthermore, the above-described Patent Document 1 also changes the photoelectric conversion characteristics of the image sensor when flicker is detected in a shooting environment, and responds to changes in the presence or absence of flicker due to changes in the light source. It does not control the presence or absence of flicker countermeasures.

  Accordingly, an object of the present invention is to provide an imaging apparatus capable of adaptively taking measures against flicker according to a change in a light source and a control method therefor.

  In order to solve the above-described problem, the present invention converts the light from the subject into charges for each pixel and accumulates them, reads the charges accumulated by the image sensor for each pixel, and outputs them as an image signal for one screen. Reading means, a light source change detecting means for detecting a change in a light source for irradiating light on a subject, a flicker detecting means for detecting flicker included in an imaging signal, and a flicker detected by a flicker detecting means in the imaging signal. When a change in the light source is detected by the flicker suppressing unit and the light source change detecting unit, the flicker is detected by the flicker detecting unit, and when the flicker is detected by the flicker detecting unit, the suppression process by the flicker suppressing unit is performed. And an image pickup apparatus having control means for controlling the image pickup device.

  In addition, the present invention reads out the charge accumulated by the imaging device that converts and accumulates light from the subject into charges for each pixel and outputs the charges as a one-screen imaging signal, and irradiates the subject with light. A light source change detection step for detecting a change in the light source, a flicker detection step for detecting flicker included in the imaging signal, a flicker suppression step for suppressing flicker detected by the flicker detection step in the imaging signal, and a light source change detection step And a control step for performing control so as to perform flicker detection in the flicker detection step when flicker is detected in the flicker detection step when a change in the light source is detected by the flicker detection step. It is the control method of the imaging device.

  With such a configuration, according to the present invention, it is possible to adaptively take measures against flicker according to changes in the light source.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 2 is a cross-sectional view of an example of a digital single lens reflex camera (hereinafter simply referred to as a digital camera) to which the present invention can be applied. 2A is a cross-sectional view of the digital camera 101 in a normal state, and FIG. 2B is a cross-sectional view in a state at the time of exposure and live view, that is, a live view mode. The normal state is a state in which an object image by light incident through the photographing optical path can be confirmed with an optical viewfinder.

  In FIG. 2A, a photographic lens 102 is attached to the front surface of the digital camera 101. The taking lens 102 can be exchanged, and the digital camera 101 and the taking lens 102 are also electrically connected via the mount contact group 112. Further, the photographing lens 102 has a diaphragm 113 so that the amount of light taken into the digital camera 101 can be adjusted.

  The main mirror 103 composed of a half mirror is obliquely arranged on the photographing optical path in the finder observation state, and reflects the photographing light beam from the photographing lens 102 to the finder optical system, while the transmitted light passes through the sub mirror 104 and the AF unit 105. Incident to. Note that AF is an abbreviation for Auto Focus. The AF unit 105 includes a phase difference detection type AF sensor, and performs processing for moving the focusing lens to a focus position. The AF unit 105 detects the focus adjustment state of the photographic lens 102 by forming the secondary imaging surface of the photographic lens 102 on the focus detection line sensor, and drives a focusing lens (not shown) based on the detection result. Perform auto focus detection.

  A light source detection unit 116 as a light source detection unit detects a spectral distribution of a subject. The spectral sensitivity distribution is different between the AF unit 105 and the image sensor. Therefore, the focus position may vary depending on the spectral distribution of the subject due to the longitudinal chromatic aberration of the photographing lens 102. For example, the subject's spectral distribution differs depending on the spectral distribution of the light source under illumination with tungsten light, which is a light source of reddish light, and under illumination with a fluorescent lamp, which is a light source of bluish light. May also be different. The light source detection unit 116 detects the type of the light source, and corrects the focus position obtained by the AF unit 105 based on the detection result.

  During automatic focus detection control, focusing calculated from the output of the AF unit 105 based on the output of the light source detection unit 116 and axial chromatic aberration information stored in a ROM (not shown) provided in the photographing lens 102. Correct the lens drive amount. Details of the light source detection unit 116 will be described later.

  A focal plane shutter 107 (hereinafter referred to as shutter 107), a low-pass filter 106, and an image sensor 108 are arranged on the optical axis of the photographing lens 102 via a main mirror 103 and a sub mirror 104.

  A focus plate 109 is disposed on the planned image plane of the photographic lens 102 constituting the finder optical system. The pentaprism 110 changes the light path of the light incident from the photographing lens 102 and reflected by the main mirror 103 so as to be directed to the eyepiece 114. The photographer can confirm the photographing screen by observing the focus plate 109 from the eyepiece 114. An AE (Auto Exposure) unit 111 performs automatic measurement of an exposure amount, and is used when performing photometry.

  The live view start / end button 117 switches ON / OFF of a live view mode for performing live view every time it is operated. In the live view mode or exposure, as illustrated in FIG. 2B, the sub mirror 104 is retracted out of the photographing optical path, and the shutter 107 is opened to guide the photographing light flux to the image sensor 108.

  The release button 115 is a two-stage push-in switch that is half-pressed and fully pressed. By pressing the release button 115 halfway, preparatory operations such as an AE operation and an AF operation are performed. When the release button 115 is fully pressed, the state illustrated in FIG. 2B is obtained, and the image sensor 108 is exposed with light from the subject incident through the photographing lens 102, and the photographing process is performed. Done.

  A display unit 118 is attached to the back of the digital camera 101. The display unit 118 is configured by a liquid crystal panel, for example, and can directly check an image taken by the photographer. In the live view mode, a moving image based on the imaging signal output from the image sensor 108 is sequentially displayed.

  FIG. 3 shows an exemplary configuration of the light source detection unit 116. The light source detection unit 116 includes a first photometric sensor 301 and a second photometric sensor 302, and filters 303 and 304 that limit light incident on the first and second photometric sensors 301 and 302. The filter 303 is provided in common with the first and second photometric sensors 301 and 302. On the other hand, the filter 304 is provided in one of the first and second photometric sensors 301 and 302. In the example of FIG. 3, the filter 304 is provided for the second photometric sensor 302. That is, light enters the second photometric sensor 302 via the filters 303 and 304.

  FIG. 4 shows an example of spectral sensitivity characteristics (i) of the first and second photometric sensors 301 and 302 and spectral transmission characteristics (ii) and (iii) of the filters 303 and 304. In FIG. 4, the horizontal axis represents the wavelength, and the vertical axis represents the relative sensitivity when the peak sensitivity is 1 in the case of the photometric sensors 301 and 302. In the case of the filters 303 and 304, the vertical axis indicates the transmittance. The first and second photometric sensors 301 and 302 have spectral sensitivity characteristics in which the sensitivity increases gradually from the vicinity of the wavelength of 300 nm, the sensitivity is peaked at the wavelength of near 800 nm, and the sensitivity decreases at the wavelength of 800 nm or more. Practically, the wavelength is used in a band of 400 nm or more.

  The filter 303 is an IR cut filter that cuts light having a wavelength of about 750 nm or more, as exemplified by the spectral transmission characteristic (ii) in FIG. The filter 304 is formed by coating the front surface of the second photometric sensor 302, for example, and cuts light having a wavelength of about 600 nm or less, as exemplified by the spectral transmission characteristic (iii) in FIG. . The filters 303 and 304 function as bandpass filters that selectively transmit light having a wavelength from about 600 nm to a wavelength of about 750 nm to the second photometric sensor 302.

  The light source detection unit 116 compares the outputs of the first and second photometric sensors 301 and 302 whose spectral characteristics of incident light are made different by the filters 303 and 304, respectively, and compares the spectral distribution characteristics of the subject, that is, the type of the light source. Is determined.

  FIG. 5 shows examples of spectral distribution characteristics of typical various light sources. In FIG. 5, the horizontal axis indicates the wavelength, and the vertical axis indicates the relative energy of light. A characteristic L1 indicates a spectral distribution characteristic of an example of a flood lamp that is tungsten light. The flood lamp has a spectral distribution characteristic in which energy increases from the short wavelength side toward the long wavelength side. A characteristic L2 indicates a spectral distribution characteristic of an example of a fluorescent lamp. As is well known, a fluorescent lamp emits mercury vapor by arc discharge and converts this light into visible light by a phosphor. Therefore, in the spectral distribution characteristics, there is an energy peak at a wavelength corresponding to the three primary colors of light due to the emission characteristics of mercury vapor, and the energy is extremely small in portions other than the peak. A characteristic L3 indicates the spectral distribution characteristic of sunlight. In the spectral distribution characteristic of sunlight, the ratio of the long wavelength component having a wavelength of 650 nm or more is smaller than the spectral distribution characteristic of tungsten light shown in the characteristic L1.

  The first photometric sensor 301 in the light source detection unit 116 receives light having a wavelength range of about 400 nm to 750 nm through the filter 303. On the other hand, in the second photometric sensor 302, light having a wavelength range of about 600 nm to nm is incident by the filters 303 and 304. That is, the output of the first photometric sensor 301 is obtained by integrating the energy of light in the wavelength range of about 400 nm to 750 nm according to the spectral characteristics of the sensor itself. The output of the second photometric sensor 302 is obtained by integrating the energy of light in the wavelength range of about 600 nm to 750 nm according to the spectral characteristics of the sensor itself.

  As shown by the characteristic L1 in FIG. 5, in the tungsten light, the energy of the spectral component on the long wavelength side having a wavelength of about 600 nm or more is larger than the energy of the spectral component on the short wavelength side having a wavelength of about 600 nm or less. On the other hand, as shown by the characteristic L2 in FIG. 5, sunlight has a shorter wavelength component energy and a longer wavelength component energy smaller than tungsten light at a wavelength of 650 nm. Accordingly, when comparing the tungsten light and sunlight with respect to the output difference between the first photometric sensor 301 and the second photometric sensor 302, the output difference in the case of tungsten light is greater than the output difference in the case of sunlight. Smaller than.

  The output difference between the first and second photometric sensors 301 and 302 uses the ratio of the outputs of the first and second photometric sensors 301 and 302.

  As for the fluorescent lamp, as illustrated in the characteristic L2, the energy of the long wavelength component having a wavelength of 600 nm or more is very small except for the peak wavelength. For this reason, the energy of the long wavelength component of the light of the fluorescent lamp is significantly smaller than that of tungsten light or sunlight. Further, in a fluorescent lamp, when comparing a long wavelength component and a short wavelength component with a wavelength of 600 nm as a boundary, the energy of the short wavelength side spectral component is larger than the energy of the long wavelength side spectral component. Therefore, the output difference between the first and second photometric sensors 301 and 302 becomes very large.

As described above, in the light source detection unit 116, the output difference between the first and second photometric sensors 301 and 302 is the largest when the light source is a fluorescent lamp, and then the output difference when sunlight is the largest. The output difference due to is the smallest. Therefore, it is possible to determine the type of light source by performing a threshold determination for the output difference between the first and second photometric sensors 301 and 302. For example, it is conceivable to set a threshold Th ₁ for determining whether the light source is a fluorescent lamp or sunlight, and a threshold Th ₂ for determining whether the light source is sunlight or tungsten light.

  FIG. 6 shows an exemplary configuration of the digital camera 101 applicable to the present invention. In FIG. 6, the same reference numerals are given to the portions common to FIG. 2 described above, and detailed description thereof is omitted. In FIG. 6, the mirror box 201 includes the main mirror 103 and the sub mirror 104 described above. The mirror box 201 guides incident light to the AF unit 105 and the AE unit 111 (see FIG. 2A), and retracts the main mirror 103 and the sub mirror 104 from the incident light path (see FIG. 2B). Switch with.

  In a state where the incident light is guided to the AF unit 105 and the AE unit 111, the mirror box 201 divides the incident light into transmitted light and reflected light by the main mirror 103. Then, the transmitted light is applied to the sub mirror 104 and the reflected light is guided to the AE unit 111 through the focus plate 109 and the pentaprism 110. Further, the light irradiated to the sub mirror 104 is reflected by the sub mirror 104 and guided to the AF unit 105.

  The photographing control unit 202 as a control unit includes, for example, a microcomputer, has a ROM and a RAM (not shown), and uses the RAM as a work memory in accordance with programs and data stored in the ROM in advance. To control.

  A shutter drive unit 203, a mirror drive unit 204, an aperture drive unit 205, and a lens drive unit 206 are connected to the imaging control unit 202. Although not shown, the output of the AF unit 105 and the output of the AE unit 111 are supplied to the imaging control unit 202.

  The shutter driving unit 203, the mirror driving unit 204, and the aperture driving unit 205 drive the shutter 107, the main mirror 103, the sub mirror 104, and the aperture 113, respectively, under the control of the imaging control unit 202. The imaging control unit 202 performs the shutter driving unit 203, the mirror driving unit 204, and the aperture driving in accordance with various setting values set by operating an operation unit (not shown), the output of the AE unit 111, the operation on the release button 115, and the like. The unit 205 is controlled.

  Further, the output of the light source detection unit 116 is supplied to the imaging control unit 202. The imaging control unit 202 controls the lens driving unit 206 based on the output of the light source detection unit 116 and the output of the AF unit 105 to drive a focusing lens (not shown) in the imaging lens 102.

  The image sensor 108 is composed of, for example, a CMOS image sensor, and charges are read by a rolling electronic shutter system in accordance with a clock generated by a clock generator (not shown) under the control of the imaging controller 202. That is, in the image sensor 108, one line of charge is read in the horizontal direction from the pixel at the upper left corner of the screen in the pixel unit, and further, the charge for each line is read from the top to the bottom in the vertical direction. The read charge is converted into an electrical signal and output as an imaging signal. This imaging signal is converted into a digital signal by the A / D converter 207. The image sensor 108 and the imaging control unit 202 constitute reading means.

  The signal processing unit 208 uses a memory (not shown) and performs signal processing on the imaging signal converted into a digital signal by the A / D converter 207 to form a luminance signal and a color signal. At this time, the signal processing unit 208 as white balance processing means performs white balance processing based on the imaging signal, and forms a luminance signal and a color signal from the resulting signal. Information indicating the result of the white balance process is passed to the imaging control unit 202.

  The white balance process can be performed by the following method. That is, R (red), G (green), and B (blue) signals are formed from the image pickup signal, and these R, G, and B signals are respectively integrated on one screen of the image sensor 108. Then, the R, G, and B signals are compared with the level integrated for one screen, and the gains of the R and B signals are adjusted so that the levels of the R, G, and B signals are equal to each other based on the comparison result. , R, G and B signal level balance is corrected. As information indicating the correction amount of the level balance of the R, G, and B signals by the white balance processing, the gain adjustment values of the R and B signals can be used.

  The display unit 209 converts the video data and image data output from the signal processing unit 208 into a signal in a format that can be displayed on the display unit 118. Then, the display unit 209 drives the display unit 118 with the converted signal, and causes the display unit 118 to display a video or an image.

  The recording unit 210 records the image data output from the signal processing unit 208 along with shooting on a recording medium. As the recording medium, a rewritable nonvolatile memory can be applied. The recording unit 210 can perform compression encoding on the image data using a predetermined compression encoding method, and can record the compression encoded image data on a recording medium.

  In such a configuration, for example, when the live view start / end button 117 is operated by the photographer, the shooting control unit 202 controls each unit of the digital camera 101 and sets the operation mode of the digital camera 101 to the live view mode. Transition. In the live view mode, the imaging control unit 202 controls the mirror driving unit 204 to retract the main mirror 103 and the sub mirror 104 from the incident optical path, as illustrated in FIG. At the same time, the photographing control unit 202 controls the shutter driving unit 203 so that the shutter 107 is opened.

  Light from the subject is irradiated to the image sensor 108 through the photographing lens 102, the diaphragm 113 and the shutter 107. The image sensor 108 converts the irradiated light into electric charge by photoelectric conversion, reads out the electric charge for each pixel and line by a rolling electronic shutter, and outputs it as an imaging signal. The imaging signal is continuously read at a predetermined frame rate in units of frames, that is, one screen of the image sensor 108, and is output as a moving image, that is, a video signal. In the following description, it is assumed that the image signal output from the image sensor 108 in the live view mode is a video signal.

  The video signal output from the image sensor 108 is converted into a digital signal by the A / D converter 207 and supplied to the signal processing unit 208. The signal processing unit 208 performs signal processing on the supplied video signal to form a luminance signal and a color signal, and generates a color video signal. The signal processing unit 208 forms R, G, and B signals from the video signal, and performs white balance processing based on these R, G, and B signals. The processing result of the white balance processing is passed from the signal processing unit 208 to the shooting control unit 202.

  The color video signal output from the signal processing unit 208 is supplied to the display unit 209 and the recording unit 210. The display unit 209 converts the supplied color video signal into a signal in a format that can be displayed on the display unit 118 and supplies the converted signal to the display unit 118.

  Further, the luminance signal formed by the signal processing unit 208 is passed to the imaging control unit 202. The photographing control unit 202 as a measuring unit performs a photometric calculation of the subject based on the luminance signal, and performs a so-called imaging surface AE operation. That is, the amount of light from the subject is measured by the imaging surface AE. Since this imaging surface AE is a known technique, a detailed description thereof will be omitted here. The imaging control unit 202 determines the aperture value and shutter speed at the time of exposure based on the calculation result of the imaging surface AE, and controls the shutter driving unit 203 and the aperture driving unit 205.

  Furthermore, the shooting control unit 202 continuously calculates the contrast of the video signal obtained by the signal processing unit 208 while driving a focusing lens (not shown) in the shooting lens 102, so that a so-called contrast detection method is used. AF processing is also performed. The AF processing by this contrast detection method is a known technique and is not related to the present invention, and thus description thereof is omitted. The imaging control unit 202 determines the position of the focusing lens based on the result of the AF processing by this contrast detection method, and controls the lens driving unit 206 so as to move the focusing lens to the determined position.

  The output of the light source detection unit 116 is supplied to the imaging control unit 202. The imaging control unit 202 determines the type of light source as described above based on the output of the light source detection unit 116.

  FIG. 7 is a flowchart showing an exemplary operation in the live view mode in the present embodiment. When the live view start / end button 117 is operated and the live view mode is activated, in step S10, the photographing control unit 202 performs photometry of the subject using the imaging surface AE, and the amount of light from the subject is determined. Measured.

  In the next step S <b> 11, the photographing control unit 202 determines a threshold for the measurement result in step S <b> 10. That is, in the live view mode, if the subject is under fluorescent lamp illumination, it is necessary to execute the flicker elimination mode and perform the line flicker suppression process. Here, in general, even if the fluorescent lamp is directly viewed, the Ev (Exposure Value) is about 14 brightness, so if the subject is extremely bright, it can be considered that the main light source is not the fluorescent lamp. Does not require flicker detection. Therefore, a threshold determination is performed on the measurement result, and flicker detection is not performed when the brightness of the subject exceeds the threshold. For example, Ev = 14 described above can be used as the threshold value. If it is determined that the brightness obtained as a result of the measurement in step S10 exceeds the threshold, the process proceeds to step S15 described later.

  On the other hand, if it is determined in step S11 that the brightness obtained as a result of the measurement in step S10 is equal to or less than the threshold value, the process proceeds to step S12. In step S12, the flicker detection function is activated by the photographing control unit 202 as flicker detection means, and flicker in the image pickup signal is detected. Details of the operation by the flicker detection function will be described later.

  The process proceeds to step S13, and whether or not line flicker has occurred in the video signal to be displayed on the display unit 209 based on the result of the flicker detection operation started in step S12 by the imaging control unit 202. To be judged. If it is determined that no line flicker has occurred, the process proceeds to step S14. In step S14, the imaging control unit 202 executes the live view mode in a normal mode in which flicker suppression is not performed.

  In the next step S15, the photographing control unit 202 as the light source change detecting means determines whether or not there is a change in the light source. In other words, if the lighting environment of the subject changes from a non-fluorescent light to a fluorescent light or changes from a fluorescent light to a non-fluorescent light while the live view mode is being executed, the flicker detection function is activated again. It is desirable to start and check the presence of line flicker. Therefore, in step S15, the presence or absence of a change in the light source is detected. If a change in the light source is detected, the process returns to step S12 to activate the flicker detection function again.

  In the present embodiment, the following three methods are used as a method of detecting a change in the light source. The first method is a method using the measurement result of the imaging surface AE. That is, it can be considered that the light source has changed when the change amount of the measurement result by the imaging surface AE is equal to or larger than a predetermined amount during the live view mode operation. As an example, when the shooting scene changes, such as when the shooting is switched from indoor shooting to outdoor shooting, or vice versa, the light source may change with this change.

  The second method is a method using the result of white balance processing. That is, during the live view mode operation, it can be considered that the light source has changed when the amount of change in the white balance processing result is equal to or greater than a predetermined amount. As an example, it is conceivable to compare the ratio of the gain adjustment value for each of the R signal and the B signal by the white balance processing with a threshold value.

  The third method is a method using the output result of the light source detection unit 116. As described above, the light source detection unit 116 can determine the type of light source of the subject. Therefore, it can be considered that the light source has changed when the amount of change in the output of the light source detection unit 116 is equal to or greater than a preset threshold value during the live view mode operation.

  In the present embodiment, the change of the light source in step S15 is detected using the change amount of the measurement result of the imaging surface AE, the change amount of the white balance processing result, and the change amount of the output result of the light source detection unit 116. That is, in this embodiment, it is determined by determining that the light source has changed when any one of these change amounts is equal to or greater than a predetermined amount set for each. Not limited to this, the measurement result of the imaging surface AE, the white balance processing result, and the output result of the light source detection unit 116 may be combined to determine whether or not the light source has changed, and the change in the light source may be detected.

  If a change in the light source is not detected in step S15 described above, the process proceeds to step S16, and the shooting control unit 202 determines whether or not the live view mode has ended. If the live view start / end button 117 is not operated, it is determined that the live view mode has not ended, and the process returns to step S15 to detect a change in the light source. On the other hand, when the live view start / end button 117 is operated, the shooting control unit 202 ends the live view mode, and a series of processes in the flowchart of FIG.

  If it is determined in step S13 described above that line flicker has occurred, the process proceeds to step S17. In step S17, processing for detecting the cycle of line flicker is performed. A specific example of this line flicker cycle detection process will be described later. If the line flicker cycle is detected in step S17, the process proceeds to step S18.

  In step S18, the imaging control unit 202 as flicker suppressing means executes the live view mode in a flicker erasing mode that controls to suppress line flicker. Specifically, if the charge accumulation time in the image sensor is controlled to be an integral multiple of the flicker cycle, the amount of charge accumulation based on flicker is aligned for each line, so that line flicker is suppressed.

  For example, when the frequency of the commercial power supply is 50 Hz, the light emission frequency of the fluorescent lamp is 100 Hz and the flicker frequency is also 100 Hz, so the accumulation time is controlled to an integral multiple of 10 msec. For example, when the frequency of the commercial power source is 60 Hz, the emission frequency of the fluorescent lamp is 120 Hz and the flicker frequency is also 120 Hz, so the accumulation time is an integer of 8.33... Msec (= 1000/120 msec). Control twice.

  The process proceeds to step S19, and a change in the light source is detected by the imaging control unit 202 in the same manner as in step S15 described above. If a change in the light source is detected, the process returns to step S12, and the photographing control unit 202 activates the flicker detection operation again. If line flicker is not detected as a result of the flicker detection operation (step S13), the process proceeds to step S14. Then, the shooting control unit 202 stops the operation in the flicker elimination mode executed in the previous step S18, and the live view mode is executed in the normal mode in which flicker suppression is not performed.

  On the other hand, if a change in the light source is not detected in step S19, the process proceeds to step S20, and it is determined whether or not the live view mode is ended as in step S16 described above. If it is determined that the live view mode has not ended, the process returns to step S19, and a change in the light source is detected. On the other hand, if it is determined that the live view mode has ended, a series of processes according to the flowchart of FIG. 7 is ended.

  As described above, in the present embodiment, in the live view mode, the change in the light source is detected regardless of whether or not line flicker occurs in the video displayed on the display unit 209. When a change in the light source is detected, a flicker detection function is activated to detect whether or not line flicker has occurred in the video displayed on the display unit 209. Therefore, the flicker erasing mode can be activated adaptively with respect to changes in the light source.

  Next, the flicker detection operation activated in step S12 described above will be described. In step S12, step S13, and step S17 in the flowchart of FIG. 7, the presence / absence of occurrence of line flicker in the imaging signal and the cycle of line flicker are obtained as described below.

  FIG. 8 is a diagram for explaining the principle of occurrence of line flicker. FIG. 8 shows the relationship between the light emission period of the fluorescent lamp, the signal of each line in the image sensor 108, and an example display on the display screen 400 of the display unit 118. As described above, in this embodiment, a CMOS image sensor is used as the image sensor 108.

  As illustrated in FIG. 8, the light amount of the fluorescent lamp varies with a certain period, and the charge accumulation timing in the image sensor 108 changes sequentially for each line. In this case, the image pickup signal read from the image pickup device 108 fluctuates in accordance with the light quantity of the fluorescent lamp at the timing when charges are accumulated in each line, and the display screen 400 is illustrated on the left side of FIG. A stripe pattern due to line flicker occurs. Since the interval between the stripes corresponds to the light emission cycle of the fluorescent lamp, the cycle of the line flicker can be obtained by calculating the interval between the stripes. Specifically, the projection in the line direction of the image signal displayed on the display screen 400 (referred to as horizontal projection) may be taken to calculate the period.

  Here, when the subject is actually photographed, since there are various reflectances in the screen, the horizontal projection of the image signal is also affected by the reflectance of the subject. That is, the horizontal projection of the image signal is obtained by multiplying the amount of light that changes according to the light emission period of the fluorescent lamp, that is, the line, by the reflectance corresponding to each line of the subject.

  Here, as illustrated in FIGS. 9A and 9B, two images A and B having the same subject and having stripe patterns caused by line flickers shifted by 180 ° are combined. Think. For each of the images A and B, the horizontal projections Ah and Bh of the image signal are calculated, and the ratio Ah / Bh is obtained for these horizontal projections Ah and Bh. The component based on the reflectance of the subject has a ratio value of 1 in the horizontal projections Ah and Bh, so that it is possible to obtain a signal obtained by removing only the flicker component by removing the influence of the reflectance of the subject.

  For example, it is conceivable that an image signal based on the luminance component of the video signal from the image sensor 108 is developed in the memory of the signal processing unit 208, and the signal processing unit 208 performs calculations regarding the horizontal projections Ah and Bh. Not limited to this, an image signal based on this luminance component is transferred from the signal processing unit 208 to the imaging control unit 202, and the imaging control unit 202 develops this image signal in the RAM, and performs calculations relating to the horizontal projections Ah and Bh. You may make it perform. Furthermore, the calculation regarding the horizontal projections Ah and Bh can be performed using a video memory (not shown) included in the display unit 209.

  Here, the frequency of the commercial power supply is 50 Hz or 60 Hz. Reading by the image sensor 108 is performed by the image sensor 108 at a frame rate of 45 msec, that is, 22.22... Fps, for example. By doing so, the phase of the striped pattern due to the line flicker is shifted by approximately 180 ° on the two continuously read screens at any frequency of 50 Hz and 60 Hz.

  When the frequency of the commercial power source is 50 Hz, the cycle of the power source is 20 msec, and the light emission cycle of the fluorescent lamp is ½ of this and becomes 10 msec. Therefore, if a signal shifted by a half cycle with respect to the light emission cycle of the fluorescent lamp is desired, reading by the image sensor 108 may be performed at a timing after, for example, 10 msec × n + 10/2 msec, where n is an integer. When the frequency of the commercial power source is 60 Hz, the cycle of the power source is 16.66... Msec (= 1000/60 msec), and the luminous cycle of the fluorescent lamp is halved to 8.33. (= 1000/120 msec). Therefore, if a signal deviated by a half period with respect to the light emission period of the fluorescent lamp is desired, m is an integer, for example, 8.33... Msec × m + 8.33. Reading may be performed at

  For example, by setting n = 4 and m = 5, when the frequency of the power supply is 50 Hz, after reading for one screen, the next reading is performed after 45 msec, and the stripe patterns due to line flicker are mutually connected. Two screens shifted by 180 ° can be obtained. In addition, when the frequency of the power source is 60 Hz, the next reading is performed after A which has been read for one screen, and 45.833... Msec, and the stripe patterns due to the line flicker are shifted from each other by approximately 180 °. Two screens can be obtained.

  A process for determining the presence or absence of line flicker will be described with reference to FIG. FIG. 10A shows an example of a waveform by Ah / Bh calculated from the horizontal projections Ah and Bh of the images A and B in which the stripe pattern due to the line flicker is shifted by 180 ° as described above. In FIG. 10A, the horizontal axis is the number of lines, for example, the number of lines from the upper end of the display screen 400. The horizontal axis represents time if the product with the scanning time of one horizontal line is taken. The vertical axis is, for example, a luminance value.

  In the example of FIG. 10A, if the amplitude of the waveform is larger than a predetermined amount, it can be determined that line flicker has occurred. Actually, there is a possibility of erroneous detection due to uneven illuminance in the screen. Therefore, as illustrated in FIG. 10A, two windows for determining the observation area are set for the waveform, such as window # 1 and window # 2, and the waveforms in these windows # 1 and # 2 are set. Calculate the autocorrelation for. Then, as illustrated in FIG. 10B, a waveform based on the autocorrelation calculation result is obtained. In FIG. 10B, the horizontal axis represents the number of lines or time, and the vertical axis represents the correlation value.

  Here, the autocorrelation is calculated by shifting the waveform of window # 2 with respect to the waveform of window # 1. The correlation value is 100 when the peak of the waveform of window # 1 and the peak of the waveform of window # 2 overlap, and the peak of the waveform of window # 1 and the bottom of the waveform of window # 2 overlap. In this case, the correlation value is 0.

  The period of the waveform based on the autocorrelation calculation result in FIG. 10B is the line flicker period. Further, when no line flicker occurs, the amplitude of the waveform in FIG. 10B becomes small. Therefore, by setting a threshold value for the correlation value of this waveform and performing the threshold determination, The presence or absence of occurrence can be determined. As an example, a first threshold value and a second threshold value that is smaller than the first threshold value are set, and the waveform peak value is greater than the first threshold value, or the waveform bottom value is the first threshold value. When the threshold value is smaller than 2, it is determined that line flicker has occurred. However, the present invention is not limited to this, and it is also conceivable to determine whether or not line flicker occurs using only the peak of the waveform and only the bottom of the waveform.

  Regarding the determination of the presence / absence of the line flicker, the imaging control unit 202 and the signal processing unit 208 use the memory of the signal processing unit 208, the RAM of the imaging control unit 202, and the like, similarly to the processing related to the horizontal projections Ah and Bh described above. Thus, autocorrelation can be calculated.

  In the above description, the present invention has been described as applied to a digital camera mainly for taking still images. However, the present invention is not limited to this example, and the present invention is a digital video mainly for taking moving images. You may make it apply to a camera.

  In the above description, the image sensor applied to the present invention uses a rolling electronic shutter to sequentially reset the pixels for each line, accumulate charges, and sequentially read the charges from the pixels for each line. It was described as being a sensor. This is not limited to this example, and an imaging device such as a CCD that uses a global electronic shutter and starts and ends charge accumulation time at the same timing for all pixels can also be applied to the present invention.

  When a CCD is used as an image sensor, flicker detection is performed by integrating the luminance value for each frame of the image signal read from the image sensor at the frame timing and comparing the luminance value between frames. Can be considered.

It is a basic diagram for demonstrating light emission of the fluorescent lamp using a choke coil. It is sectional drawing of an example of the digital single-lens reflex camera applicable to this invention. It is a basic diagram which shows the structure of an example of a light source detection unit. It is a basic diagram which shows the example of the spectral sensitivity characteristic (i) of an example of a 1st and 2nd photometric sensor, and the spectral transmission characteristics (ii) and (iii) of two filters. It is a basic diagram which shows the example of the spectral distribution characteristic of typical various light sources. It is a block diagram which shows the structure of an example of the digital camera applicable to this invention. It is a flowchart which shows operation | movement of an example in the live view mode in this embodiment. It is a basic diagram for demonstrating the generation | occurrence | production principle of a line flicker. It is a basic diagram for demonstrating the detection method of a line flicker. It is a basic diagram for demonstrating the process which judges the presence or absence of a line flicker.

Explanation of symbols

101 Digital Camera 102 Shooting Lens 103 Main Mirror 111 AE Unit 116 Light Source Detection Unit 117 Live View Start / End Button 118 Display Unit 202 Shooting Control Unit 208 Signal Processing Unit 209 Display Unit

Claims (7)

An image sensor that converts and accumulates light from the subject into charges for each pixel;
A readout unit that reads out the electric charge accumulated in the imaging element for each pixel and outputs it as an imaging signal of one screen;
A light source change detecting means for detecting a change in a light source for irradiating the subject with light;
Flicker detection means for detecting flicker included in the imaging signal;
Flicker suppression means for suppressing the flicker detected by the flicker detection means in the imaging signal;
When the change of the light source is detected by the light source change detection unit, the flicker is detected by the flicker detection unit, and when the flicker is detected by the flicker detection unit, the suppression process by the flicker suppression unit is performed. An image pickup apparatus comprising a control means for controlling. A light source detection means for detecting the type of the light source;
The imaging apparatus according to claim 1, wherein the light source change detection unit detects that the light source has changed when the type of the light source detected by the light source detection unit changes. White balance processing means for correcting white balance for the imaging signal;
3. The light source change detection unit detects that the light source has changed when the correction amount of the white balance by the white balance processing unit changes to a predetermined amount or more. The imaging device described in 1. A measuring means for measuring the amount of light from the subject;
2. The light source change detection unit detects that the light source has changed when the amount of light from the subject changes to a predetermined amount or more based on a measurement result of the measurement unit. The imaging device according to any one of claims 3 to 4. The control means includes
The flicker detection by the flicker detection unit is controlled so as not to be performed regardless of the detection result of the light source change detection unit based on the measurement result of the measurement unit if the light amount is equal to or greater than a threshold value. Item 5. The imaging device according to Item 4. A measuring means for measuring the amount of light from the subject;
The control means includes
Based on the measurement result by the measurement means, if the amount of light from the subject is greater than or equal to a threshold value, control is performed so that flicker detection by the flicker detection means is not performed regardless of the detection result of the light source change detection means. The imaging device according to any one of claims 1 to 3, wherein A readout step of reading out the charges accumulated by the image sensor for converting light from the subject into charges for each pixel and storing the charges as an image signal for one screen;
A light source change detection step for detecting a change in a light source that irradiates the subject with light; and
A flicker detection step for detecting flicker included in the imaging signal;
A flicker suppression step for suppressing the flicker detected by the flicker detection step in the imaging signal;
When the change of the light source is detected by the light source change detection step, the flicker is detected by the flicker detection step, and when the flicker is detected by the flicker detection step, the suppression process by the flicker suppression step is performed. And a control step for controlling the imaging apparatus.