JP2008109253A - Device and method for detecting light source state, and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect whether a light source of an imaging apparatus is a fluorescent lamp and the driving frequency of the fluorescent lamp in a simple method. <P>SOLUTION: A normal exposure image of 1/60-second exposure and a short-time exposure image of 1/240-second exposure are alternately photographed at 60 fps by using a rolling shutter. Then vertical luminance distributions of a normal exposure image and a short-time exposure image which are temporarily adjacent are contrasted with each other to detect whether the light source of the imaging device is a non-inverter type fluorescent lamp. When the light source is the fluorescent lamp, vertical luminance distributions of short-time exposure images which are temporarily adjacent are contrasted with each other to detect whether the driving frequency of the fluorescent lamp is 60 or 50 Hz. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a light source state detection device and a light source state detection method for detecting the state of a light source of an image sensor, and an imaging device using them. The present invention particularly detects a technology for detecting whether or not a light source is a non-inverter type fluorescent lamp in an imaging apparatus that performs imaging with a rolling shutter, and detects a frequency of a commercial AC power source that drives the fluorescent lamp. Related to technology.

  When shooting is performed using an imaging device (such as an XY address type CMOS image sensor) having a rolling shutter characteristic under the illumination of a fluorescent lamp that is directly lit by a commercial AC power supply, the exposure time is determined from the light emission cycle of the fluorescent lamp. If it is remarkably shortened, vertical luminance unevenness occurs in each image. Further, when the photographing frame rate and the frequency of the commercial AC power source that drives the fluorescent lamp are different, the state of uneven luminance in the vertical direction changes, and luminance flicker (so-called fluorescent lamp flicker) occurs in the time direction. This is because a fluorescent lamp as a light source blinks at twice the frequency of a commercial AC power supply, whereas a rolling shutter cannot expose all pixels at the same timing unlike a global shutter.

  When taking a still image, the exposure timing of each pixel can be made uniform by using a mechanical shutter. However, this problem is particularly noticeable during moving image shooting because a mechanical shutter cannot be used.

  For example, let us consider a case where photographing is performed under illumination of a non-inverter type fluorescent lamp when the frequency of the commercial AC power supply is 60 Hz (Hertz). The non-inverter type fluorescent lamp means a fluorescent lamp that is directly lit by a commercial AC power source without using an inverter. In this case, the luminance change period of the fluorescent lamp as the light source is 1/120 seconds.

  In this case, as shown in FIG. 12, when the exposure time of each pixel is set to 1/60 seconds, luminance unevenness in the vertical direction does not occur. When an image sensor having a rolling shutter characteristic is used, the exposure timing differs for each horizontal line, but the exposure time of each pixel is an integral multiple of the luminance change period of the light source (in this case, 1/120 seconds). This is because the exposure amount of each pixel becomes constant. For this reason, during normal photographing, the exposure time of each pixel is set to 1/60 seconds, for example.

In FIG. 12 and FIGS. 13 and 14 to be described later, the downward direction on the paper corresponds to the time advance direction. In FIG. 12, a period between timings t _{1 and} t ₂ represents a period in which pixels on the uppermost horizontal line of an image 200 are exposed, and a period between timings t _{2 and} t ₃ represents the horizontal line at the lowermost end of the image 200. It represents the period during which the pixels on the line are exposed. In addition, the exposure timing of each pixel is sequentially shifted even in the same horizontal line, but the timing difference is sufficiently short compared to the period of the luminance change of the fluorescent lamp, so the exposure timing between pixels on the same horizontal line is Can be considered the same.

By the way, in many cases, an imaging device is required to perform short-time exposure photography (photographing in a sports photography mode) in order to clearly capture a subject performing sports or the like. However, if the exposure time at the time of short exposure photography is 1/240 seconds, for example, as shown in FIG. 13, the exposure amount for each horizontal line is different, resulting in vertical luminance unevenness. In FIG. 13, a period between timings t _{4 and} t ₅ represents a period during which pixels on the uppermost horizontal line of an image 201 are exposed, and a period between timings t _{6 and} t ₇ represents the lowest end of the image 201. Represents a period during which pixels on the horizontal line are exposed.

  In addition, the imaging apparatus is often required to have a high-speed shooting function. When the frame rate during normal shooting is 60 fps (frame per second), for example, in high-speed shooting, the frame rate is 300 fps and the exposure time of each pixel is 1/300 seconds.

FIG. 14 shows a photographed image when high-speed photographing is performed under illumination of a non-inverter fluorescent lamp when the frequency of the commercial AC power supply is 50 Hz. As shown in FIG. 14, luminance unevenness in the vertical direction occurs in each photographed image. Furthermore, since the situation of luminance unevenness between the captured images changes with time, luminance flicker occurs in the time direction. In FIG. 14, a period between timings t _{8 and} t ₉ represents a period during which pixels on the uppermost horizontal line of an image 202 are exposed, and a period between timings t _{9 and} t ₁₀ represents the lowest end of the image 202. Represents a period during which pixels on the horizontal line are exposed.

JP 2005-184413 A

  Although there is a technique for eliminating luminance unevenness and temporal luminance flicker in each image by image correction processing, in order to determine whether or not to execute this image correction processing, first, the light source that illuminates the subject is a non-inverter type It is necessary to determine whether it is a fluorescent lamp. When it is determined that the light source is a non-inverter type fluorescent lamp, it is necessary to determine the frequency of the commercial AC power source that drives the fluorescent lamp (in other words, the frequency of the luminance change of the light source). Therefore, it is important to develop a technique for realizing the light source state detection necessary for these determinations by a simple method.

  Therefore, an object of the present invention is to provide a light source state detection device and a light source state detection method for detecting the state of a light source by a simple method, and an imaging device using them.

  In order to achieve the above object, a light source state detection device according to the present invention is a light source state detection device that detects a light source state for an image pickup device that performs photographing while varying exposure timing between different horizontal lines. In the predetermined detection period, the first image is photographed by the first exposure time and the second image is photographed by the second exposure time different from the first exposure time. The first and second images And detecting means for detecting whether or not the luminance of the light source periodically changes.

  Specifically, for example, the detecting unit compares the luminance distribution in the vertical direction of the first image with the luminance distribution in the vertical direction of the second image, so that the luminance of the light source changes periodically. Detect whether or not.

  More specifically, for example, the detection means adjusts the brightness of the first and second images in a direction in which a brightness difference based on a difference between the first exposure time and the second exposure time is eliminated, By comparing the vertical luminance distributions of the first and second images after the adjustment, it is detected whether or not the luminance of the light source is periodically changing.

  Further, for example, the light source state detection apparatus causes the imaging device to capture two images as the second image, and an exposure timing difference with respect to the same horizontal line between the two images is an integral multiple of a predetermined period. When the detection unit determines that the luminance of the light source is periodically changed, the detection unit further detects the frequency of the luminance change of the light source based on the two images.

  For example, the detection means detects the frequency of the luminance change of the light source by comparing the luminance distribution in the vertical direction between the two images.

  For example, the detection means detects whether the frequency is the reciprocal of the predetermined period based on the two images.

  In order to achieve the above object, an imaging apparatus according to the present invention includes the light source state detection apparatus and the imaging element.

  In addition, for example, the imaging apparatus further includes a storage unit that can store an image captured by the imaging device, and a motion detection unit that detects a motion of an image in the image captured by the imaging device, The light source state detection device can provide the detection period during moving image shooting in which images taken one after another are stored in the storage means one after another. In which period the period for detection is provided is set based on the detected movement.

  This makes it possible to detect the state of the light source even during moving image shooting. Further, by setting the detection period based on the motion of the image, it is possible to reduce the adverse effects caused by providing the detection period during moving image shooting.

  Further, for example, the imaging apparatus further includes a display unit, and the display unit displays the first image when the first image is captured during the detection period, while the second image is captured. When this is done, the first image taken immediately before the second image is displayed.

  For example, when the exposure time in the normal period different from the detection period matches the first exposure time, the user may feel discomfort when displaying the second image based on the second exposure time. In order to alleviate this, it is configured as described above.

  In order to achieve the above object, a light source state detection method according to the present invention includes a light source state detection method for detecting a light source state of an image pickup device that performs photographing while varying exposure timing between different horizontal lines. An image sensor is used to capture a first image with a first exposure time and a second image with a second exposure time different from the first exposure time within a predetermined detection period. It is characterized in that it is detected whether or not the luminance of the light source periodically changes based on two images.

  According to the present invention, the state of the light source of the image sensor can be detected by a simple method.

  Hereinafter, embodiments of the present invention will be specifically described with reference to the drawings. In each of the drawings to be referred to, the same part is denoted by the same reference numeral, and redundant description regarding the same part is omitted in principle. The first to fourth embodiments will be described later. First, matters that are common to each embodiment or items that are referred to in each embodiment will be described.

  FIG. 1 is an overall block diagram of an imaging apparatus 1 according to an embodiment of the present invention. The imaging device 1 is a digital video camera, for example. The imaging device 1 can shoot moving images and still images, and can also shoot still images simultaneously during moving image shooting.

  The imaging apparatus 1 includes an imaging unit 11, an AFE (Analog Front End) 12, a video signal processing unit 13, a microphone 14, an audio signal processing unit 15, a compression processing unit 16, and an SDRAM as an example of an internal memory. (Synchronous Dynamic Random Access Memory) 17, memory card (storage means) 18, decompression processing unit 19, video output circuit 20, audio output circuit 21, TG (timing generator) 22, CPU (Central Processing Unit) ) 23, a bus 24, a bus 25, an operation unit 26, a display unit 27, and a speaker 28. The operation unit 26 includes a recording button 26a, a shutter button 26b, an operation key 26c, and the like. Each part in the imaging apparatus 1 exchanges signals (data) between the parts via the bus 24 or 25.

  First, basic functions of the imaging device 1 and each part constituting the imaging device 1 will be described.

  The TG 22 generates a timing control signal for controlling the timing of each operation in the entire imaging apparatus 1, and provides the generated timing control signal to each unit in the imaging apparatus 1. Specifically, the timing control signal is given to the imaging unit 11, the video signal processing unit 13, the audio signal processing unit 15, the compression processing unit 16, the expansion processing unit 19, and the CPU 23. The timing control signal includes a vertical synchronization signal Vsync and a horizontal synchronization signal Hsync.

  The CPU 23 comprehensively controls the operation of each unit in the imaging apparatus 1. The operation unit 26 receives an operation by a user. The operation content given to the operation unit 26 is transmitted to the CPU 23. The SDRAM 17 functions as a frame memory. Each unit in the imaging apparatus 1 temporarily records various data (digital signals) in the SDRAM 17 during signal processing as necessary.

  The memory card 18 is an external recording medium, for example, an SD (Secure Digital) memory card. In this embodiment, the memory card 18 is illustrated as an external recording medium. However, the external recording medium is composed of one or a plurality of randomly accessible recording media (semiconductor memory, memory card, optical disk, magnetic disk, etc.). can do.

  FIG. 2 is an internal configuration diagram of the imaging unit 11 of FIG. By using a color filter or the like for the imaging unit 11, the imaging device 1 is configured to generate a color image by shooting. The imaging unit 11 includes an optical system 35 including a plurality of lenses including a zoom lens 30 and a focus lens 31, an aperture 32, an imaging element 33, and a driver 34. The driver 34 includes a motor or the like for realizing movement of the zoom lens 30 and focus lens 31 and adjustment of the opening amount of the diaphragm 32.

  Incident light from the subject enters the image sensor 33 through the zoom lens 30, the focus lens 31, and the diaphragm 32. The lens constituting the optical system 35 forms an optical image of the subject on the imaging surface (light receiving surface) of the imaging element 33. The TG 22 generates a drive pulse for driving the image sensor 33 in synchronization with the timing control signal, and applies the drive pulse to the image sensor 33.

  The image sensor 33 is, for example, an XY address scanning type CMOS (Complementary Metal Oxide Semiconductor) image sensor. A CMOS image sensor is formed by forming a plurality of pixels arranged two-dimensionally in a matrix, a vertical scanning circuit, a horizontal scanning circuit, a pixel signal output circuit, and the like on a semiconductor substrate capable of forming a CMOS. In the imaging element 33, an imaging surface is formed by a plurality of pixels arranged two-dimensionally, and the imaging surface has a plurality of horizontal lines and a plurality of vertical lines.

  The image sensor 33 has an electronic shutter function, and each pixel is exposed by a so-called rolling shutter. In the rolling shutter, the timing (time) at which each pixel on the imaging surface is exposed differs in the vertical direction for each horizontal line. That is, the exposure timing differs between different horizontal lines on the imaging surface. For this reason, it is necessary to consider vertical luminance unevenness and flicker under fluorescent lamp illumination (details will be described later).

  The image sensor 33 photoelectrically converts an optical image incident through the optical system 35 and the diaphragm 32, and sequentially outputs an electrical signal obtained by the photoelectric conversion to the subsequent AFE 12. More specifically, in each shooting, each pixel on the imaging surface stores a signal charge having a charge amount corresponding to the exposure time. Each pixel sequentially outputs an electrical signal having a magnitude proportional to the amount of stored signal charge to the subsequent AFE 12. When the optical images incident on the optical system 35 are the same and the aperture amount of the diaphragm 32 is the same, the magnitude (intensity) of the electrical signal from the image sensor 33 (each pixel) increases in proportion to the exposure time. To do.

  The driver 34 controls the optical system 35 based on a control signal from the CPU 23 and controls the zoom magnification and focal length of the optical system 35. Further, the driver 34 controls the opening amount (size of the opening) of the diaphragm 32 based on a control signal from the CPU 23. When the optical images incident on the optical system 35 are the same, the amount of incident light per unit time on the image sensor 33 increases as the aperture of the diaphragm 32 increases.

  The AFE 12 amplifies the analog signal output from the imaging unit 11 (image sensor 33), and converts the amplified analog signal into a digital signal. The AFE 12 sequentially outputs the digital signals to the video signal processing unit 13.

  The video signal processing unit 13 generates a video signal representing an image captured by the imaging unit 11 (hereinafter also referred to as “captured image”) based on the output signal of the AFE 12. The video signal is composed of a luminance signal Y representing the luminance of the photographed image and color difference signals U and V representing the color of the photographed image. The video signal generated by the video signal processing unit 13 is sent to the compression processing unit 16 and the video output circuit 20.

  FIG. 3 shows an internal block diagram of the video signal processing unit 13. The video signal processing unit 13 includes an AF evaluation value detection unit 41 that detects an AF evaluation value according to the contrast amount in the focus detection region in the captured image, and an AE evaluation that detects an AE evaluation value according to the brightness of the captured image. A value detection unit 42, a motion detection unit 43 that detects the movement of the image in the captured image, a light source state detection unit 44, and the like are included.

  Various signals generated by the video signal processing unit 13 including the AF evaluation value and the like are transmitted to the CPU 23 as necessary. The CPU 23 adjusts the position of the focus lens 31 via the driver 34 in FIG. 2 according to the AF evaluation value, thereby forming an optical image of the subject on the imaging surface of the image sensor 33. Further, the CPU 23 adjusts the opening amount of the diaphragm 32 (and the amplification degree of signal amplification in the AFE 12 as necessary) via the driver 34 in FIG. Control). In addition, camera shake correction or the like is performed based on the motion of the image detected by the motion detection unit 43. The function of the light source state detection unit 44 that is a characteristic part of the imaging device 1 will be described later.

  In FIG. 1, a microphone 14 converts audio (sound) given from the outside into an analog electric signal and outputs it. The audio signal processing unit 15 converts an electrical signal (audio analog signal) output from the microphone 14 into a digital signal. The digital signal obtained by this conversion is sent to the compression processing unit 16 as an audio signal representing the audio input to the microphone 14.

  The compression processing unit 16 compresses the video signal from the video signal processing unit 13 using a predetermined compression method. During video or still image shooting, the compressed video signal is sent to the memory card 18. The compression processing unit 16 compresses the audio signal from the audio signal processing unit 15 using a predetermined compression method. At the time of moving image shooting, the video signal from the video signal processing unit 13 and the audio signal from the audio signal processing unit 15 are compressed while being temporally related to each other by the compression processing unit 16, and after compression, they are stored in the memory card 18. Sent to. A so-called thumbnail image is also compressed by the compression processing unit 16.

  The recording button 26a is a push button switch for the user to instruct the start and end of shooting of a moving image (moving image), and the shutter button 26b is a button for the user to instruct shooting of a still image (still image). It is a button switch. Moving image shooting is started and ended according to the operation on the recording button 26a, and still image shooting is performed according to the operation on the shutter button 26b. One captured image (frame image) is obtained in one frame. The length of each frame is 1/60 seconds, for example. In this case, a group of frame images (stream images) sequentially acquired at a frame period of 1/60 seconds constitutes a moving image.

  The operation mode of the imaging apparatus 1 includes a shooting mode capable of shooting moving images and still images, and a playback mode for reproducing and displaying moving images or still images stored in the memory card 18 on the display unit 27. Transition between the modes is performed according to the operation on the operation key 26c.

  When the user presses the recording button 26a in the shooting mode, under the control of the CPU 23, the video signal of each frame after the pressing and the corresponding audio signal are sequentially recorded on the memory card 18 via the compression processing unit 16. Is done. That is, the captured image of each frame is sequentially stored in the memory card 18 together with the audio signal. When the user presses the recording button 26a again after starting the moving image shooting, the moving image shooting ends. That is, recording of the video signal and the audio signal to the memory card 18 is completed, and shooting of one moving image is completed.

  In the shooting mode, when the user presses the shutter button 26b, a still image is shot. Specifically, under the control of the CPU 23, the video signal of one frame after the depression is recorded on the memory card 18 via the compression processing unit 16 as a video signal representing a still image.

  When the user performs a predetermined operation on the operation key 26 c in the reproduction mode, a compressed video signal representing a moving image or a still image recorded on the memory card 18 is sent to the expansion processing unit 19. The decompression processing unit 19 decompresses the received video signal and sends it to the video output circuit 20. In the shooting mode, the generation of the video signal by the video signal processing 13 is normally performed regardless of whether or not a moving image or a still image is being shot, and the video signal is sent to the video output circuit 20. It is done.

  The video output circuit 20 converts a given digital video signal into a video signal (for example, an analog video signal) in a format that can be displayed on the display unit 27 and outputs the video signal. The display unit 27 is a display device such as a liquid crystal display, and displays an image corresponding to the video signal output from the video output circuit 20.

  When a moving image is reproduced in the reproduction mode, a compressed audio signal corresponding to the moving image recorded on the memory card 18 is also sent to the expansion processing unit 19. The decompression processing unit 19 decompresses the received audio signal and sends it to the audio output circuit 21. The audio output circuit 21 converts a given digital audio signal into an audio signal in a format that can be output by the speaker 28 (for example, an analog audio signal) and outputs the audio signal to the speaker 28. The speaker 28 outputs the sound signal from the sound output circuit 21 to the outside as sound (sound).

  Next, the function of the light source state detection unit 44 (see FIG. 3) that is a characteristic part of the imaging device 1 will be described. Whether the light source state detection unit 44 periodically changes the luminance of the light source that illuminates the imaging region (subject within the imaging region) of the imaging unit 11 with reference to the plurality of detection images captured by the imaging unit 11. Detect whether or not. More specifically, it is detected whether or not the light source is a non-inverter type fluorescent lamp (in other words, whether or not photographing is being performed under illumination of the fluorescent lamp).

  The non-inverter type fluorescent lamp means a fluorescent lamp that is directly lit by a commercial AC power source without using an inverter. The luminance of the non-inverter fluorescent lamp periodically changes at a frequency twice as high as the frequency of the commercial AC power source that drives the fluorescent lamp. For example, when the frequency of the commercial AC power supply is 60 Hz (Hertz), the luminance change period of the fluorescent lamp is 1/120 second. Hereinafter, the light source that illuminates the imaging region of the imaging unit 11 is simply referred to as “light source”, and when simply referred to as “fluorescent lamp”, it refers to a non-inverter type fluorescent lamp.

  When the light source state detection unit 44 determines that the luminance of the light source is periodically changing, the light source state detection unit 44 further detects the frequency of the luminance change of the light source based on the plurality of detection images. More specifically, when it is determined that the light source is a fluorescent lamp, the frequency of the commercial AC power source that drives the fluorescent lamp (in other words, the frequency of the luminance change of the fluorescent lamp) is detected. In Japan, since the frequency of the commercial AC power supply is fixed to 50 Hz or 60 Hz, the light source state detection unit 44 detects whether the frequency of the commercial AC power supply that drives the fluorescent lamp is 50 Hz or 60 Hz.

Hereinafter, unless otherwise specified, it is assumed that the frame rate of the imaging apparatus 1 is 60 fps (frame per second) and the exposure time of each pixel is 1/60 second (therefore, the length of each frame is 1/60). Second). Also,
Exposure in which the exposure time of each pixel is 1/60 second is called normal exposure,
Exposure in which the exposure time of each pixel is 1/240 seconds is called short-time exposure. And
A captured image obtained by normal exposure is called a normal exposure image.
A captured image obtained by short-time exposure is called a short-time exposure image.

  In order to realize the detection process by the light source state detection unit 44, a detection period is provided. In the detection period, shooting of a normal exposure image by normal exposure and shooting of a short exposure image by short exposure are alternately performed. More specifically, in the detection period, the first detection image is captured by the normal exposure, the second detection image is captured by the short exposure, the third detection image is captured by the normal exposure, and the short time. Shooting of the fourth detection image by exposure is performed in this order.

FIG. 4 shows the state of each detection image taken under fluorescent lamp illumination driven at 60 Hz. Reference numeral 101 represents the luminance of a fluorescent lamp as a light source. The downward direction on the paper corresponds to the time advance direction. In FIG. 4, images C _A1 , C _A2 , C _A3, and C _A4 represent first, second, third, and fourth detection images, respectively. For convenience of illustration, in FIG. 4 and FIGS. 5 and 7A and 7B described later, the horizontal size of each detection image is shown small.

4, between the timing T ₁ -T ₂ represents a period during which a pixel on the horizontal line of the uppermost end of the image C _A1 is exposed, between the timing T ₂ -T ₄ is horizontal lowermost end of the image C _A1 It represents the period during which the pixels on the line are exposed. In FIG. 4, a period between timings T _{3 and} T ₄ represents a period during which pixels on the horizontal line at the uppermost end of the image C _A2 are exposed, and a period between timings T _{5 and} T ₆ is the center of the image C _A2 . This represents a period during which pixels on the horizontal line are exposed. Between timings T _{7 and} T ₈ represents a period during which pixels on the lowermost horizontal line of the image C _A2 are exposed.

In FIG. 4, a period between timings T _{4 and} T ₈ represents a period during which pixels on the uppermost horizontal line of the image C _A3 are exposed, and a period between timings T _{8 and} T ₁₀ is the lowest end of the image C _A3 . Represents a period during which pixels on the horizontal line are exposed. In FIG. 4, a period between timings T _{9 and} T ₁₀ represents a period during which pixels on the uppermost horizontal line of the image C _A4 are exposed, and a period between timings T _{11 and} T ₁₂ is the center of the image C _A4 . This represents a period during which pixels on the horizontal line are exposed. Between timings T _{13 and} T ₁₄ represents a period during which pixels on the lowermost horizontal line of the image _CA4 are exposed.

Then, in accordance with the definition of the normal exposure and short exposure described above, between time T ₁ -T _2, between T ₂ -T _4, T ₄ between -T ₈ and T the length of time between ₈ -T ₁₀ is 1 / is 60 seconds, between time T ₃ -T _4, T between ₅ -T _6, between T ₇ -T _8, between T ₉ -T _10, T ₁₁ between -T ₁₂ and T ₁₃ times length between -T ₁₄ The length is 1/240 seconds.

FIG. 5 shows a state of each detection image photographed under fluorescent lamp illumination driven at 50 Hz. Reference numeral 102 represents the luminance of a fluorescent lamp as a light source. The downward direction on the paper corresponds to the time advance direction. In FIG. 5, images C _B1 , C _B2 , C _B3, and C _B4 represent the first, second, third, and fourth detection images, respectively.

  The light source state detection unit 44 captures each captured detection image by dividing it in a plurality of regions in the vertical direction (up and down direction). FIG. 6 shows this division. Each detection image is divided into six equal parts in the vertical direction as shown in FIG. 6, for example. The six regions obtained by this division are divided into divided regions AR [1], AR [2], AR [3], AR [4], AR [5] and AR [6] from the upper end side in the vertical direction. Call. The number of divisions 6 is an example, and the number of divisions may be other than six.

FIG. 7A shows images C _A1 , C _A2 , C _A3, and C _A4 subjected to the above-described division, and FIG. 7B illustrates images C _B1 , C _B2 , C _B3 and C _B4 are shown.

The light source state detection unit 44 integrates luminance values (values of luminance signals) for each of the divided areas AR [1] to AR [6] with respect to each detection image. An integrated value of luminance values of pixels included in the divided area AR [i] of the m-th detection image is represented by Y _m [i]. Here, i takes each integer in the range of 1-6, and m takes each integer in the range of 1-4. The luminance value of each pixel represents the value of the luminance signal Y of each pixel generated in the video signal processing unit 13, and for a certain pixel, the luminance of the pixel increases as the luminance value of the pixel increases. And

However, since the exposure time of the second (and fourth) detection image is ¼ that of the first detection image, the luminance of each pixel of the second (and fourth) detection image The value is multiplied by 4, and Y _m [i] is calculated using the luminance value after the multiplication.

First, the light source state detector 44 periodically changes the luminance of the light source by comparing the vertical luminance distribution of the first detection image with the vertical luminance distribution of the second detection image. Whether or not the light source is a fluorescent lamp is detected. Specifically, for example, the light source state detection unit 44 calculates the difference (Y ₁ [i] −Y ₂ [i]) of the luminance integrated value for each divided region between the first and second detection images. Calculate the absolute value. Then, to calculate the sum of the six absolute values calculated for each of the divided regions as the evaluation value E _A. The evaluation value E _A is calculated according to the following formula (1). FIG. 8 shows a block diagram for calculating an evaluation value E _A (or an evaluation value E _B described later). When comparing the vertical luminance distribution between the first and second detection images, the luminance difference between the two detection images based on the difference in the exposure time is excluded as described above (by the 4-times processing). Is done.

As understood with reference to FIGS. 7A and 7B, under fluorescent lamp illumination, the vertical luminance distribution of the first detection image and the vertical luminance of the second detection image because the distribution is significantly different, the evaluation value E _A is relatively large. This is the same whether the frequency of the commercial AC power supply is 60 Hz or 50 Hz. On the other hand, the light source if there like the sun rather than fluorescent lamps, the brightness distribution in the vertical direction of the first and second detection image is substantially the same, the evaluation value E _A is relatively small.

Using this characteristic, the light source state detection unit 44, when the evaluation value E _A is greater than a predetermined threshold TH1, it is determined that the luminance of the light source is changed periodically (the light source is a fluorescent lamp) when the evaluation value E _a is less than the threshold value TH1, the luminance of the light source is not changed periodically (the light source is not a fluorescent lamp) is determined and.

When it is determined that the luminance of the light source is periodically changed, the light source state detection unit 44 further performs a vertical luminance distribution of the second detection image and a vertical luminance distribution of the fourth detection image. Is detected, the frequency of the luminance change of the light source (in other words, the frequency of the commercial AC power source that drives the fluorescent lamp) is detected. Specifically, for example, the light source state detection unit 44 calculates a difference (Y ₂ [i] −Y ₄ [i]) of luminance integrated values for each divided region between the second and fourth detection images. Calculate the absolute value. Then, to calculate the sum of the six absolute values calculated for each of the divided regions as the evaluation value E _B. The evaluation value E _B is calculated according to the following formula (2).

As can be understood with reference to FIG. 4 and FIG. 7 (a), the luminance distribution in the vertical direction between the second and fourth detection images is similar under fluorescent lamp illumination driven at 60 Hz. The evaluation value E _B is relatively small. This is because the difference in exposure timing with respect to the same horizontal line between the second and fourth detection images is an integral multiple of the luminance change period of the light source (in this case, 1/120 seconds). To do. For example, between the second and fourth detection images, the exposure timing difference with respect to the uppermost horizontal line corresponds to the time difference between the timings T ₃ -T ₉ (or T ₄ -T ₁₀ ) in FIG. 1/30 seconds.

On the other hand, as can be understood by referring to FIGS. 5 and 7B, the luminance distribution in the vertical direction between the second and fourth detection images is greatly different under fluorescent lamp driving at 50 Hz. Therefore, the evaluation value E _B is relatively large.

Using this characteristic, the light source state detection unit 44, when the evaluation value E _B is smaller than a predetermined threshold TH2, the frequency of the luminance change of the light source is determined to be 120 Hz. In other words, it is determined that the frequency of the commercial AC power source that drives the fluorescent lamp is 60 Hz. On the other hand, when the evaluation value E _B is larger than the threshold value TH2, it is determined that the luminance change frequency of the light source is 100 Hz. In other words, it is determined that the frequency of the commercial AC power source that drives the fluorescent lamp is 50 Hz.

The case where the frame rate is 60 fps has been described as an example, and the detection process by the light source state detection unit 44 has been described. However, the detection process is similarly performed when the frame rate is 50 fps. However, when the frame rate is 50 fps, the exposure time for normal exposure corresponding to the first and third detection images is 1/50 second, and the short-time exposure corresponding to the second and fourth detection images. The exposure time is 1/200 second. When the frame rate is 50 fps, when the evaluation value E _B is smaller than the predetermined threshold value TH2, it is determined that the frequency of luminance change of the light source is 100 Hz, and when the evaluation value E _B is larger than the threshold value TH2, It is determined that the luminance change frequency of the light source is 120 Hz. The determination based on the evaluation value E _A is the same as when the frame rate is 60 fps.

  Hereinafter, each embodiment using the detection function of the light source state detection unit 44 will be described. Each embodiment can be arbitrarily combined as long as there is no contradiction, and the matters described in one embodiment can be applied to other embodiments as long as there is no contradiction. The processing for detecting whether or not the luminance of the light source is periodically changed and the processing for detecting the frequency of the luminance change of the light source are collectively referred to as “light source state detection processing”.

<< First Example >>
First, the first embodiment will be described. FIG. 9 is a flowchart illustrating an operation procedure of the imaging apparatus 1 according to the first embodiment. In the first embodiment, it is assumed that the frame rate is always 60 fps. In addition, the shooting modes that can shoot moving images and still images include a normal shooting mode and a sports shooting mode. The exposure time in the normal shooting mode is 1/60 seconds. In principle, the exposure time in the sports shooting mode is 1/240 seconds. Transition between the modes is performed in response to an operation on the operation key 26c.

  First, when the mode is changed to the photographing mode (or when the image pickup apparatus 1 is turned on), the exposure time is set to 1/60 seconds (step S1). The TG 22 sequentially generates and outputs a vertical synchronization signal at a period corresponding to the frame rate (that is, a period of 1/60 seconds). In step S2, it is confirmed whether a vertical synchronization signal is output from the TG 22 or not. A vertical synchronization signal is generated and output at the start of each frame. If the vertical synchronization signal is output from the TG 22, the process proceeds to step S3. If not output, the process of step S2 is repeated.

  In step S3, it is determined whether recording is currently being performed (moving image recording). If the recording is in progress, the process proceeds to step S12. If the recording is not in progress, the process proceeds to step S4. Note that recording and movie shooting are synonymous, and recording and movie shooting are synonymous. During recording, the video signal of each frame and the corresponding audio signal are sequentially recorded on the memory card 18 via the compression processing unit 16.

  In step S4, it is confirmed whether recording is started by pressing the recording button 26a. If the recording is started, the recording is in progress (step S11), and the process proceeds to step S12. If recording has not started, the process proceeds from step S4 to step S5.

  The period during which steps S5 to S8 are executed via steps S2 to S4 corresponds to the above-described detection period. Steps S5 to S8 are executed within a so-called preview period. The preview period corresponds to a period of waiting for a recording start instruction based on the operation of the recording button 26c (instruction means) while displaying an image photographed by the image sensor 33 on the display unit 27. Recording of the captured image on the card 18 is not performed. It should be noted that a period other than the detection period in which the normal exposure images that are sequentially photographed are updated and displayed one after another at the frame rate can also be referred to as a normal period.

  In step S5, the exposure time is set so that the exposure time is alternately switched to 1/60 seconds or 1/240 seconds while referring to the previously set exposure time to realize the above-described light source state detection processing. Set. In subsequent step S <b> 6, image data representing each detection image photographed at the set exposure time is read from the image sensor 33.

  In step S7 following step S6, the image obtained by the reading in step S6 is displayed on the display unit 27. At this time, only the normal exposure image is displayed, and the short-time exposure image that may cause remarkable luminance unevenness is not displayed. That is, during the detection period, when the first detection image is captured, the first detection image is displayed, and when the second detection image is captured, the normal image captured immediately before is displayed. An exposure image, that is, a first detection image is displayed. The same applies to the third and fourth detection images.

  And in step S8 following step S6 and S7, the light source state detection part 44 implements the above-mentioned light source state detection process based on each image for a detection read in step S7. After the process of step S8, the exposure time is returned to 1/60 seconds, and the process proceeds to step S2. For example, when the light source state detection process in step S8 is executed only once and the light source state detection process is executed once, the execution of the processes in steps S5 and S8 is prohibited thereafter (steps S5 and S8 are skipped). ) Alternatively, the light source state detection process may be repeatedly executed. When the light source state detection process is repeatedly executed, branch processing (steps S13 and S14) of each step during recording is performed based on the detection result of the latest light source state detection process.

  In step S12, it is confirmed whether the end of recording has been instructed. If the end of recording has been instructed, the process proceeds to step S19. If the end of recording has not been instructed, the process proceeds to step S13. In step S13, based on the detection result in step S8, it is determined whether the light source is a fluorescent lamp. If the light source is a fluorescent lamp, the frequency of the commercial AC power source that drives the fluorescent lamp in step S14 is 60 Hz. And 50 Hz. If the frequency is 60 Hz, the minimum exposure time Lmin is limited to 1/120 second (step S16). If the frequency is 50 Hz, the minimum exposure time Lmin is limited to 1/100 second (step S16). S17), the process proceeds to step S18. On the other hand, if the light source is not a fluorescent lamp, the process proceeds from step S13 to step S15, and the process proceeds to step S18 without any limitation on the minimum exposure time Lmin.

  In step S18, it is confirmed whether the set shooting mode is the sports shooting mode. If the set shooting mode is the sports shooting mode, the process proceeds to step S21, and the exposure time is set to 1/240 seconds in principle. However, when the minimum exposure time Lmin is limited, the exposure time is set to the minimum exposure time Lmin. For example, if the minimum exposure time Lmin is limited to 1/120 seconds in step S16, the exposure time is set to 1/120 seconds in step S21. If the set shooting mode is not the sports shooting mode, the process proceeds from step S18 to step S20, and the exposure time is set to 1/60 seconds.

  After the exposure time is set in step S20 or S21, the process proceeds to step S23, and image data representing an image shot with the set exposure time is read from the image sensor 33. An image based on the image data read in step S23 is displayed on the display unit 27 (step S25) and recorded on the memory card 18 via the compression processing unit 16 (step S26). Return to.

  In step S19, which is shifted from step S12 when the end of recording is instructed, the exposure time is set to 1/60 seconds, and then image data representing an image photographed at the set exposure time is acquired by the image sensor. 33 is read out (step S22). Thereafter, the image is displayed on the display unit 27 (step S24), and the process returns to step S2.

  By processing as described above, in the sports shooting mode, in general, the exposure time is set to 1/240 seconds, and it is possible to clearly capture a subject that is playing sports or the like. Under fluorescent lamp illumination, vertical luminance unevenness and flicker resulting from shortening the exposure time become a problem, but the exposure time is determined based on the result of the light source state detection process, and the luminance change period (1/120 seconds). Or 1/100 second), the occurrence of uneven brightness and flicker in the vertical direction is effectively prevented. FIG. 10 shows the state of each image shot in the sports shooting mode under fluorescent lamp illumination driven at 60 Hz. Due to this limitation, the subject blur (moving subject blur) in each image becomes larger than when the exposure time is 1/240 seconds, but it is possible to obtain a clearer image than shooting with normal exposure. It is.

<< Second Example >>
Next, a second embodiment will be described. In the first embodiment, attention is paid to the sports shooting mode included in the shooting mode, but the shooting mode further includes a high-speed shooting mode. In the high-speed shooting mode, in principle, the frame rate is 300 fps and the exposure time is 1/300 seconds. However, even if the shooting mode is set to the high-speed shooting mode, the frame rate is set to 60 fps during the preview period. Transition between the modes is performed according to the operation on the operation key 26c.

  When the shooting mode is set to the high-speed shooting mode by a predetermined operation on the operation key 26c, the light source state detection unit 44 performs light source state detection processing during the preview period. If it is determined that the light source is not a fluorescent lamp, the frame rate and the exposure time are set to 300 fps and 1/300 seconds, respectively, as a rule. When it is determined that the light source is a fluorescent lamp and the frequency of the commercial AC power source that drives the fluorescent lamp is 60 Hz, the frame rate and the exposure time are set to 120 fps and 1/120 seconds, respectively. When it is determined that the light source is a fluorescent lamp and the frequency of the commercial AC power source that drives the fluorescent lamp is 50 Hz, the frame rate and the exposure time are set to 100 fps and 1/100 second, respectively.

  Under fluorescent lighting, vertical brightness unevenness and flicker resulting from increasing the frame rate is a problem, but it is limited to increasing the frame rate and reducing the exposure time based on the results of the light source state detection process. Therefore, the occurrence of uneven brightness and flicker in the vertical direction is effectively prevented. FIG. 11 shows the state of each image shot in the high-speed shooting mode under fluorescent lamp illumination driven at 60 Hz.

<< Third Example >>
Next, a third embodiment will be described. In the first or second embodiment, the case where the detection period is provided in the preview period has been described. However, it is also possible to provide a detection period during moving image shooting (of course, it is also possible to provide a detection period during the preview period and during moving image shooting). This method will be described as a third embodiment. Schematically, during video recording, a detection period is provided at a timing when image movement is small. Details will be described below.

  As described above, during moving image shooting, images sequentially captured by the image sensor 33 are sequentially recorded on the memory card 18 via the video signal processing unit 13 and the compression processing unit 16. At this time, a motion detection unit 43 (see FIG. 3) included in the video signal processing unit 13 detects image motion between captured images of adjacent frames.

  The motion detection unit 43 detects a motion vector of each captured image using a known or well-known image matching method (for example, a representative point matching method), for example, by comparing captured images of adjacent frames. The motion vector specifies the magnitude and direction of image motion between captured images of adjacent frames.

The CPU 23 monitors the magnitude of the motion vector during moving image shooting, and causes the light source state detection unit 44 to execute the light source state detection process when, for example, the following first detection processing execution condition is satisfied. Further, for example, the light source state detection unit 44 may be caused to execute the light source state detection process when both of the following first and second detection processing execution conditions are satisfied.
The first detection process execution condition is a condition that “a state in which the magnitude of the motion vector is equal to or less than a predetermined value continues during a predetermined time T _TH1 during moving image shooting”.
The second detection process execution condition is a condition that “a predetermined time T _TH2 or more has elapsed since the previous execution of the light source state detection process”.

  When the first detection process execution condition (and the second detection process execution condition) is satisfied, the CPU 23 controls the TG 22 and the like to cause the image sensor 33 to detect each of the above detections in order to execute the light source state detection process. Take a picture. The light source state detection unit 44 executes light source state detection processing based on each detection image. The obtained result of the latest light source state detection processing is used for setting the minimum exposure time Lmin in the sports shooting mode (that is, the processing in steps S13 to S17 in FIG. 9) and setting the frame rate and exposure time in the high-speed shooting mode. Is done. When the light source state detection process is completed, the exposure time (and frame rate) is restored.

  When a photographer moves during moving image shooting, the light source may change (for example, the light source may change from sunlight to a fluorescent lamp). It is not desirable to perform processing for setting the minimum exposure time Lmin based on the result of the past light source state detection processing while ignoring changes in the light source. Therefore, the light source state detection process is performed even during moving image shooting as described above. Although the exposure time of each detection image captured within the detection period may be different from that of the image that should be originally captured, a period in which the image movement is small is selected as the detection period depending on the first detection processing execution condition. Therefore, the substantial influence is extremely small.

  Note that the first and third detection images taken during moving image shooting are recorded in the memory card 18 and displayed on the display unit 27. On the other hand, it is preferable that the second and fourth detection images taken during moving image shooting are not output to the memory card 18 and the display unit 27. In this case, the first detection image is recorded on the memory card 18 and displayed on the display unit 27 as the image corresponding to the frame in which the second detection image is captured, and the fourth detection image is captured. As an image corresponding to the frame, a third detection image is recorded on the memory card 18 and displayed on the display unit 27. Since a period in which the image motion is small is selected as the detection period, even if such processing is performed, the substantial influence is extremely small.

<< 4th Example >>
In the detection period, the first, second, third and fourth detection images are sequentially photographed, and the exposure time of the first and third detection images is set to 1/60 seconds (or 1/50 seconds) and Although the case where the exposure time of the second and fourth detection images is 1/240 seconds (or 1/200 seconds) is illustrated, this can be variously modified. An example of this modification will be described as a fourth embodiment.

First, the conditions necessary for detecting whether or not the luminance of the light source periodically changes based on the first and second detection images will be considered. This detection is hereinafter referred to as “fluorescent lamp detection”. In order to perform fluorescent lamp detection, the following first and second fluorescent lamp detection conditions must be satisfied, and fluorescent lamp detection can be performed as long as these conditions are satisfied.
The first fluorescent lamp detection condition is a condition that “the exposure time of the first detection image and the exposure time of the second detection image are different from each other”.
The second fluorescent light detection conditions, "exposure time of the exposure time of the first detection image and the second detection image are both not an integral multiple of the predetermined period T _P" is a condition that.

The above and the predetermined period T _P, a predetermined cycle is assumed as the period of the luminance change of the light source. For Japan, since the frequency of the commercial AC power source is 60Hz or 50 Hz, the predetermined period T _P is set to 1/120 seconds or 1/100 seconds. For example, when the frame rate is 60 fps, T _P is 1/120 seconds, and when the frame rate is 50 fps, T _P is 1/100 seconds. In the second fluorescent lamp detection condition, “integer” that is an integer multiple is a positive integer.

  If the exposure times of the first and second detection images are different from each other, the influence of the luminance change of the light source on the first and second detection images is different. Considering this, the first fluorescent lamp detection condition is determined. However, for example, if the exposure times of the first and second detection images are 1/60 seconds and 1/120 seconds, respectively, they are perpendicular to the first and second detection images under 60 Hz driven fluorescent lamp illumination. The luminance unevenness in the direction does not occur, and the fluorescent lamp cannot be detected. Therefore, the second fluorescent lamp detection condition is determined.

Next, a condition necessary for detecting the frequency (or period) of the luminance change of the light source based on the second and fourth detection images will be considered. This detection is hereinafter referred to as “frequency detection”. In order to perform frequency detection, the following first and second frequency detection conditions need to be satisfied, and as long as they are satisfied, it is possible to perform frequency detection.
First frequency detection condition is a condition that "between the second and fourth detection image, the exposure timing difference for the same horizontal line is that an integral multiple of the predetermined period T _P."
The second frequency detection condition is a condition that “the exposure time of the second detection image and the exposure time of the fourth detection image are the same”.

As described above, the predetermined period T _P is 1/120 seconds or 1/100 seconds. In the example shown in FIG. 4, for example, the exposure timing difference with respect to the horizontal line at the uppermost end corresponds to the time difference between timings T ₃ -T ₉ (or T ₄ -T ₁₀ ) between the second and fourth detection images. And it is 1/30 second. Since 1/30 seconds is four times 1/120 seconds, the first frequency detection condition is satisfied. In the first frequency detection condition, “integer” that is an integer multiple is a positive integer.

Assume second and fourth detection images that ignore the motion of the image and satisfy the first and second frequency detection conditions. In Under this assumption, for example, when the predetermined period T _P 1/120 seconds, under fluorescent lighting of 60Hz driving becomes a luminance distribution in the vertical direction of the second and fourth detection image is the same ( They are different under 50 Hz driven fluorescent lamp illumination (see FIG. 5). Frequency detection is performed using this characteristic. In Japan, when focusing on the fluorescent lamp, the frequency of the luminance change of the light source for a 120Hz or 100 Hz, whether the frequency of the luminance change of the light source is the reciprocal of the predetermined period T _P is detected by the frequency detection It can be said.

Although the minimum conditions for performing fluorescent lamp detection and frequency detection have been described, it is desirable to further consider the following matters. For example, the exposure time for the first and third detection images may be the same as the length of one frame (the reciprocal of the frame rate). The frame rate is approximately 60 fps or 50 fps. For example, the exposure time of the second and fourth detection images may be shorter than that of the first (and third) detection images. This is because when the exposure times of the second and fourth detection images are shorter, the luminance unevenness in the vertical direction due to the luminance change of the light source appears significantly in the second and fourth detection images. For example, the exposure time of the second and fourth detection images may be approximately 1/2 ⁿ of the exposure time of the first (and third) detection image.

  Moreover, although the example which performs fluorescent lamp detection based on the 1st and 2nd image for a detection image | photographed in the mutually adjacent flame | frame was mentioned above, the 1st and 2nd image for a detection for performing a fluorescent lamp detection Are not necessarily photographed in frames adjacent to each other. However, the first and second detection images for performing fluorescent lamp detection should be taken in temporally close frames in order to suppress the occurrence of false detection due to the movement of the subject. It is.

  In the examples shown in FIGS. 4 and 5, the second and fourth detection images are separated by two frames, but the second and fourth detection images for frequency detection are taken. May be performed in adjacent frames, or they may be photographed three frames or more apart. However, the second and fourth detection images for frequency detection should be taken in temporally close frames in order to suppress the occurrence of erroneous detection due to the movement of the subject. is there.

  The first, second, third and fourth detection images are taken in this order, the first and third detection images are the normal exposure image, and the second and fourth detection images are short-time. The example which makes an exposure image was mentioned above. That is, the example in which the normal exposure image and the short-time exposure image are alternately captured in the detection period has been described above. However, it is not always necessary to photograph the normal exposure image and the short exposure image alternately. For example, it is possible to shoot each detection image in the order of “first, third, second, and fourth detection images”, or omit to shoot the third detection image itself. Is possible.

<< Deformation, etc. >>
As modifications or annotations of the above-described embodiment, notes 1 to 4 are described below. The contents described in each comment can be arbitrarily combined as long as there is no contradiction.

[Note 1]
The specific numerical values shown in the above description are merely examples, and as a matter of course, they can be changed to various numerical values. Further, the “integration (or integration value)” described when calculating the evaluation values E _A and E _B may be “average (or average value)”.

[Note 2]
In Japan, the frequency of a commercial AC power supply is basically 60 Hz or 50 Hz, but the frequency usually includes a slight error (for example, several percent). The actual frame rate and exposure time also include some errors with respect to the design values. Therefore, the frequency, the period, the frame rate, and the exposure time described in this specification should be interpreted as a temporal concept including a certain degree of error.

[Note 3]
In addition, the imaging apparatus 1 in FIG. 1 can be realized by hardware or a combination of hardware and software. In particular, the function of performing fluorescent lamp detection and frequency detection can be realized by hardware, software, or a combination of hardware and software. The fluorescent lamp detection and the frequency detection are realized mainly by the light source state detection unit 44 (or the video signal processing unit 13) of FIG.

  When the imaging apparatus 1 is configured using software, a block diagram of a part realized by software represents a functional block diagram of the part. All or part of the function for realizing fluorescent lamp detection and frequency detection is described as a program, and the program is executed on a program execution device (for example, a computer) to realize all or part of the function. You may make it do.

[Note 4]
The light source state detection unit 44 (or video signal processing unit 13) in FIG. 3 includes a detection unit that performs fluorescent lamp detection and frequency detection, and functions as a light source state detection device. Since the control of the exposure time for acquiring each detection image is performed by the CPU 23 (and TG 22 and the like), it can be considered that the CPU 23 is included in the light source state detection device.

1 is an overall block diagram of an imaging apparatus according to an embodiment of the present invention. It is an internal block diagram of the imaging part of FIG. FIG. 2 is an internal block diagram of a video signal processing unit in FIG. 1. It is a figure which shows the mode of each image for a detection which concerns on embodiment of this invention and was image | photographed under the fluorescent lamp illumination driven at 60 Hz. It is a figure which shows the mode of each image for a detection which concerns on embodiment of this invention, and was image | photographed under the fluorescent lamp illumination driven at 50 Hz. It is a figure which shows the division area in each image for a detection defined in the light source state detection part of FIG. FIG. 6 is a diagram illustrating each detection image corresponding to FIG. 4 and each detection image corresponding to FIG. 5. It is a block diagram showing the evaluation value calculation process implemented in the light source state detection part of FIG. 2 is a flowchart illustrating an operation procedure of the imaging apparatus of FIG. 1 according to the first embodiment of the present invention. It is a figure which concerns on 1st Example of this invention and shows the mode of each image image | photographed in sports photography mode under the fluorescent lamp illumination driven at 60 Hz. It is a figure which concerns on 2nd Example of this invention and shows the mode of each image image | photographed in high-speed imaging mode under the fluorescent lamp illumination driven at 60 Hz. It is a figure for demonstrating the cause which the brightness | luminance nonuniformity (and flicker of a time direction) produces | generates the perpendicular | vertical direction in the image image | photographed under fluorescent lamp illumination according to a prior art. It is a figure for demonstrating the cause which the brightness | luminance nonuniformity (and flicker of a time direction) produces | generates the perpendicular | vertical direction in the image image | photographed under fluorescent lamp illumination according to a prior art. It is a figure for demonstrating the cause which the brightness | luminance nonuniformity (and flicker of a time direction) produces | generates the perpendicular | vertical direction in the image image | photographed under fluorescent lamp illumination according to a prior art.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Imaging device 11 Imaging part 12 AFE
DESCRIPTION OF SYMBOLS 13 Image signal processing part 18 Memory card 27 Display part 33 Image pick-up element 43 Motion detection part 44 Light source state detection part

Claims (10)

In a light source state detection device that detects a state of a light source for an image sensor that performs imaging while different exposure timings are set between different horizontal lines,
Using the imaging device, within a predetermined detection period, the first image is photographed by the first exposure time and the second image is photographed by a second exposure time different from the first exposure time,
A light source state detection apparatus comprising: detection means for detecting whether or not the luminance of the light source periodically changes based on the first and second images. The detection means detects whether or not the luminance of the light source is periodically changed by comparing the vertical luminance distribution of the first image with the vertical luminance distribution of the second image. The light source state detection apparatus according to claim 1. The detection means adjusts the brightness of the first and second images in a direction in which a brightness difference based on a difference between the first exposure time and the second exposure time is eliminated, and the first after the adjustment is adjusted. 3. The light source state detection device according to claim 2, wherein whether or not the luminance of the light source is periodically changed is detected by comparing the luminance distribution in the vertical direction of the second image. As the second image, the image pickup device takes two images,
Between the two images, the exposure timing difference for the same horizontal line is an integral multiple of a predetermined period,
The said detection means detects the frequency of the luminance change of the said light source further based on the said two images, when it is judged that the luminance of the said light source is changing periodically. The light source state detection device according to claim 3. The light source state detection apparatus according to claim 4, wherein the detection unit detects a frequency of a luminance change of the light source by comparing a luminance distribution in a vertical direction between the two images. The light source state detection apparatus according to claim 4 or 5, wherein the detection unit detects whether the frequency is an inverse number of the predetermined period based on the two images. The light source state detection device according to any one of claims 1 to 6,
An image pickup apparatus comprising the image pickup device. Storage means capable of storing an image captured by the image sensor;
A motion detection means for detecting the motion of the image in the image captured by the image sensor;
The light source state detection device can provide the detection period during moving image shooting in which images taken one after another are stored in the storage unit one after another.
8. The imaging apparatus according to claim 7, wherein the light source state detection device sets in which period during the moving image shooting the detection period is provided based on the detected movement. A display means,
The display means displays the first image when the first image is photographed during the detection period, and is photographed immediately before the second image when the second image is photographed. The imaging apparatus according to claim 7 or 8, wherein the first image is displayed. In a light source state detection method for detecting a state of a light source of an image pickup device that performs photographing while varying exposure timing between different horizontal lines,
Using the imaging device, within a predetermined detection period, the first image is photographed by the first exposure time and the second image is photographed by a second exposure time different from the first exposure time,
A light source state detection method, comprising: detecting whether or not the luminance of the light source periodically changes based on the first and second images.