JP2012094956A - Imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem of conventional techniques in which arithmetic processing for calculating a flickering frequency of a light source is complicate.SOLUTION: An imaging device of this invention comprises: an imaging element which outputs an image signal corresponding to incident light; a phase adjustment part which adjusts a phase of a frame period at the time of reading an image signal from an imaging element; an arithmetic part which obtains a frame-to-frame exposure from the image signal and composes the exposures of respective frames read with phases different from each other which are adjusted by the phase adjustment part; and a frequency calculation part which calculates the flickering frequency of the light source on the basis of the temporal change of the composed exposure.

Description

  The present invention relates to an imaging apparatus.

  Currently, an image pickup apparatus including a general XY address scanning type image pickup element employs a rolling shutter system that controls a charge accumulation time for each line by address designation.

  On the other hand, fluorescent lamps that are widely used as indoor light sources have horizontal stripe-like light and dark unevenness (hereinafter referred to as “flicker”) due to the influence of periodic luminance changes due to the flicker frequency (for example, a commercial power supply frequency of 50 Hz or 60 Hz). ) May occur. In view of this, there has been proposed an imaging apparatus that detects flicker frequency of a light source and suppresses flicker of a moving image to be shot (see, for example, Patent Document 1).

JP 2003-189129 A

  In the conventional technique, a plurality of flicker detection frames are set by dividing an image of one frame in the vertical scanning direction, and after extracting flicker components from luminance value data in each flicker detection frame that fluctuates over time, the light source The flicker frequency is calculated. For this reason, the imaging device of Patent Document 1 has a problem that the arithmetic processing is generally complicated.

  In view of the above problems, an object of the present invention is to provide an imaging apparatus capable of calculating the blinking frequency of a light source without complicating arithmetic processing.

  An imaging device according to the present invention includes an imaging device that outputs an image signal corresponding to incident light, a phase adjustment unit that adjusts a phase of a frame period when the image signal is read from the imaging device, and a frame from the image signal. A calculation unit that obtains an exposure amount in units and combines the exposure amounts of the respective frames that are read out in different phases that are phase-adjusted by the phase adjustment unit, and a temporal change in the exposure amount after the composition. And a frequency calculation unit for calculating the blinking frequency of the light source.

  In particular, the phase adjustment unit performs the phase adjustment at a cycle of an integer multiple of a cycle corresponding to a frequency of the greatest common divisor of a plurality of frequencies predicted in advance as the blinking frequency of the light source.

  The frame period may be less than ½ of the maximum blinking period of the light source predicted in advance.

  Further, the phase adjustment unit performs phase adjustment of a frame period at the time of reading the image signal from the imaging device at least twice, and the calculation unit has at least two different phases adjusted in phase by the phase adjustment unit. The exposure amount of each read frame is synthesized.

  The imaging apparatus according to the present invention can accurately detect the blinking frequency of the light source without complicating the arithmetic processing.

1 is a block diagram illustrating an internal configuration example of an electronic camera 100 that is an embodiment of an imaging apparatus according to the present invention. It is a figure which shows an example of the conventional flicker detection method. It is a figure which shows the operation | movement used as the basis of the flicker detection method of the electronic camera 100 which concerns on this embodiment. It is a figure which shows an example of the flicker detection method of the electronic camera 100 which concerns on this embodiment. It is a figure which shows an example of the synthetic | combination method. It is a block diagram related to flicker detection of the electronic camera 100 according to the present embodiment. 4 is a flowchart illustrating an example of flicker detection processing of the electronic camera 100. It is a flowchart which shows the modification of a flicker detection process. It is a flowchart which shows the modification of a flicker detection process.

  Hereinafter, embodiments of an imaging apparatus according to the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration example of an electronic camera 100 that is an embodiment of an imaging apparatus according to the present invention. The electronic camera 100 has a moving image shooting function and a live view (hereinafter referred to as LV) image display function for confirming the composition during shooting. It has a function of detecting flickering frequency of the light source at the start of moving image shooting or at the start of LV image display and removing flicker appearing in the shot image.

  In FIG. 1, an electronic camera 100 includes a photographing lens 101, an image sensor 102, a timing generator (hereinafter referred to as “TG”) 103, and an analog front end unit (hereinafter referred to as “AFE”) 104. An image processing unit 105, a RAM (Random Access Memory) 106, a ROM (Read Only Memory) 107, a display monitor 108, a recording interface unit (hereinafter referred to as “recording I / F unit”) 109, and An operation unit 110, a CPU (Central Processing Unit) 111, and a bus 112 are provided.

  Among these, the AFE 104, the image processing unit 105, the RAM 106, the ROM 107, the display monitor 108, the recording I / F unit 109, and the CPU 111 are connected to each other via the bus 112, and input / output of data between the blocks via the bus 112. Is done.

  The taking lens 101 includes a plurality of lens groups including a focus lens and a zoom lens. These lenses are driven by a lens driving device (not shown) and controlled by the CPU 111.

  As an example, the imaging element 102 has a plurality of pixels arranged in a two-dimensional shape (M rows and N columns: (M and N are integers)) on an imaging surface, and an arbitrary line (for example, m-th row: (m is 1 to M) This is an XY addressing type CMOS (Complementary Metal-Oxide Semiconductor) type image pickup device that can read signals by designating an integer). In moving image shooting or LV image display, a rolling shutter system is used in which the charge accumulation time is controlled in order for each line by address designation.

  The TG 103 outputs a drive signal to the image sensor 102 and the AFE 104 according to an instruction from the CPU 111, and outputs a synchronization signal for reading an image signal from the image sensor 102. For example, a timing signal for resetting the charge of the pixel and a timing signal for reading the charge accumulated in the pixel are output for each line in order to perform imaging with the rolling shutter by the image sensor 102. Alternatively, a vertical synchronization signal that gives a frame period is output. In particular, in the present embodiment, the TG 103 adjusts the phase of the vertical synchronization signal with respect to the frame period and outputs it to the image sensor 102 according to a command from the CPU 111.

  The AFE 104 is an analog front end circuit that performs gain adjustment, A / D conversion, and the like of an image signal generated by the image sensor 102. The digital image signal output from the AFE 104 is temporarily stored in the RAM 106.

  The image processing unit 105 reads the image signal recorded in the RAM 106 and performs various image processing (gradation conversion processing, contour enhancement processing, white balance processing, etc.).

  The ROM 107 uses a non-volatile flash memory that stores a program for controlling the electronic camera 100 in advance. For example, the ROM 107 stores in advance information on designated addresses when thinning and reading from the image sensor 102 under the control of the CPU 111. In addition, parameters necessary for the operation of the electronic camera 100 are stored in advance. These parameters can be appropriately changed by the user through an operation menu of the electronic camera 100 or the like.

  As the display monitor 108, for example, a liquid crystal monitor is used, and the CPU 111 displays a captured image stored in the RAM 106, an operation menu of the electronic camera 100, and the like.

  The recording I / F unit 109 includes a connector and an interface circuit for connecting a detachable recording medium 113. Then, in response to an instruction from the CPU 111, the recording medium 113 connected to the connector is accessed to record or read out a still image or a moving image. For example, a nonvolatile memory card is used as the recording medium 113.

  The operation unit 110 includes, for example, a command dial for command selection, a power button, a release button, and the like. The operation unit 110 receives an instruction input for operating the electronic camera 100. Further, the operation unit 110 receives selection of, for example, a moving image shooting mode via a command dial. Here, when the moving image shooting mode is selected, the CPU 111 executes a flicker detection program. When the still image shooting mode is selected, the CPU 111 executes a flicker detection program before displaying an LV image for confirming the shooting composition on the display monitor 108.

The CPU 111 is a processor that performs overall control of the electronic camera 100. The CPU 111 controls each unit of the electronic camera 100 by executing a program stored in advance in the ROM 107. For example, the CPU 111 reads an image signal from the image sensor 102 by using a rolling shutter system that controls the charge accumulation time for each line by XY addressing. In particular, in the present embodiment, the CPU 111 performs processing for performing flicker detection.
[Flicker detection method]
First, before describing the flicker detection method of the electronic camera 100 according to the present embodiment, an example of a conventional flicker detection method will be described as a comparative example.

  FIG. 2 is a diagram illustrating an example of flicker detection of a CMOS image sensor that is conventionally performed. FIG. 2A is a diagram schematically showing a change in luminance of the fluorescent lamp with a dotted line 151. FIG. 2B is a timing chart of the vertical synchronization signal indicating the start of reading one frame image. FIG. 2C shows an image 155 and an image 156 in the case where the image is taken by the rolling shutter method, and is an exaggerated drawing of the flicker generated due to the blinking frequency under the fluorescent lamp.

  Here, for example, the fluorescent lamp used as the light source repeats the luminance change twice in one cycle of the power supply frequency (commercial power supply frequency) to be used, so that the blinking frequency of the light source is twice the commercial power supply frequency. In the case of FIG. 2 (a), the luminance change of a fluorescent lamp having a commercial power supply frequency of 60 Hz is shown, and the blinking cycle of the light source is 1/120 seconds. Here, assuming that the frame rate of an image to be captured for flicker detection is 30 fps, which is the same as that for normal moving image capturing, vertical synchronization signals 152, 153, and 154 that give a frame period are 1/30 as shown in FIG. It is output every second, and the light source blinks four times during one frame period.

  Here, when shooting with the rolling shutter method, the exposure time (charge accumulation time) of the rolling shutter corresponds to n times the blinking cycle (1/120 seconds) of the light source (n is an integer, except 0). If the period is not a period, the exposure amount for each line of the image sensor 102 is different, and horizontal stripe flicker as shown in an image 155 or an image 156 in FIG. In FIG. 2C, for easy understanding, the image frame read from the vertical synchronization signal 152 during the period of the vertical synchronization signal 153 is read out during the period of the image 155 and the vertical synchronization signal 153 during the period of the vertical synchronization signal 154. The image frame is drawn as if it were an image 156, but strictly speaking, it is an image frame acquired in each previous frame. For example, the image 155 is an image acquired before the vertical synchronization signal 152, and the image 156 is an image acquired during the period from the vertical synchronization signal 152 to the vertical synchronization signal 153.

  As a conventional flicker detection method for a CMOS image sensor, a technique for calculating the flicker frequency of a fluorescent lamp based on horizontal stripes appearing in one screen is considered. However, it is necessary to detect a change in brightness in one screen. However, there is a problem that the arithmetic processing becomes complicated. Therefore, in the electronic camera 100 according to the present embodiment, in order to simplify the arithmetic processing, not the line flicker in which the brightness changes within one frame but the surface in which the brightness changes for each frame as shown in FIG. Flicker is generated, and the flicker frequency of the light source is accurately obtained from the surface flicker.

  Next, a method of generating surface flicker that is a base for flicker detection used in the electronic camera 100 according to the present embodiment will be described with reference to FIG. The electronic camera 100 according to the present embodiment generates a surface flicker whose brightness changes for each frame by shooting at a high-speed frame rate shorter than the blinking cycle of the light source. Then, the flicker frequency of the light source is easily obtained from the fluctuation period of the surface flicker.

  FIG. 3A is a diagram showing a change in luminance of the same light source as that in FIG. 2A, and is a diagram schematically showing a blinking cycle of a luminance change of a fluorescent lamp with a commercial power supply of 60 Hz by a dotted line 151. FIG. 3B is a timing chart showing vertical synchronization signals 161, 162, and 163 that give the frame period of an image captured at a high frame rate (for example, 600 fps) on the same time axis as the blinking period of FIG. is there. FIG. 3C is a diagram showing an example of an image frame shot at each light source blinking cycle, and five 600 fps frames are shot at every light source blinking cycle (every 1/120 second). Here, each image frame shown in FIG. 3C is shot by, for example, a rolling shutter method during the period of the vertical synchronization signal 161 and the vertical synchronization signal 162, and the luminance of the light source slightly changes during this shooting period. Although the exposure amount for each line differs slightly, for example, by averaging for each frame, an image frame with a uniform exposure amount can be obtained as shown in FIG. In FIG. 3C, as in FIG. 2C, the image frame 157 is drawn corresponding to the period from the vertical synchronization signal 161 to the vertical synchronization signal 162 for easy understanding. The image 157 is an image frame obtained after the vertical synchronization signal 162. Similarly, the image frame 158 is an image frame obtained after the vertical synchronization signal 163.

  In addition, since shooting is performed at a frame rate faster than the blinking cycle of the light source, as shown in FIG. 3C, it can be observed as a surface flicker whose brightness changes for each frame according to the luminance change of the light source. Then, the blinking cycle of the light source can be easily obtained from the fluctuation cycle of the surface flicker. For example, if the peak period of the exposure amount of the frame is detected, the peak period becomes the blinking period of the light source.

  In this way, by shooting at a frame rate faster than the blinking cycle of the light source and converting it to surface flicker, the blinking cycle of the light source can be obtained without performing complicated processing or calculation. Since the accuracy of flicker detection is limited by the number of frames per hit, it is preferable to increase the frame rate as much as possible. However, if the frame rate is increased, the exposure time is shortened, so that it is difficult to discriminate flicker, and the burden on hardware increases. Therefore, in the electronic camera 100 according to the present embodiment, even when the frame rate is about 600 fps shown in FIG. 3, detection accuracy equivalent to that when shooting at a frame rate of 1000 fps or more and detecting flicker is obtained.

  Next, a flicker detection method of the electronic camera 100 according to the present embodiment using the flicker detection method described in FIG. 3 as a base will be described.

  In the electronic camera 100 according to the present embodiment, the output of the image frame of the surface flicker acquired by adjusting the phase of the frame period and different phases is synthesized on the same time axis in synchronism with the blinking period of the light source. Therefore, the detection accuracy of the flicker frequency of the light source can be improved without requiring a very fast frame rate and without performing complicated calculations.

  FIG. 4A is a diagram showing a change in luminance of the same light source as in FIGS. 3A and 2A, and schematically shows a blinking cycle of a luminance change of a fluorescent lamp with a commercial power supply of 60 Hz by a dotted line 151. FIG. FIG. 4B is a diagram corresponding to FIG. 3B, and vertical synchronization signals 161, 162, and 163 that give the frame period of an image captured at a high frame rate (for example, 600 fps), and 1 for the frame period. FIG. 5 is a timing chart depicting vertical synchronization signals 181, 182 and 183 corresponding to the blinking cycle of the light source in FIG. 4A when the phase of the vertical synchronization signal is shifted by / 2 frames. Here, FIG. 4B is different from FIG. 3B in that the phase of the vertical synchronization signal is delayed by a ½ frame period at a period of 50 ms. The reason why the period is 50 ms will be described later.

  FIG. 4C is a diagram showing an example of an image frame taken at every blinking cycle of the light source (every 1/120 second), similarly to FIG. 3C, and the blinking cycle of the light source (dotted line 151). ) 5 image frames are taken at 600 fps during one cycle. Further, in FIG. 4C, a frame that is shot with a delay of ½ frame period at a period of 50 ms is also displayed at a frame rate of 600 fps with a ½ frame period delayed from the blinking period 171 of the light source. Image frames are taken continuously. Similar to FIG. 3C, these image frames are also observed as surface flicker whose brightness changes for each frame.

  In FIG. 4C, as in FIG. 3C, the image 157 is drawn corresponding to the period from the vertical synchronization signal 161 to the vertical synchronization signal 162 for easy understanding, but strictly, The image 157 is an image frame obtained after the vertical synchronization signal 162. Similarly, the image 158 is an image frame obtained after the vertical synchronization signal 163, and the image 159 is an image frame obtained after the vertical synchronization signal 182.

  In the electronic camera 100 according to the present embodiment, the exposure amount of the frame before delaying the phase for ½ frame and the exposure amount of the frame after delay are combined at the timing of 50 ms, and the exposure amount after combining is combined. The flicker fluctuation period, that is, the flickering period of the light source is obtained from the temporal change of.

  Here, the reason why the phase of the vertical synchronization signal is adjusted at a period of 50 ms will be described. In the above description, a fluorescent lamp with a commercial power supply of 60 Hz has been described as the light source. However, since it is unclear whether the commercial power supply is actually 60 Hz, the electronic camera 100 according to the present embodiment is assumed. The blinking frequency of the light source is set in the ROM 107 in advance, and the blinking frequency of the light source is specified from the preset. The 50 ms described above is an example in the case of frequencies of 50 Hz and 60 Hz of a commercial power source assumed in Japan. Since the blinking frequency in the case of a commercial power supply of 50 Hz is 100 Hz and the blinking frequency in the case of a commercial power supply of 60 Hz is 120 Hz, the frequency 20 Hz of the greatest common divisor between 100 Hz and 120 Hz is the phase adjustment frequency. In this case, the phase adjustment period is 50 ms (1/20).

  Thus, the phase adjustment period is an integral multiple of the assumed blinking frequency of all of the plurality of light sources, so that the phase of the frame period can be adjusted in synchronization with the luminance change of the light sources. Thereby, the exposure amount of a series of image frames acquired by adjusting the phase of the frame period for each phase adjustment period can be synthesized on the same time axis.

  Next, an example in which exposure amounts of image frames having different frame periods are combined on the same time axis will be described with reference to FIG. FIG. 5 shows an example of a frame rate of 480 fps with respect to a light source blinking cycle of 120 Hz, and one light source blinking cycle includes four frames.

  FIG. 5A is a diagram showing the blinking pattern of the light source and the vertical synchronization signal on the same time axis when the phase of the vertical synchronization signal (the phase of the frame period) is not shifted. In FIG. 5A, a frame shot during the timing T1 and T2 of the vertical synchronization signal is an image frame IMG1, a frame shot during the timing T2 and T3 is an image frame IMG2, and so on from timing T3 to T9. In each frame period, the image frames IMG3 to IMG7 are photographed. In FIG. 5 (a), image frames acquired in the period of each vertical synchronization signal are acquired in the next frame period without being simplified as in FIGS. 3 (c) and 4 (c). It is shown strictly. Then, as described with reference to FIGS. 3C and 4C, these image frames are processed (for example, averaged) so that the exposure amount in the frame becomes uniform. In FIG. 5A, the bar graph drawn corresponding to each image frame indicates the output level L1 to the output level L7 corresponding to the exposure amount of each frame from the image frame IMG1 to the image frame IMG7.

  FIG. 5B shows the blinking pattern of the light source when the phase of the vertical synchronization signal (the phase of the frame period) is shifted by 1/2 phase, and the vertical synchronization signal of FIG. It is the figure shown on the same time axis in synchronization. In other words, the vertical synchronization signal at timing 11 in FIG. 5B has a timing relationship delayed by 1/2 phase with respect to the frame period with respect to the vertical synchronization signal at timing T1 in FIG.

  In FIG. 5B, a frame shot in the period of the timing T11 and T12 of the vertical synchronization signal is an image frame IMG11, a frame shot in the period of the timing T12 and T13 is an image frame IMG12, and so on. In each frame period, the image frame IMG13 to the image frame IMG17 are photographed. In FIG. 5B as well, image frames acquired in each vertical synchronization signal period are acquired in the next frame period without simplification as in FIGS. 3C and 4C. It is shown strictly. These image frames are also subjected to processing such as averaging so that the amount of exposure in the frames is uniform. Further, in FIG. 5A, the bar graph drawn corresponding to each image frame indicates the output level L11 to the output level L17 corresponding to the exposure amount of each frame from the image frame IMG11 to the image frame IMG17.

  FIG. 5C shows the output level of each image frame in FIG. 5A in which the phase of the vertical synchronizing signal is not shifted and each image in FIG. 5B in which the phase of the vertical synchronizing signal is shifted by 1/2 phase. It is the figure which showed the output level of the flame | frame on the same time axis synchronizing with the blinking period of a light source. That is, the timings T11 to T19 of the vertical synchronization signal in FIG. 5B are arranged on the same time axis with a ½ phase delay from the timing T1 to T9 of the vertical synchronization signal in FIG.

  In FIG. 5C, the output level L21 between the timings T2 and T12 is the output level L1 between the timings T2 and T3 in FIG. 5A and the output between the timings T11 and T12 in FIG. An output level obtained by combining the level L11. In the synthesis method in FIG. 5C, the average value of the output level L1 and the output level L11 is used, but for example, the sum of the output level L1 and the output level L11 may be used. Similarly, the output level L22 between timings T12 and T3 in FIG. 5C is an average value of the output level L1 in FIG. 5A and the output level L11 in FIG. Similarly, the average value of the output level of the image frame whose phase of the vertical synchronization signal is not shifted and the output level of the image frame whose phase is shifted by ½ phase is obtained on the same time axis, and the combined output level (exposure amount) ).

  In this way, the combined output levels L21 to L34 are obtained. The output level after the synthesis is substantially the same as increasing the frame rate by double because eight outputs are obtained per cycle of the blinking cycle of the light source. In the case of FIG. 5, the physical frame rate is 480 fps, but the frame rate for detecting flicker is 960 fps. Furthermore, since the output levels of the two image frames are calculated (sum, average, etc.), the contrast ratio between light and dark becomes large, and it becomes easy to detect a flicker change.

  In the example of FIG. 5, the phase is shifted by 1/2. However, if the exposure amount of each frame is acquired by shifting the phase by 1/3 and three different phases, the combined frame rate is 1440 fps, which is 3 times. And more accurate flicker detection can be performed.

In particular, the processing that becomes the base of the flicker detection processing of the electronic camera 100 according to the present embodiment is the same as the surface flicker detection processing described with reference to FIG. 3, so the processing does not become extremely complicated.
[Process of CPU 111]
Next, the configuration of the CPU 111 for realizing the above processing will be described in detail with reference to FIG. FIG. 6 is a block diagram centered on blocks related to flicker detection in the electronic camera 100 of FIG. 1, and the same reference numerals as those in FIG. 1 denote the same blocks.

  In FIG. 6, the CPU 111 includes a phase adjustment unit 201, a calculation unit 202, and a light source frequency calculation unit 203. The CPU 111 has a control unit 204 that controls these blocks and controls the entire electronic camera 100. In this embodiment, in FIG. 6, the phase adjustment unit 201, the calculation unit 202, and the light source frequency calculation unit 203 are shown as a part of the processing of the CPU 111, but for example, an ASIC (Image Engine: Application Specific) Each block may be aggregated into one functional unit as in Integrated Circuit).

  The phase adjustment unit 201 adjusts the phase of the frame period when reading an image signal from the image sensor 102 when shooting a moving image or an LV image. Here, when flicker is detected, the CPU 111 sets the TG 103 so that an image signal is read out from the image sensor 102 at a frame period that is less than ½ of the maximum blinking period of the light source predicted in advance. As an example, when a fluorescent lamp using a commercial power supply of 60 Hz is a light source, the blinking frequency of the light source is 120 Hz, so the frame rate is set to be greater than 240 fps (frames / second). If this is expressed as a period in order to consider it on the time axis, the frame period (less than 1/240) is set to be less than 1/2 of the blinking period 1/120 of the light source in the above case. Actually, in order to ensure the flicker detection accuracy, a frame rate such as 480 fps or 600 fps exemplified in FIGS. 3, 4, 5, etc. is used as a frame period in which several frames are included in one blinking period of the light source. Is set. When there are a plurality of light sources assumed in advance, the frame rate is set based on the maximum blinking period among them. For example, when a fluorescent lamp using a commercial power supply of 60 Hz and a fluorescent lamp using a commercial power supply of 50 Hz are assumed, the frame rate is set based on the blinking frequency of 120 Hz of the light source of 60 Hz.

  Then, the phase adjustment unit 201 sets the phase of the frame period at a cycle (called a phase adjustment cycle) every integer multiple of the cycle corresponding to the frequency of the greatest common divisor of a plurality of frequencies predicted in advance as the blinking frequency of the light source. adjust. For example, as described above, since the greatest common divisor of 50 Hz and 60 Hz is 20 Hz, the phase of the frame period is adjusted at a period of 50 ms (1/20). The phase adjustment width of the frame period is set to a preset phase (1/2 phase, 1/3 phase, etc.), and the phase of the frame period is delayed by the phase adjustment width for each phase adjustment period. For example, when the frame rate is 600 fps and ½ phase is delayed, the vertical synchronization signal output from the TG 103 to the image sensor 102 is delayed by 1/300 sec for each phase adjustment period. The phase adjustment unit 201 includes a timer 201a for measuring the phase adjustment period.

  In this way, the phase adjustment unit 201 controls the TG 103 so as to delay by a preset phase adjustment width, and image signals of frames having different phases are read from the image sensor 102 and temporarily stored in the RAM 106. Is done.

  The calculation unit 202 includes an exposure amount calculation unit 211 that performs processing that is a base of the flicker detection method of the present embodiment described with reference to FIG. 3, and an exposure amount synthesis unit that synthesizes the exposure amounts of frames read in different phases. 212.

  The exposure amount calculation unit 211 performs an exposure amount calculation process for obtaining an exposure amount for each frame from the image signal stored in the RAM 106. For example, an image signal acquisition process to be read at a high frame rate such as 600 fps and an average process for calculating an average value of the brightness (exposure amount) of the image signal for each acquired frame are performed. At this time, line thinning readout may be performed from the image sensor 102 in order to realize a high-speed frame rate such as 600 fps. In this case, the average value of the brightness of the image signal read out by line thinning is used as the brightness (exposure amount) of the frame. Alternatively, without performing line thinning, the image signals may be added in units of a plurality of lines and read out from the image sensor 102 as an image signal for one line. In this case, since there are no lines to be skipped, flicker detection can be performed even when there is a local light source flicker on the shooting screen. In order to cope with the high-speed frame rate, it is preferable to provide the image pickup device 102 with an addition unit that performs signal addition in hardware.

  In this way, when performing rolling shutter imaging with a normal XY address type CMOS image sensor, flicker appearing in a horizontal stripe shape within one frame can be converted to surface flicker whose brightness changes for each frame. . In the case of using a CCD type image pickup device, photoelectric conversion is simultaneously performed on all pixels, so that an output of surface flicker can be obtained without performing the above processing. Therefore, in the case where flicker detection is performed by applying the same processing as in the present embodiment to a CCD image sensor, the frame rate may be increased and the phase adjustment unit 201 may adjust the phase.

  The exposure amount synthesizing unit 212 performs an exposure amount synthesizing process for synthesizing the exposure amounts of frames (frames converted into surface flicker) read out with different phases read out by the phase adjustment unit 201. . As for the combining method, as described with reference to FIG. 5, the output levels of the frames read at different phases are superimposed on the same time axis for each phase adjustment period to obtain the combined exposure amount.

The light source frequency calculation unit 203 calculates the blinking frequency of the light source based on the temporal change in the combined exposure amount obtained by the exposure amount combining unit 212 of the calculation unit 202. For example, the blinking frequency of the light source may be calculated from an interval at which the combined exposure amount reaches a peak, or frequency detection may be performed using a mathematical method such as FFT.
[Flicker detection processing]
FIG. 7 is a flowchart illustrating an example of the flicker detection operation of the electronic camera 100. This flowchart is started by the CPU 111 when the moving image shooting mode is selected by the command dial or when the LV image is displayed in the still image shooting mode.

  It is assumed that parameters such as the phase adjustment period, phase adjustment width, flicker detection frame acquisition time, and flicker detection frame rate are stored in the ROM 107 in advance. In the following example, the phase adjustment period is 50 ms, the phase adjustment width is 1/2 phase, the flicker detection frame acquisition time is 100 ms, and the frame rate at the time of flicker detection is 600 fps. Here, since the acquisition time of the flicker detection frame is 100 ms, it is possible to synthesize the flicker detection images of the 50 ms period without phase adjustment and the 50 ms period shifted by 1/2 phase.

Hereinafter, the flicker detection process will be described in order.
(Step S <b> 101) The control unit 204 of the CPU 111 reads flicker detection parameters stored in advance in the ROM 107, and performs settings for capturing a flicker detection image. Specifically, the control unit 204 sends a drive signal to the image sensor 102 and the AFE 104 via the TG 103 and issues an instruction to acquire an image signal at a frame rate of 600 fps. Here, the control unit 204 sequentially reads out image signals from the image sensor 102 by a rolling shutter method.

For convenience of explanation, the frame rate of 600 fps is for flicker detection. At the time of actual moving image shooting, the control unit 204, for example, a frame rate of 24 fps, 30 fps, 50 fps, 60 fps, or the like according to the moving image format. Set to. That is, when flicker is detected, the control unit 204 makes the frame rate faster than that during normal moving image shooting.
(Step S102) The phase adjustment unit 201 initializes the time t of the timer 201a to 0.
(Step S103) The phase adjustment unit 201 determines whether or not the time t of the timer 201a has reached the phase adjustment period Ts (50 ms). If t = Ts, the process proceeds to step S105. If t ≠ Ts, the process proceeds to step S104.
(Step S104) The phase adjustment unit 201 outputs a vertical synchronization signal from the TG 103 to the image sensor 102 at a frame rate of 600 fps, and reads an image frame for flicker detection from the image sensor 102.
(Step S105) The phase adjustment unit 201 controls the TG 103 so as to shift the vertical synchronization signal to be output next by ½ phase of the frame period.
(Step S106) The phase adjustment unit 201 initializes the time t of the timer 201a to zero.
(Step S107) The exposure amount calculation unit 211 of the calculation unit 202 calculates an exposure amount for each image frame for flicker detection from the image sensor 102. The exposure amount calculation unit 211 temporarily stores the obtained exposure amount in the RAM 106.
(Step S108) The phase adjustment unit 201 determines whether or not the time t of the timer 201a has reached the flicker detection frame acquisition time Te (100 ms). If t = Te, the process proceeds to step S109. If t ≠ Te, the process returns to step S103. In practice, t = Te is determined as t≈Te including errors such as a slight processing delay.
(Step S109) The exposure amount composition unit 212 sets the exposure amount (output level) of the image frames for flicker detection having different phases temporarily stored in the RAM 106 in step S107 as described in FIG. Each time, synthesis is performed on the same time axis, and an output level after synthesis as shown in FIG.
(Step S110) The light source frequency calculation unit 203 calculates the blinking frequency of the light source from the temporal change of the combined output level obtained in step S109, and outputs it to the control unit 204. Specifically, the light source frequency calculation unit 203 reads the output level of a frame for 50 ms from the RAM 106, for example. Next, the light source frequency calculation unit 203 calculates a cycle in which a frame with a relatively low output level (meaning a frame with a dark screen) appears based on the frame rate. For example, when the frame rate is 480 fps, when a frame having a relatively low output level appears every four frames, the blinking frequency of the light source is calculated to be 120 Hz and the blinking cycle is 1/120 sec. Alternatively, the light source frequency calculation unit 203 may calculate the blinking frequency of the light source based on the period in which frames with relatively high output levels appear. Alternatively, the light source frequency calculation unit 203 may obtain a period in which a frame with a relatively low output level and a frame with a relatively high output appear, and use these average values as the blinking frequency of the light source. If there is no periodicity in the temporal change in the output level of the frame captured in the RAM 106, for example, when shooting is not performed under a fluorescent lamp, the blinking cycle cannot be calculated. Output to the control unit 204.
(Step S111) The control unit 204 determines whether or not the light source frequency calculation unit 203 has detected the blinking frequency of the light source in step S110. When the light source frequency calculation unit 203 can detect the blinking frequency of the light source, the process proceeds to step S112, and when it cannot be detected, the process proceeds to step S113.
(Step S112) The control unit 204 obtains the exposure time of the rolling shutter that does not cause flicker according to the blinking frequency of the light source obtained in Step S110, and performs settings for moving image shooting and LV image display. Specifically, the CPU 111 sends a drive signal to the image sensor 102 and the AFE 104 via the TG 103 and issues an instruction to acquire an image signal at a moving image shooting frame rate (for example, 30 fps). For example, the control unit 204 sets the rolling shutter speed to n times the blinking cycle. As a result, flicker does not appear in the captured image.
(Step S113) Since there is no influence of flicker, the control unit 204 can arbitrarily set the rolling shutter speed during moving image shooting.

  After performing the flicker detection process as described above, moving image shooting and LV image display are started.

As described above, according to the electronic camera 100 of the present embodiment, it is not necessary to perform complicated processing for detecting the horizontal stripe flicker appearing in the frame, and it is easily blinked from the surface flicker generated between a plurality of frames. The frequency can be calculated. Further, the blinking frequency of the light source can be obtained with high accuracy without extremely increasing the frame rate.
(Modification 1)
In the flowchart of FIG. 7 described above, step S109 for synthesizing the output levels of the frames having different phases is executed after step S108 in which the output levels of all the frames have been acquired. The described combining process may be performed every time each frame is acquired. In this case, when the image frames IMG1 to IMG7 of FIG. 5A are read from the image sensor 102, the output levels of the image frames IMG11 to IMG17 of FIG. The synthesis process (c) is performed. For example, since the output levels of the image frames IMG11 to IMG17 when the image frame having the phase adjustment period of 50 ms in the first half is acquired are 0, the image frames IMG1 to IMG7 are the same as in FIG. The same composite output as the output level is obtained. Then, the combined output level when the latter half of the image frame having the phase adjustment period of 50 ms is acquired is the same composite output as in FIG. 5C every time the output levels of the image frames IMG11 to IMG17 are read. Is obtained.

In this way, by sequentially performing the synthesis process every time one image frame is read, the memory capacity required for the process can be reduced. In this case, as shown in the flowchart of FIG. 7, the order of steps S109 and S108 in the flowchart of FIG.
(Modification 2)
In the flowchart of FIG. 7 described above, the phase adjustment unit 201 determines whether or not the time t of the timer 201a exceeds the flicker detection frame acquisition time Te (100 ms), and when t = Te, flicker detection processing is performed. However, the number of times of phase adjustment (phase adjustment number of times Ke) may be set in advance, and the flicker detection process may be ended after performing the phase adjustment for the set number of times. For example, in the above example, the phase adjustment of 50 ms is performed once, and the process is ended when the phase adjustment period after the phase adjustment ends and the second time is counted.

  A flowchart in this case is shown in FIG. In FIG. 8, processing steps having the same reference numerals as those in the flowchart of FIG. In the flowchart of FIG. 8, step S201 is provided between step S102 and step S103 of the flowchart of FIG. 7, and step S202 is further provided between step S106 and step S104 of the flowchart of FIG. Further, step S108 in the flowchart of FIG. 7 is changed to step S203.

Hereinafter, processing different from the flowchart of FIG. 7 (steps S201, S202, and S203) will be described.
(Step S201) The phase adjustment unit 201 initializes a soft counter variable k provided for counting the number of phase adjustments to zero.
(Step S202) The phase adjustment unit 201 increments the counter variable k by one (k = k + 1).
(Step S203) The phase adjustment unit 201 indicates that the count Ke of the counter variable k set in advance is 2 times (for example, the phase adjustment period after the first phase adjustment is ended and the second time is counted). ) Is determined. If k = Ke, the process proceeds to step S109, and if k ≠ Ke, the process returns to step S103.

  In this way, after performing the phase adjustment for the preset number of phase adjustments, the output levels of the frames read at different phases are superimposed on the same time axis for each phase adjustment period, The amount of exposure can be determined.

  In the present embodiment, two frequencies of 50 Hz and 60 Hz are frequencies of a commercial power source used for a light source such as a fluorescent lamp, but other frequencies may be used.

  In the present embodiment, the image sensor 102 that captures the main image to be recorded on the recording medium 113 is used as a sensor that detects flicker. However, an image sensor (separate from the image sensor 102 that captures the main image) ( For example, a photometric sensor) may be employed.

  Furthermore, in this embodiment, the image sensor 102 is an analog output sensor, and the TG 103 and the AFE 104 are provided separately from the image sensor 102. However, the TG 103 may be built in the image sensor 102. In the present invention, the AFE 104 may be built in the image sensor 102 to serve as a digital output sensor. In the case of a digital output sensor, blocks such as the TG 103, the image processing unit 105, the RAM 106, the ROM 107, and the CPU 111 may be freely combined and built in the image sensor 102.

  As described above, the imaging apparatus according to the present invention has been described by way of example in each embodiment. However, the imaging apparatus can be implemented in various other forms without departing from the spirit or main features thereof. Therefore, the above-described embodiment is merely an example in all respects and should not be interpreted in a limited manner. The present invention is defined by the claims, and the present invention is not limited to the text of the specification. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

DESCRIPTION OF SYMBOLS 100 ... Electronic camera, 101 ... Shooting lens, 102 ... Image sensor, 103 ... TG, 104 ... AFE, 105 ... Image processing part, 106 ... RAM, 107 ... ROM, 108 ... display monitor, 109 ... recording I / F unit, 110 ... operation unit, 111 ... CPU, 112 ... bus, 201 ... phase adjustment unit, 202 ... Calculation unit, 203 ... light source frequency calculation unit, 204 ... control unit, 211 ... exposure amount calculation unit, 212 ... exposure amount synthesis unit

Claims (4)

An image sensor that outputs an image signal according to incident light;
A phase adjustment unit that performs phase adjustment of a frame period when the image signal is read from the image sensor;
An arithmetic unit that obtains an exposure amount in units of frames from the image signal, and synthesizes the exposure amounts of the respective frames that are read out in different phases that are phase-adjusted by the phase adjustment unit;
An imaging apparatus comprising: a frequency calculation unit that calculates a blinking frequency of a light source based on a temporal change in the exposure amount after the synthesis. The imaging device according to claim 1,
The imaging apparatus according to claim 1, wherein the phase adjustment unit performs the phase adjustment at a cycle of every integral multiple of a cycle corresponding to a frequency of a greatest common divisor of a plurality of frequencies predicted in advance as a blinking frequency of the light source. The imaging device according to claim 1 or 2,
The frame period is less than half of the maximum blinking period of the light source predicted in advance. In the imaging device according to any one of claims 1 to 3,
The phase adjustment unit performs phase adjustment of a frame period at the time of reading the image signal from the image sensor at least twice,
The said calculating part synthesize | combines the said exposure amount of each flame | frame read by the at least 2 different phase adjusted by the said phase adjustment part. The imaging device characterized by the above-mentioned.

Images