JP2017085462A - Imaging apparatus, flicker detection method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To more accurately detect a flicker generation factor in a state where an affection (flicker stripe) of a flicker during photographing a moving image (including a live vies display) is reduced as much as possible.SOLUTION: A flicker detection method comprises the steps of: determining whether or not a phase of a flicker stripe generated in each of two flame images is an inverse phase by using difference data of two flame image in an interval of N+0.5 times of a period of the flicker in a state where the period of a photographing flame at flicker detection is set to N+0.5 times (N is an integer number) of the flicker period as a detection target (steps A12 to A14); and determining that a light of the flicker period as the detection target becomes the factor of flicker generation (step A15), if the phase is the inverse phase (step A14 is YES).SELECTED DRAWING: Figure 3

Description

  The present invention relates to an imaging apparatus, a flicker detection method, and a program for detecting a flicker occurrence factor based on a change in brightness of a captured frame image.

  When an imaging device such as a digital video camera captures an image in an environment of a light source (for example, a fluorescent tube) whose brightness changes periodically according to a commercial power frequency (for example, 50 Hz, 60 Hz), the commercial power frequency. Is 50 Hz, the brightness of the light source changes in a cycle of 1/100 seconds, and in the case of 60 Hz, the brightness of the light source changes in a cycle of 1/120 seconds. Stripes (flicker stripes) may appear. In this case, in the imaging device (CMOS) of the rolling shutter method (line exposure sequential readout method), when detecting the presence or absence of flicker generation factors such as a fluorescent lamp, the state of occurrence of bright and dark stripes generated in the photographed image is analyzed. In addition, there is known a technique for detecting only a flicker component by a fluorescent lamp while eliminating the influence of stripes due to movement of a subject (see Patent Document 1).

JP 2004-7402 A

  However, the flicker detection method described in the above-mentioned patent document frequency-analyzes the fluctuation of the brightness ratio of two images with different shutter times (exposure times), and the influence of flicker intentionally occurs. Because the shooting conditions are set to be easy to perform, the flicker effect (flicker fringes) appears prominently on the movie data and monitor screen during detection, and flicker detection is performed during movie shooting (including live view display). It was not appropriate to do.

  An object of the present invention is to make it possible to detect a flicker occurrence factor more accurately in a state where the influence (flicker fringes) of flicker during moving image shooting (including live view display) is minimized.

In order to solve the above-described problems, the present invention
In an imaging device including a rolling shutter type imaging device,
Setting means for setting the period of the shooting frame at the time of flicker detection to N + 0.5 times (N is an integer) the flicker period to be detected;
Using the information of the difference or ratio between the two frame images at an interval of N + 0.5 times the flicker period set by the setting means, the fringe phase generated in each of the two frame images is opposite in phase. Determination means for determining whether or not,
Determining means for determining that the light of the flicker cycle to be detected is a cause of flicker when the determining means determines that the phase is opposite;
It is characterized by providing.

  According to the present invention, it is possible to more accurately detect the cause of flicker while reducing the influence of flicker (flicker fringes) during moving image shooting (including live view display) as much as possible.

The block diagram which showed the basic component of the digital camera applied as an imaging device. 6 is a flowchart for explaining a part of the operation under a photographing mode (flicker detection operation that is characteristic of the present embodiment). The flowchart following the operation | movement of FIG. The flowchart for explaining in detail the process (step A10, A13 of FIG. 3) which compares the phase of the fringe which generate | occur | produces in two frame images. The flowchart which showed the other example in the process (step A10 of FIG. 3, A13) which compares the phase of the fringe which generate | occur | produces in two frame images as a modification to this embodiment.

Hereinafter, an embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is a block diagram showing basic components of a digital camera applied as an imaging apparatus.
The imaging device is a digital camera provided with a flicker detection function in addition to an imaging function for capturing still images and moving images, a playback function for displaying recorded and stored images, a timing function for obtaining timing information, and the like. This flicker detection function is a function for detecting fringes caused by flicker appearing in an image captured in the environment of a light source (for example, a fluorescent tube) whose brightness periodically changes according to the commercial power supply frequency. .

  In the present embodiment, as will be described in detail later, a special frame period is temporarily used as a shooting frame period at the time of flicker detection in the case of imaging using a rolling shutter type (line exposure sequential readout type) imaging device (CMOS). It is characterized by being set to. The control unit 1 operates by supplying power from the power source unit (secondary battery) 2 and controls the overall operation of the digital camera in accordance with various programs in the storage unit 3. A CPU (Central Processing Unit) and a memory (not shown) are provided.

  The storage unit 3 includes, for example, a ROM, a flash memory, and the like, and stores programs and various applications for realizing the present embodiment in accordance with operation procedures shown in FIGS. A program memory 3A, and a work memory 3B for temporarily storing various kinds of information (for example, flags) necessary for the operation of the camera. The storage unit 3 may include a removable portable memory (recording medium) such as an SD card or a USB memory, and is connected to a network via a communication function (not shown). The state may include a storage area on a predetermined server device side.

  Although not shown, the operation unit 4 instructs a mode change button for switching between an operation mode (shooting mode) in which shooting can be performed and an operation mode (playback mode) for playing back a captured image (stored image), and to start shooting. In addition to the release button, it is equipped with various keys for performing operations such as setting exposure conditions, shutter speed, aperture, ISO sensitivity, and other shooting conditions. The control unit 1 responds to input operation signals from the operation unit 4. As the processing, for example, mode change processing, shooting processing, setting of shooting conditions, and the like are performed. The display unit 5 is a high-definition liquid crystal display or an organic EL (Electro Luminescence) display, or a monitor screen (live view screen) that displays captured image data (live view image) in real time, or a captured image. It may be a playback screen for playing back data.

  The imaging unit 6 constitutes a camera unit capable of photographing a subject with high definition. The lens unit 6A includes a zoom lens 6B, a focus lens (focusing lens) 6C, an aperture / shutter 6D, and an imaging element 6E. Is provided. The imaging device 6E is a two-dimensional arrangement of pixels in a matrix form of a matrix, and each pixel arranged on a horizontal line (row line) and a vertical line (column line) is arranged in a horizontal line (1 This is a rolling shutter type CMOS image sensor that performs imaging while shifting the exposure timing for each line or several lines).

  Although not shown, the image sensor 6E includes a charge storage unit and a charge transfer unit, and extends from the first line (horizontal line at the top end (row)) to the last line (horizontal line at the bottom end (row)). Are sequentially exposed and read out for each horizontal line while shifting the timing every predetermined time. The terms “vertical” and “horizontal” used herein refer to the pixel data readout direction for the sake of convenience when performing exposure as described above in a rolling shutter type imaging device. It is not limited to the mounting direction of the image sensor or the like. Hereinafter, the horizontal line may be simply referred to as a line.

  The image signal (analog signal) read out from the image sensor 6E is converted into a digital signal (image signal) by an A / D converter (not shown) and subjected to predetermined image display processing, and then displayed. The monitor 5 displays the live view image in real time on the unit 5. In response to an operation of a release button (not shown), the control unit 1 performs a predetermined image process (development process, etc.) on the captured image to generate a captured image, and performs an image compression process to perform a standard operation. After being converted into a typical file format, it is recorded and saved on a recording medium in the storage unit 3.

  As described above, the flicker detection function in the imaging apparatus (digital camera) including the imaging device 6E of the rolling shutter system is the state of occurrence of fringes generated in the captured image, that is, as described in detail later, Confirm that the phase of the fringes generated in each phase is the same and opposite, and eliminate the influence of the fringes due to relative movement between the subject and the imaging device, and detect the occurrence of the fringes caused by flicker In order to minimize the flicker effect (flicker fringes) during movie shooting (including live view display), the flicker occurrence factor can be detected more accurately even if the strength of the fringes generated in the image is small. It is what I did.

  By the way, in general, when video shooting (or live view shooting) is performed in an environment of a light source (for example, a fluorescent tube) whose brightness periodically changes according to a commercial power supply frequency (for example, 50 Hz, 60 Hz), An image without flicker fringes and an image with flicker fringes are obtained according to the exposure time to the image sensor 6E. That is, when the commercial power source frequency is 50 Hz (flicker cycle 1/100 s) or the commercial power source frequency is 60 Hz (flicker cycle 1/120 s), the exposure time (for example, commercial frequency) is an integral multiple of the flicker cycle (10 ms or 8.3 ms). If the power frequency is 50 Hz, the exposure time is 10 ms, 20 ms, 30 ms, etc. If the image is taken at 60 Hz, 8.3 ms, 16.7 ms, 25 ms, etc. Since the effects of flicker accumulated therein are equal, an image free from flicker fringes can be obtained. In addition, when shooting is performed with an exposure time other than an integral multiple of the flicker cycle, an image with flicker fringes is obtained because the influence of flicker differs for each line.

  The flicker stripes have different blinking positions for each frame depending on the frame rate. In the following description, the commercial power supply frequency 50 Hz (flicker frequency 100 Hz: flicker cycle 10 ms) is simply referred to as a 50z power supply, and the commercial power supply frequency 60 Hz (flicker frequency 120 Hz: flicker cycle 8.3 ms) is simply referred to as a 60 Hz power supply. For example, when the frame rate is 30 fps (a cycle of 33 ms), a flicker stripe of a 50 Hz power supply (cycle 10 ms) appears every 3 frames (100 ms) at the same position in the frame, and 1 / Flicker stripes appear at positions shifted by three cycles. By utilizing this fact, it is possible to detect flicker of a 50 Hz power supply (cycle 10 ms) at the time of operation at 30 fps (cycle 33 ms). That is, by comparing the frame images separated by three frames and confirming that there is no relative movement between the subject and the imaging device (subject blur), the adjacent frames are compared with each other, and the comparison result is 50 Hz. It becomes possible to detect the flicker of the power supply (cycle 10 ms).

  Since flicker fringes appear at a position shifted by 1/3 period, a shift amount of about 1/3 (a blinking change amount) does not have a sufficiently large value and affects detection accuracy. On the other hand, when detecting flicker of a 50 Hz power supply (period 10 ms), it takes at least 100 ms to confirm that there is no relative movement of the subject with the imaging device. That is, when the frame rate of moving image shooting is 30 fps (cycle 33 ms), the synchronization with the 50 Hz power supply (flicker frequency 100 Hz, flicker cycle 10 ms) is every 100 ms (the least common multiple of 33 ms and 10 ms), and the same synchronized If there is no change in the two frame images of the phase, it is determined that there is no movement of the subject, and thus it takes 100 ms.

  Therefore, in the present embodiment, the period of the photographic frame (flicker detection frame period) at the time of flicker detection is temporarily set to N + 0.5 times (N is an integer) the flicker period to be detected. ing. That is, for example, when the flicker cycle to be detected is a 50 Hz power supply (10 ms), the flicker detection frame rate (frame cycle) is N = 0, 200 fps (5 ms), N = 1, 67 fps (15 ms), N = 2 is set to 40 fps (25 ms), N = 3 is set to 29 fps (35 ms), N4 is set to 22 fps (45 ms),... Similarly, when the flicker cycle to be detected is a 60 Hz power supply (8.3 ms), the flicker detection frame rate (frame cycle) is 240 fps (4.1 ms) when N = 0, and 80 fps (12.12) when N = 1. 5 ms), N = 2 to 48 fps (20.8 ms), N = 3 to 34 fps (29.2 ms), N = 4 to 27 fps (37.5 ms), N = 5 to 22 fps (45.8 ms),. Either one is set.

  In this embodiment, as described above, in the state where the flicker detection frame rate (frame period) is temporarily set to N + 0.5 times (N is an integer) the flicker period to be detected, exposure, As a transferred image, two frame images with an interval of N + 0.5 times are compared, and using the brightness difference data of the images, the phase of the fringes generated in each of the two frame images is opposite in phase. It is determined whether it is in phase or in phase. In this case, each frame has an opposite phase and the same phase. Here, the difference in brightness of the image is not a difference in brightness obtained by averaging the entire image, but is a difference in brightness indicating how much the difference in brightness for each pixel is in the entire image. Considering subject blur, etc., comparison may be made in units of a plurality of pixels, not necessarily in units of one pixel. If a change in brightness in one direction is a major problem, such as a stripe caused by flicker, 1 or The difference in brightness of the entire image compared in units of multiple lines.

  When the phase of the fringes generated in each of the two frame images having an interval of N + 0.5 times is reversed, that is, when the phase of the fringes is different by 180 °, the difference in image brightness is maximized. Thus, compared to the case of using two frame images having a phase shift of 1/3, the detection of flicker is more reliable, and the fringe phase is the same for every two frames. It is possible to compare the difference between two frame images at short intervals. Further, in the present embodiment, in order to make flicker detection more reliable, for example, in addition to determining whether or not the phases of the fringes are opposite phases for two adjacent frame images, It is determined whether or not the phase of the fringes generated in each of the two frame images separated by one frame is the same phase, and the phase of the fringes generated in each of the two frame images separated by one frame is confirmed to be the same phase. After confirming that there is no relative movement between the subject and the imaging device, when the phase of the fringes generated in each of the two adjacent frame images is opposite, the flicker of the flicker cycle to be detected It is determined that there is a cause.

  Next, the operation concept of the imaging apparatus in the present embodiment will be described with reference to the flowcharts shown in FIGS. Here, each function described in these flowcharts is stored in the form of a readable program code, and operations according to the program code are sequentially executed. Further, it is possible to sequentially execute the operation according to the above-described program code transmitted via a transmission medium such as a network. In other words, in addition to the recording medium, an operation unique to the present embodiment can be executed using a program / data supplied externally via a transmission medium. 2 and 3 are flowcharts showing an outline of the operation of the characteristic part of the present embodiment in the entire operation of the imaging apparatus. When the flow of FIG. 2 and FIG. Return to the main flow (not shown).

2 and 3 are flowcharts for explaining a part of the operation under the shooting mode (flicker detection operation which is characteristic of the present embodiment), and is executed prior to the start of moving image shooting.
First, the control unit 1 sets a shooting frame rate corresponding to the current mode (shooting mode arbitrarily selected from various moving image shooting modes) (step A1). At this time, if the shooting mode is the live view display, 30 fps is set as the shooting frame rate. Next, the shooting operation corresponding to the current mode is started, and if the shooting mode is the live view display mode, the display operation of the captured image is further started (step A2). In this state, it is checked whether it is a predetermined flicker detection timing (for example, every 1 second) (step A3). If it is not the detection timing (NO in step A3), another operation in the shooting mode is performed. If it is the detection timing (YES in step A3), the process proceeds to the following flicker detection process (steps A4 to A17), and whether or not the light of the flicker cycle to be detected is a flicker occurrence factor. Determine.

  The control unit 1 selects one of the 50 Hz power source (10 ms) and 60 Hz power source (8.3 ms) of the flicker cycle to be detected as the current detection target (step A4). Then, each value of N + 0.5 times (N = 0, 1, 2, 3,...) Of the selected flicker cycle of the detection target is determined as a flicker detection frame rate (frame cycle) candidate (step A5). After that, the frame rate closest to the shooting frame rate set in step A1 is selected from the candidates (step A6).

  That is, when determining the flicker detection frame rate (frame period) to be detected this time, in order to enable flicker detection without reducing the shooting frame rate as much as possible, the frame closest to the shooting frame rate is used. Select a rate from the candidates. For example, if the shooting frame rate is 30 fps and the flicker cycle to be detected is a 50 Hz power supply (10 ms), N = 3 29 fps (35 ms) is selected as the flicker detection frame rate (frame cycle). If the power source is 60 Hz (8.3 ms), N = 3 27 fps (37.5 ms) is selected as the flicker detection frame rate (frame period).

  Then, the selected flicker detection frame rate (frame period) is temporarily set (step A7), and then three frame images continuously captured at the flicker detection frame rate are acquired to obtain the work memory 3B. (Step A8). Then, two frame images having an interval of M times the flicker cycle (M is an integer), for example, when M = 1, two frame images separated by one frame (the first frame image and the third frame image) Designation is made as a comparison target (step A9), and the process proceeds to a process of comparing the phase of the fringes generated in the two frame images (step A10).

FIG. 4 is a flowchart for explaining in detail the phase comparison process (steps A10 and A13 in FIG. 3) of fringes generated in two frame images.
First, the control unit 1 integrates the two frame images to be compared for each horizontal line, and extracts brightness variation data in the vertical direction (step B1). That is, since the fringes caused by flicker appear in the vertical direction of the frame image, when image analysis is performed, all the pixel data arranged in the horizontal direction in each horizontal line is horizontally processed for one frame image. Integration is performed for each line, and a plurality of integration values corresponding to a plurality of horizontal lines are extracted as brightness fluctuation data in the vertical direction. Next, for the other frame image, similarly, all the pixel data arranged in the horizontal direction in each horizontal line are integrated for each horizontal line, and a plurality of integration values corresponding to the plurality of horizontal lines are integrated in the vertical direction. Extracted as brightness fluctuation data. Each integration described above is not limited to being performed for each line, but may be performed for each of a plurality of lines.

  Next, the two brightness fluctuation data extracted as described above are compared for each horizontal line to obtain a difference for each horizontal line, and the difference for each horizontal line is integrated in the vertical direction to obtain the accumulated value. (Step B2), it is determined whether the fringes generated in the two frame images have the same phase or the opposite phase by comparing the accumulated difference value with a predetermined threshold (Steps B3 to B7). That is, the difference accumulated value of the two images is compared with a preset first threshold value (reference value for in-phase determination) to check whether the difference accumulated value is within the first threshold value (step B3). If the accumulated difference value is within the first threshold (YES in step B3), it is determined that the fringe phases generated in the two frame images are the same phase (step B7).

  On the other hand, if the difference accumulated value of the two images is not within the first threshold value (NO in step B3), the difference accumulated value is compared with a preset second threshold value (reference value for antiphase determination). Then, it is checked whether the difference accumulated value is equal to or greater than the second threshold (step B4). If the accumulated difference value is equal to or greater than the second threshold value (YES in step B4), it is determined that the fringe phases generated in the two frame images are opposite phases (step B5). If the accumulated difference value of the two images is larger than the first threshold value (NO in step B3) and smaller than the second threshold value (NO in step B4), it is determined that the phase is other than the same phase and the opposite phase ( Step B6).

  When such a phase comparison process (step A10 in FIG. 3) is completed, it is checked whether the result determined in this phase comparison process is the same phase (step A11). If the result is not the same phase (NO in step A11) ), Returning to the above step A8, and after acquiring three newly captured continuous frame images, the phase of fringes generated in the first frame image and the third frame image is determined to be the same phase. The above-described operation is repeated until the above (steps A8 to A11). Note that the number of repetitions may be counted, and the flow of FIG. 4 may be forcibly terminated when detection is impossible when the count value exceeds a predetermined value.

  When two frame images separated by one frame (the first frame image and the third frame image) are designated as comparison targets (step A9), the above-described phase comparison process (step A10 in FIG. 3) is executed. If there is a fringe attributed to flicker in the two frame images, the difference accumulated value of the two images becomes equal to or less than the first threshold (YES in step B3 in FIG. 4), and the phase of the fringe Are determined to be in phase (step B7). That is, since the phase of fringes caused by flicker occurring in two frame images at an interval of N + 0.5 times the flicker cycle to be detected becomes the same phase every two frames, two frame images separated by one frame If a fringe due to flicker occurs in (the first frame image and the third frame image), the phase of the fringe is the same phase.

  Thus, when the phase of the fringes generated in two frame images separated by one frame is the same phase (YES in step A11), the process proceeds to the next step A12, and two frame images adjacent to each other temporally before and after. After specifying (for example, the first frame image and the second frame image) as a comparison target, the process proceeds to a process of comparing the phase of the fringes generated in the two adjacent frame images (step A13). The phase comparison process in this case is also the same as the flow of FIG. 4 described above.

  That is, two frame images are integrated for each horizontal line to extract brightness fluctuation data in the vertical direction (step B1 in FIG. 4), and the extracted two brightness fluctuation data are compared for each horizontal line to compare the horizontal line. Each difference is integrated in the vertical direction to obtain the accumulated value (step B2). In this case, if a fringe due to flicker occurs in these two frame images, the difference accumulated value is equal to or greater than the second threshold (YES in step B4), and the phase of the fringe generated in each of the two adjacent frame images Is determined to be in opposite phase (step B6). That is, the fringe phase caused by flicker occurring in two frame images at an interval of N + 0.5 times the flicker cycle to be detected has the same phase and opposite phase for each frame. If the image (the first frame image and the second frame image) has stripes due to flicker, the phases of the stripes are opposite to each other.

  When this phase comparison process (step A13 in FIG. 3) is completed, it is checked whether the result determined by this phase comparison process is in reverse phase (step A14). If it is in reverse phase (YES in step A14), detection is performed. Although it is determined that there is flicker due to the light with the flicker cycle to be detected, that is, it is determined that the light with the flicker cycle to be detected is the cause of flicker generation (step A15), but is not in reverse phase ( NO in step A14), it is determined that there is no flicker due to the light of the flicker cycle to be detected (step A16).

  Thereafter, the process proceeds to step A17 to check whether there are other flicker cycles to be detected, that is, whether all flicker cycles have been selected as detection targets (whether there are unselected flicker cycles as detection targets). Now, when the flicker cycle (10 ms) of the 50 Hz power supply is selected for the first time, the flicker cycle (8.3 ms) of the 60 Hz power supply is not selected (YES in step A17), so the process returns to the above step A4. A flicker cycle is selected as a detection target. Thereafter, N + 0.5 times the selected flicker cycle (N = 1, 2, 3,...) Is determined as a flicker detection frame rate (frame cycle) candidate (step A5), and then the above operation is repeated. (Steps A6 to A17). When all the flicker periods have been selected as detection targets (NO in step A17), the process of returning from the flicker detection frame rate to the shooting frame rate (step A18) is performed, and then FIG. 3 and the operation moves to another operation (not shown) under the photographing mode.

  In this case, processing for changing shooting conditions (exposure time, etc.) is performed based on the occurrence of flicker detected by the above-described flicker detection processing and the type of flicker (flicker of 50 Hz power supply, flicker of 60 Hz power supply). Thus, the exposure time (shutter speed) is changed to an integral multiple of the flicker cycle (10 ms or 8.3 ms) based on the type of flicker detected (flicker of 50 Hz power supply, flicker of 60 Hz power supply). For example, the exposure time is changed to any one of 10 ms, 20 ms, 30 ms, etc. under a 50 Hz power supply environment, and is changed to any one of 8.3 ms, 16.7 ms, 25 ms, etc. under a 60 Hz power supply environment. Is done. In this state, when the start of shooting is instructed, moving image shooting is started under the above-described shooting conditions.

  As described above, in the present embodiment, the control unit 1 sets the period of the shooting frame (flicker detection frame period) at the time of flicker detection to N + 0.5 times (N is an integer) the flicker period to be detected. In this state, using the difference data between the two frame images at intervals of N + 0.5 times the flicker period, it is determined whether or not the fringe phase generated in each of the two frame images is opposite in phase. If it is discriminated and the phase is reversed, it is determined that the flicker cycle light to be detected is the cause of flicker. Therefore, the influence of flicker during movie shooting (including live view display) is affected. Even if the intensity of the fringes generated in the image is reduced in order to reduce it as much as possible, it is possible to detect the flicker occurrence factor more accurately. That is, if there are fringes due to flicker in two frame images that are N + 0.5 times apart, the difference in image brightness is different when the phases of the fringes are opposite and 180 ° apart. For example, flicker detection is more reliable than when two frame images having a phase shift of 1/3 are used.

  Since the control unit 1 determines whether or not the phases of the fringes are opposite phases for two adjacent frame images, for example, two frames having a phase shift of 1/3 Compared to the case of using an image, the difference between two frame images can be compared at a short interval of two frames, flicker detection can be performed quickly, and subject blur (relative between the subject and the imaging device) can be detected. And the effects of camera shake can be reduced accordingly.

  The control unit 1 has a phase of fringes generated in each of two frame images at an interval of N + 0.5 times the flicker cycle and an antiphase, and is at an interval of M times the flicker cycle (M is an integer). When it is determined that the phase of the fringes generated in each of the two frame images is the same phase, it is determined that there is a flicker generation factor of the flicker cycle to be detected. The phase can be confirmed after confirming that there is no relative movement between the image pickup apparatus and the image pickup device, and flicker detection is avoided by avoiding timing that is not suitable for flicker detection, such as when the image pickup position of the subject changes greatly. And flicker detection becomes even more accurate.

  When the phase of the flicker fringe generated in each of the two adjacent frame images is opposite in phase and the phase of the flicker fringe generated in each of the two frame images separated by one frame is the same phase, the control unit 1 Since it is determined that there is a flicker generation factor of the flicker cycle to be detected, even if the opposite phase is confirmed after confirming that the phase is the same, the difference between the two frame images is as short as two frames. And flicker detection can be performed quickly.

  When there are a plurality of flicker periods to be detected such as 50 Hz power supply (10 ms) and 60 Hz power supply (8.3 ms), the control unit 1 determines the period of the shooting frame at the time of flicker detection for each flicker period. Is set to N + 0.5 times the flicker cycle (N is an integer) and the difference between the two frame images at the interval N + 0.5 times the flicker cycle to be detected is used. Whether or not the phase of the fringes generated in each of them is an opposite phase is determined for each flicker cycle to be detected, so that the frame cycle for flicker detection can be shared even under an environment where the commercial power supply frequency is different. It is possible to set using “N + 0.5 times flicker cycle”.

  When determining whether or not the phase of the fringes generated in each of the two frame images is opposite in phase, the control unit 1 accumulates the difference in brightness between the two frame images and uses the accumulated value for determination. Since it is performed by comparing with the threshold value, it is easy and reliable to determine whether or not the phase is opposite.

  The control unit 1 determines that the fringe phase is an opposite phase when the cumulative value of the difference in brightness between two adjacent frame images is equal to or greater than a predetermined threshold, and determines the brightness of two frame images separated by one frame. Since the fringe phase is determined to be the same phase when the accumulated difference value is less than the predetermined threshold, it is easy and reliable to determine whether the phase is the opposite phase or the same phase. .

  The control unit 1 performs flicker detection on a frame period closest to a frame period corresponding to a preset shooting frame rate among a plurality of frame periods obtained by N + 0.5 times the flicker period to be detected. Since it is set as the period of the shooting frame at that time (frame period for flicker detection), it is possible to detect flicker without reducing the shooting frame rate as much as possible.

  In the above-described embodiment, the difference data (brightness difference accumulated value) between two frame images at intervals of N + 0.5 times the flicker cycle is used to generate each of the two frame images. Although it is determined whether or not the phase of the fringes is an antiphase, as shown in the flowchart of FIG. 5, the two frames are used by using the ratio data (brightness ratio) of the two frame images. You may make it discriminate | determine whether the phase of the fringe which arises in each of an image is an antiphase.

FIG. 5 is a flowchart showing another example of phase comparison processing (steps A10 and A13 in FIG. 3) of fringes generated in two frame images as a modification of the present embodiment.
First, similarly to step A1 in FIG. 3, the control unit 1 integrates each frame image to be compared for each horizontal line in order to cause stripes caused by flicker to appear in the vertical direction of the frame image. Direction brightness variation data is extracted (step C1). Next, the brightness ratio for each horizontal line of the two brightness fluctuation data extracted as described above is calculated (step C2), and the flicker to be detected is detected from the calculated ratio data for each horizontal line. A range corresponding to N (integer) times the period is sampled, and Fourier spectrum analysis is performed at frequency N (step C3).

  In other words, the value obtained by integrating the two frame images to be compared for each horizontal line is compared for each horizontal line, and the brightness ratio for each horizontal line is obtained. A range corresponding to N (integer) times the flicker period to be detected is specified, and data of a plurality of ratios in the specified image area are extracted. In this case, since a change in brightness periodically appears according to the flicker cycle, the ratio extracted from the image area corresponding to an integral multiple of 10 ms or more, preferably twice or more of the time when the flicker cycle is 10 ms is detected. When the flicker cycle is detected at 8.3 ms, data of a ratio extracted from the image area corresponding to an integral multiple of 8.3 ms or more, preferably twice or more of that time is required.

  Then, the brightness change in the vertical direction (brightness ratio fluctuation data) of the ratio data extracted from the image area is represented by the frequency N (N = 1, 2, 3,... Integer) corresponding to the flicker cycle. The spectrum is analyzed by Fourier spectrum analysis (analyze how many times the frequency component corresponding to the frequency is included when the entire image area is one period). As a result of the Fourier spectrum analysis, the spectrum value of frequency N is compared with a preset third threshold value (reference value for in-phase determination), and is the spectrum value of frequency N equal to or less than the third threshold value? Is discriminated (step C4).

  If the spectrum value of the frequency N is less than or equal to the first threshold value (YES in step C3), it is determined that the phase of the fringes generated in the two frame images is the same phase (step C8). On the other hand, if the spectrum value of frequency N is not less than or equal to the first threshold value (NO in step C4), the spectrum value of frequency N is compared with a preset fourth threshold value (reference value for antiphase determination), It is checked whether the spectrum value of the frequency N is greater than or equal to the fourth threshold value (step C5). Here, if it is greater than or equal to the fourth threshold value (YES in step C5), it is determined that the phase of the fringes generated in the two frame images is opposite (step C6). If the spectrum value of frequency N is larger than the third threshold value (NO in step C4) and smaller than the fourth threshold value (NO in step C5), it is determined that the phase is other than in-phase and anti-phase (step C7). ).

  In this way, when determining whether or not the phase of the fringes generated in each of the two frame images is opposite, the ratio of the brightness of the two images is subjected to Fourier spectrum analysis, and the Fourier spectrum value corresponding to the flicker period Is compared with the threshold for determination, it is possible to accurately determine whether or not the subject is in reverse phase even when the brightness of the subject itself periodically varies in the vertical direction.

  Further, when the Fourier spectrum value of the brightness ratio of two adjacent images is equal to or greater than a predetermined threshold, it is determined that the phase of the flicker fringe is opposite, and the Fourier of the ratio of the two frame images separated by one frame. If it is determined that the phase of the flicker fringe is the same phase when the spectrum value is less than the predetermined threshold value, it is easy and reliable to determine whether the phase is the opposite phase or the same phase.

  In the above-described embodiment, the flicker cycle is 50 Hz power source (10 ms) and 60 Hz power source (8.3 ms) as a detection target, and flicker detection is performed for one flicker cycle (for example, 10 ms), and then the other flicker cycle. Flicker detection (for example, 8.3 ms) is performed. However, the present invention is not limited to this, and the following may be performed. That is, in the state where the previous flicker detection result detected at the previous power on is stored and retained after the power is turned off, the previous flicker detection result is read at the current power on, and the previous detection result is 50 Hz flicker present. If the flicker cycle of the 50 Hz power supply is determined to be present, the flicker cycle of the 50 Hz power supply is selected as the detection target, and if the previous detection result is 60 Hz flicker (the flicker due to the flicker cycle of the 60 Hz power supply). If it is determined that there is a flicker), the flicker cycle of the 60 Hz power supply may be selected as a detection target to perform flicker detection. Thus, by selecting the current detection target with reference to the previous detection result, it is possible to suppress the occurrence of flicker following the previous time when the power is turned on this time.

  In the above-described embodiment, the case where the present invention is applied to a digital camera as an imaging device has been described. However, the present invention is not limited thereto, and a personal computer with a camera function, a PDA (personal portable information communication device), a tablet terminal device, and a smartphone. It may be a mobile phone, an electronic game, a music player, or the like.

  Further, the “apparatus” and “unit” shown in the above-described embodiments may be separated into a plurality of cases by function, and are not limited to a single case. In addition, each step described in the above-described flowchart is not limited to time-series processing, and a plurality of steps may be processed in parallel or separately.

The embodiment of the present invention has been described above. However, the present invention is not limited to this, and includes the invention described in the claims and the equivalent scope thereof.
Hereinafter, the invention described in the claims of the present application will be appended.
(Appendix)
(Claim 1)
The invention described in claim 1
In an imaging device including a rolling shutter type imaging device,
Setting means for setting the period of the shooting frame at the time of flicker detection to N + 0.5 times (N is an integer) the flicker period to be detected;
Using the information of the difference or ratio between the two frame images at an interval of N + 0.5 times the flicker period set by the setting means, the fringe phase generated in each of the two frame images is opposite in phase. Determination means for determining whether or not,
Determining means for determining that the light of the flicker cycle to be detected is a cause of flicker when the determining means determines that the phase is opposite;
It is characterized by providing.
(Claim 2)
The invention according to claim 2 is the imaging apparatus according to claim 1,
The determination means determines whether or not the phase of the fringes is an opposite phase for two adjacent frame images.
It is characterized by that.
(Claim 3)
The invention according to claim 3 is the imaging apparatus according to claim 1 or 2,
The determination means has a phase of fringes generated in each of two frame images at an interval of N + 0.5 times the flicker period by the determination means, and is M times the flicker period (M is an integer). When it is determined that the phase of the fringes generated in each of the two frame images at the interval is the same phase, it is determined that there is a flicker generation factor of the flicker cycle to be detected.
It is characterized by that.
(Claim 4)
The invention according to claim 4 is the imaging apparatus according to claim 3,
The determination means is such that the phase of flicker fringes generated in each of two adjacent frame images by the determination means is opposite in phase, and the phase of flicker fringes generated in each of two frame images separated by one frame is the same phase. If it is determined that there is a flicker occurrence factor of the flicker cycle to be detected,
It is characterized by that.
(Claim 5)
The invention according to claim 5 is the imaging apparatus according to any one of claims 1 to 4,
When there are a plurality of flicker periods to be detected, the setting means sets the period of the shooting frame at the time of flicker detection to N + 0.5 times the flicker period to be detected (N is an integer) for each flicker period. Set,
The discriminating means uses the information of the difference or ratio between the two frame images at intervals of N + 0.5 times the flicker period set for each flicker period to be detected by the setting means. Whether or not the fringe phase generated in each of the images is an opposite phase is determined for each flicker cycle to be detected,
It is characterized by that.
(Claim 6)
The invention according to claim 6 is the imaging apparatus according to any one of claims 1 to 5,
When determining whether or not the phase of the fringes generated in each of the two frame images is opposite in phase, the determining unit accumulates the difference in brightness between the two frame images and uses the accumulated value for determination. By comparing with the threshold,
It is characterized by that.
(Claim 7)
The invention according to claim 7 is the imaging apparatus according to claim 6,
The discriminating unit discriminates that the fringe phase is in the opposite phase when the cumulative value of the brightness difference between two adjacent frame images is equal to or greater than a predetermined threshold value, and determines the brightness of two frame images separated by one frame. Determining that the fringe phase is the same phase when the cumulative value of the difference is less than a predetermined threshold,
It is characterized by that.
(Claim 8)
The invention according to claim 8 is the imaging apparatus according to any one of claims 1 to 7,
The discrimination means performs a Fourier spectrum analysis on the ratio of the brightness of the two images, and compares the Fourier spectrum value corresponding to the flicker period with a threshold for determination.
It is characterized by that.
(Claim 9)
The invention according to claim 9 is the imaging apparatus according to claim 8,
The discriminating unit discriminates that the phase of the flicker fringe is opposite in phase when the Fourier spectrum value of the brightness ratio of two adjacent images is equal to or greater than a predetermined threshold value, and the two frame images separated by one frame are identified. When the Fourier spectrum value of the ratio is less than a predetermined threshold, it is determined that the phase of the flicker fringe is the same phase.
It is characterized by that.
(Claim 10)
According to a tenth aspect of the present invention, in the imaging apparatus according to any one of the first to ninth aspects,
The setting means detects a flicker that is closest to a frame period corresponding to a preset frame rate for photographing among a plurality of frame periods obtained by N + 0.5 times the flicker period to be detected. Set as shooting frame period at time,
It is characterized by that.
(Claim 11)
The invention according to claim 11
A flicker detection method in an imaging device including a rolling shutter type imaging device,
A process of setting the period of the shooting frame at the time of flicker detection to N + 0.5 times (N is an integer) the flicker period to be detected;
Whether or not the phase of the flicker fringes generated in each of the two frame images is opposite in phase using the information on the difference or ratio of the two frame images at an interval of N + 0.5 times the set flicker period. Processing to determine whether or not
When it is determined that the phase is opposite, a process for determining that light having a flicker period to be detected is a flicker occurrence factor;
It is characterized by including.
(Claim 12)
The invention according to claim 12
For a computer of an image pickup apparatus having a rolling shutter type image pickup device,
A function for setting the period of the photographing frame at the time of flicker detection to N + 0.5 times (N is an integer) the flicker period to be detected;
Whether or not the phase of the flicker fringes generated in each of the two frame images is opposite in phase using the information on the difference or ratio of the two frame images at an interval of N + 0.5 times the set flicker period. A function to determine whether or not
A function for determining that the light of the flicker cycle to be detected is a cause of flicker when it is determined that the phase is opposite;
It is a program for realizing.

DESCRIPTION OF SYMBOLS 1 Control part 3 Memory | storage part 3A Program memory 3B Work memory 4 Operation part 6 Imaging part 6E Imaging element

Claims (12)

In an imaging device including a rolling shutter type imaging device,
Setting means for setting the period of the shooting frame at the time of flicker detection to N + 0.5 times (N is an integer) the flicker period to be detected;
Using the information of the difference or ratio between the two frame images at an interval of N + 0.5 times the flicker period set by the setting means, the fringe phase generated in each of the two frame images is opposite in phase. Determination means for determining whether or not,
Determining means for determining that the light of the flicker cycle to be detected is a cause of flicker when the determining means determines that the phase is opposite;
An imaging apparatus comprising: The determination means determines whether or not the phase of the fringes is an opposite phase for two adjacent frame images.
The imaging apparatus according to claim 1. The determination means has a phase of fringes generated in each of two frame images at an interval of N + 0.5 times the flicker period by the determination means, and is M times the flicker period (M is an integer). When it is determined that the phase of the fringes generated in each of the two frame images at the interval is the same phase, it is determined that there is a flicker generation factor of the flicker cycle to be detected.
The imaging apparatus according to claim 1 or 2, wherein The determination means is such that the phase of flicker fringes generated in each of two adjacent frame images by the determination means is opposite in phase, and the phase of flicker fringes generated in each of two frame images separated by one frame is the same phase. If it is determined that there is a flicker occurrence factor of the flicker cycle to be detected,
The imaging apparatus according to claim 3. When there are a plurality of flicker periods to be detected, the setting means sets the period of the shooting frame at the time of flicker detection to N + 0.5 times the flicker period to be detected (N is an integer) for each flicker period. Set,
The discriminating means uses the information of the difference or ratio between the two frame images at intervals of N + 0.5 times the flicker period set for each flicker period to be detected by the setting means. Whether or not the fringe phase generated in each of the images is an opposite phase is determined for each flicker cycle to be detected,
The image pickup apparatus according to claim 1, wherein the image pickup apparatus is an image pickup apparatus. When determining whether or not the phase of the fringes generated in each of the two frame images is opposite in phase, the determining unit accumulates the difference in brightness between the two frame images and uses the accumulated value for determination. By comparing with the threshold,
The imaging apparatus according to any one of claims 1 to 5, wherein The discriminating unit discriminates that the fringe phase is in the opposite phase when the cumulative value of the brightness difference between two adjacent frame images is equal to or greater than a predetermined threshold value, and determines the brightness of two frame images separated by one frame. Determining that the fringe phase is the same phase when the cumulative value of the difference is less than a predetermined threshold,
The imaging apparatus according to claim 6. The discrimination means performs a Fourier spectrum analysis on the ratio of the brightness of the two images, and compares the Fourier spectrum value corresponding to the flicker period with a threshold for determination.
The image pickup apparatus according to claim 1, wherein the image pickup apparatus is an image pickup apparatus. The discriminating unit discriminates that the phase of the flicker fringe is opposite in phase when the Fourier spectrum value of the brightness ratio of two adjacent images is equal to or greater than a predetermined threshold value, and the two frame images separated by one frame are identified. When the Fourier spectrum value of the ratio is less than a predetermined threshold, it is determined that the phase of the flicker fringe is the same phase.
The imaging apparatus according to claim 8. The setting means detects a flicker that is closest to a frame period corresponding to a preset frame rate for photographing among a plurality of frame periods obtained by N + 0.5 times the flicker period to be detected. Set as shooting frame period at time,
The image pickup apparatus according to claim 1, wherein the image pickup apparatus is an image pickup apparatus. A flicker detection method in an imaging device including a rolling shutter type imaging device,
A process of setting the period of the shooting frame at the time of flicker detection to N + 0.5 times (N is an integer) the flicker period to be detected;
Whether or not the phase of the flicker fringes generated in each of the two frame images is opposite in phase using the information on the difference or ratio of the two frame images at an interval of N + 0.5 times the set flicker period. Processing to determine whether or not
When it is determined that the phase is opposite, a process for determining that light having a flicker period to be detected is a flicker occurrence factor;
A flicker detection method comprising: For a computer of an image pickup apparatus having a rolling shutter type image pickup device,
A function for setting the period of the photographing frame at the time of flicker detection to N + 0.5 times (N is an integer) the flicker period to be detected;
Whether or not the phase of the flicker fringes generated in each of the two frame images is opposite in phase using the information on the difference or ratio of the two frame images at an interval of N + 0.5 times the set flicker period. A function to determine whether or not
A function for determining that the light of the flicker cycle to be detected is a cause of flicker when it is determined that the phase is opposite;
A program to realize

Images