JP2014007692A - Image pick-up device and control method thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress false detection of an unattended object, regardless of the amount of infrared ray, when switching an infrared light cut filter.SOLUTION: The image pick-up device includes image capturing mode switching means 116 for switching the image capturing mode by inserting and withdrawing an infrared light cut filter 102, reference image storage means 110 for storing the reference image of each image capturing mode, difference detection means 111 for calculating the difference value of luminance between the reference image and an image under capturing, unattended object detection means 109 performing unattended object detection based on the difference value, luminance change calculation means 105 for calculating a luminance change due to insertion and withdrawal of the infrared light cut filter, and gain control means 104 for controlling the gain of an electric signal in the image capturing means. The gain control means adjusts the gain so as to reduce the luminance change thus calculated, when the image capturing mode is switched.

Description

  The present invention relates to an imaging apparatus having an unmoving object detection function and an infrared light cut filter switching function, and a control method thereof.

  Conventionally, when a sudden luminance change occurs, such as when an infrared light cut filter is removed during unmoving object detection, the unmoving object detection malfunctions, and the entire screen may be detected as an unmoving object. . As a prior art for suppressing this erroneous detection, Patent Document 1 discloses a technique for updating a stored reference image and switching the determination by the first detection determination unit and the second detection determination unit.

JP 2010-050637 A

  However, in the prior art disclosed in Patent Document 1 described above, since the image itself does not change, there is a possibility of erroneous detection due to the amount of infrared light. For example, when the amount of infrared light is very large, the high and low luminance areas are all overexposed, and even if the non-moving object disappears at the time of switching, the same luminance is displayed. There is.

(Object of invention)
The objective of this invention is providing the imaging device which can suppress the misdetection of a non-moving object detection, and its control method, regardless of the amount of infrared light quantity at the time of infrared light cut filter switching.

  In order to achieve the above object, an image pickup apparatus according to the present invention includes an image pickup unit that converts an object image formed by an image pickup optical system into an electrical signal, and a red that blocks infrared light from incident light on the image pickup unit. An external light cut filter, an infrared light cut filter driving means for driving insertion / extraction of the infrared light cut filter on an optical path, and a photographing mode switching means for switching a photographing mode by inserting / removing the infrared light cut filter A reference image storage unit that stores a reference image for each shooting mode, a difference detection unit that calculates a difference value of luminance between the reference image and the image being shot, and unmoving object detection based on the difference value An imaging apparatus including a non-moving object detection unit, wherein a luminance change calculation unit that calculates a luminance change due to insertion / extraction of the infrared light cut filter and a gain of an electric signal in the imaging unit are controlled. In-control means, and the gain control means adjusts the gain so as to reduce the luminance change calculated by the luminance change calculation means when the photographing mode is switched. To do.

  According to the present invention, erroneous detection of unmoving object detection can be suppressed regardless of the amount of infrared light when the infrared light cut filter is switched.

1 is a block diagram illustrating a configuration example of an imaging apparatus according to Embodiment 1 of the present invention. 3 is a flowchart illustrating an example of an unmoving object detection processing procedure according to the first embodiment. It is an image figure for demonstrating a non-moving object detection process. 3 is a flowchart illustrating an example of an operation according to the first exemplary embodiment. It is a graph which shows an example of the relationship between the brightness | luminance at the time of IRCF operation | movement, and time. It is a graph which shows an example of the gain control at the time of IRCF operation | movement. It is a graph which shows an example of the relationship between a brightness | luminance at the time of aiming at a sensitivity improvement also using WB gain at the time of IRCF operation | movement. It is a graph which shows an example of gain control when a sensitivity improvement is aimed at also using WB gain at the time of IRCF operation. It is a block diagram which shows the structural example of the imaging device which concerns on Example 2 of this invention. 10 is a flowchart illustrating an example of an operation according to the second exemplary embodiment. FIG. 10 is a diagram illustrating an example of a method for creating a reference image for infrared light illumination according to a second embodiment.

  The mode for carrying out the invention is as described in Examples 1 and 2 below.

  Hereinafter, the configuration of an imaging apparatus having an IRCF switching function and a non-moving object detection function according to the first embodiment of the present invention will be described with reference to FIG.

  In FIG. 1, a subject image passes through a lens 101 composed of several lenses, and if an infrared light cut filter (IRCF) 102 is present on the optical path, the infrared component of incident light is cut off, and CCD, CMOS The image is formed on the image pickup device 103 such as, and is converted into an electric signal. The lens 101 constitutes an imaging optical system. The electrical signal output from the image sensor 103 is converted into various signals by an analog front end circuit (AFE) 104 including a correlated double sampling process (CDS), an analog gain control (AGC), an analog digital converter (ADC), and the like. Processing is done. And it outputs as digital image data (image signal).

  In the image signal output from the AFE 104, the luminance change calculation unit 105 calculates an infrared light component from the ratio of the luminance component and the color difference component, and calculates the luminance change when the IRCF 102 is removed from the optical path. Is done. Further, the image luminance calculation unit 106 calculates the luminance value of the image signal. The image signal is amplified by an auto gain control (AGC) 107 as necessary. The image signal output from the AGC 107 is subjected to predetermined image processing in an image processing unit 108 including a digital signal processor (DSP).

  The image signal processed by the image processing unit 108 is input to the unmoving object detection unit 109.

  Here, the non-moving object detection unit 109 will be described with reference to FIGS. 2 and 3 below.

  FIG. 2 is a flowchart showing details of the operation of the unmoving object detection process 109. On the other hand, FIG. 3 shows an image for explaining the unmoving object detection process. FIG. 3A is a reference image, FIG. 3B is an image currently being shot, and FIG. 3C is a current image. A difference image between a captured image and a reference image is shown.

  In step s201, an image serving as a reference for detecting an unmoving object as shown in FIG. 3A is taken and stored in the reference image storage unit 110 as a reference image. The unmoving object detection function compares the reference image with an image obtained when a new cylindrical object 300 is added to the monitoring area of the currently captured image as shown in FIG. Body detection is performed. In the non-moving object detection, the difference detection unit 111 calculates a difference value of each pixel between the currently captured image and the reference image. At this time, a large difference is detected on the newly added cylindrical object 300 as shown in FIG.

  In step s202, when the calculated difference value reaches a predetermined threshold value or more, the detection determination unit 112 determines that a new object has been added in the monitoring area.

  In step s203, the position or coordinates of the pixel area of the new object 300 are stored in the detection area storage unit 113.

  In step s204, the timer counter unit 114 counts the duration for each pixel.

  On the other hand, when the calculated difference value does not reach the predetermined threshold value in step s202, the detection determination unit 112 determines that no new object has been added to the monitoring area.

  In step s205, if the pixel area is stored in the detection area storage unit 113, it can be determined that the object is a moving object, and thus the stored pixel area is cleared.

  In step s206, the count value of the timer counter unit 114 is also cleared. Therefore, a moving object or the like is not detected as a non-moving object.

  In step s207, if the timer count is equal to or greater than a predetermined time, the area stored in the detection area storage unit 113 is detected as a non-moving object pixel area. The detection notification unit 115 notifies the user.

  Next, the configuration around the shooting mode switching unit 116 will be described.

  When the IRCF 102 is on the optical path and the luminance value calculated by the image luminance calculation unit 106 is equal to or less than a predetermined luminance value, an operation signal is transmitted from the imaging mode switching unit 116 to the IRCF driving unit 117. Therefore, the IRCF driving unit 117 operates the IRCF 102 to the outside of the optical path. Similarly, when the IRCF 102 is out of the optical path, an operation signal is transmitted from the imaging mode switching unit 116 to the IRCF driving unit 117 when the luminance value calculated by the image luminance calculating unit 106 is equal to or greater than a predetermined luminance value. Is done. Therefore, the IRCF driving unit 117 operates the IRCF 102 on the optical path. The IRCF driving unit 117 constitutes infrared light cut filter driving means.

  Further, when the shooting mode switching unit 116 switches the shooting mode, a switching signal is transmitted to the reference image switching unit 118. The reference image switching unit 118 that has received the switching signal switches to a color image (color mode) when the IRCF 102 is on the optical path. On the other hand, when the IRCF 102 is out of the optical path, it is switched to a monochrome image (monochrome mode). The reference image storage unit 110 stores a reference image for each shooting mode.

  Hereinafter, the processing operation will be described with reference to FIG.

  FIG. 4 is a flowchart of processing operations for preventing the unmoving object detection function from causing erroneous detection when the IRCF 102 operates from the optical path to the outside of the optical path.

  In step s 401, image data of the subject image is acquired through the lens 101, IRCF 102, image sensor 103, and AFE 104.

  In step s402, the luminance change calculation unit 105 predicts an infrared component from the ratio between the color difference signal and the luminance signal of the acquired image data. The amount of change in luminance when the IRCF 102 is operated is calculated from the amount of the infrared component that is the predicted value. The luminance change calculation unit 105 not only calculates the luminance change from the infrared component prediction, but also registers the predicted luminance change in the preset value of the white balance light source, and calculates the luminance change. It may be used as brightness. In addition, the first time of insertion / extraction of the IRCF 102 accepts the luminance variation and stores the luminance variation. Thereafter, the luminance fluctuation amount may be used as the calculation luminance after the second time. After the IRCF 102 operates from the optical path to the outside of the optical path, the luminance variation calculated last is held in the luminance change calculation unit 105. When the IRCF 102 is outside the optical path, the luminance change calculation unit 105 continues to hold the value.

  In step s403, the image luminance calculation unit 106 measures the luminance value of the acquired image data. The luminance value to be measured is obtained by weighted average for each part by dividing the screen. Further, it may be the average luminance average of the acquired image data on the screen.

  In step s404, if the luminance calculated by the image luminance calculation unit 106 is equal to or lower than a predetermined luminance set in advance for the operation of the IRCF 102, the IRCF 102 is operated out of the optical path in step s405. On the other hand, if the brightness is equal to or higher than the predetermined luminance, the IRCF 102 is not operated in step s406 and is left on the optical path.

  In step s407, if the luminance change calculated in step s402 is equal to or greater than the luminance threshold for detecting the unmoving object, there is a possibility that the entire screen is recognized as an unmoving object and timer counting is started. In order to prevent this, in step s408, the luminance change is corrected with a gain, and the gain is adjusted so as to reduce the luminance change, so that a sudden luminance change does not occur. That is, the gain is adjusted so as to cancel out the luminance change, so that a sudden luminance change does not occur. In the present embodiment, the luminance change is canceled out, but a rapid luminance change can be suppressed if the luminance change is reduced without canceling out.

  Thereafter, in step s409, the shooting mode switching unit 116 notifies the reference image switching unit 118 that the shooting mode has been switched. When the IRCF 102 is placed outside the optical path, the color balance is lost due to the influence of the infrared component, so that the color mode is changed to the monochrome mode. Therefore, when the shooting mode switching notification is received, the reference image switching unit 118 switches the reference image from the color image to the monochrome image in step s410.

  In step s411, a process of returning the corrected gain to the original level in accordance with the operation of the IRCF 102 is performed. The gain control will be described with reference to FIGS.

  FIG. 5 is a graph showing the relationship between the operation of the IRCF 102 and the luminance. The vertical axis represents luminance Y and the horizontal axis represents elapsed time t. On the other hand, FIG. 6A shows an operation at the time of normal switching when the gain correction processing of the first embodiment is applied. FIG. 6B shows an operation when the gain correction process is applied during the non-moving object timer count. The vertical axis represents luminance Y and the horizontal axis represents elapsed time t.

  5 and 6 are graphs when the IRCF 102 operates from the optical path to the outside of the optical path.

  As shown in FIG. 5, when the surrounding brightness gradually decreases, the luminance calculated by the image luminance calculation unit 106 becomes equal to or lower than a predetermined threshold luminance for mode switching. The IRCF 102 operates from the optical path to the outside of the optical path. As a result, the infrared component of the incident light that has been blocked until now is sensed by the image sensor 103, so that the luminance varies greatly at the IRCF switching point in FIG. For this reason, there is a possibility that the unmoving object detection unit 109 may erroneously detect the entire image as an unmoving object. Thereafter, as the surroundings become darker with time, the luminance value gradually decreases.

  FIG. 6A shows the operation when the unmoving object timer is not counted in the first embodiment.

  As in FIG. 5, when the brightness of the surroundings becomes dark and the brightness calculated by the image brightness calculation unit 106 is equal to or lower than the threshold for mode switching, the IRCF 102 moves from the optical path to the outside of the optical path. And works. At the same time, the luminance change calculated in the luminance change calculation unit 105 is corrected by the gain by the AGC 107, thereby reducing the luminance variation like the IRCF switching point in FIG. Suppresses false positives. Thereafter, the gain correction for suppressing the fluctuation in luminance is gradually returned to the original state as shown in FIG. For example, when the unmoving object detection unit 109 operates at 5 fps and the camera unit gain control operates at 30 fps, the gain for each luminance below the threshold for unmoving object detection is 6 frames. Will rise. By doing so, there is no possibility of being detected as an unmoving object, and the reference image is updated smoothly.

  FIG. 6B shows an operation during the unmoving object timer count (unmoving object detection) in the first embodiment.

  In the case of FIG. 6B, the non-moving object is detected and the timer is being counted before the imaging mode of the IRCF 102 is switched. As in the case of FIG. 6A, when the brightness of the surroundings becomes dark and the brightness calculated by the image brightness calculation unit 106 becomes equal to or lower than the threshold for mode switching, the IRCF 102 is moved from the optical path. Operates out of the optical path. And the part is correct | amended with a gain and the misdetection of the unmoving object detection part 109 is suppressed. Thereafter, since the timer is being counted, the gain is controlled so that the luminance value of the non-moving object portion, which is the difference value of the luminance difference between the image being captured and the reference image, is kept constant so as not to lose sight of the unmoving object. For example, in the case of an imaging device having automatic exposure control, the luminance target value is set as a threshold value for mode switching, so that even if the surrounding darkness is dark during the timer count, the luminance is appropriately controlled by AGC 107 Gain control is performed with the value held. After the unmoving object timer count is finished and the user detects it as an unmoving object, the gain is increased as in FIG. If the non-moving object is taken away during the non-moving object timer count, the gain is increased immediately after the timer count is cleared as in FIG.

  Next, when the IRCF 102 is operated, the image becomes a black and white image, and color noise can be suppressed.

  For example, as shown in FIG. 7, when the IRCF 102 is operated and the image becomes black and white, the white balance gain (WB gain) that is unnecessary is used to increase the brightness, and then the IRCF 102 is operated. In order to further increase the brightness. In such a case, as shown in FIG. 8A, the IRCF 102 is switched in the same manner as in FIG. Thereafter, in the process of increasing the gain, first, the gain corresponding to the IRCF operation luminance fluctuation is increased smoothly. Next, there is a technique in which the brightness is finally increased by smoothly increasing the WB gain, and erroneous detection of unmoving object detection is further suppressed. Further, as shown in FIG. 7, when the operation is performed at the time of switching the IRCF 102, as shown in FIG. 8 (b), first, the luminance increase due to the WB gain is corrected, and then the switching of the IRCF 102 is performed. Performs gain correction for brightness increase due to.

  On the other hand, in step s407, if the luminance change is less than the luminance threshold value for detecting the unmoving object, the entire screen is not recognized as an unmoving object, and therefore, no gain correction is performed in step s412.

  Hereinafter, the configuration of an imaging apparatus having an IRCF switching function and a non-moving object detection function in an imaging apparatus having infrared light illumination according to a second embodiment of the present invention will be described with reference to FIG.

  In the second embodiment, in addition to the configuration of the first embodiment shown in FIG. 1, an infrared light illumination 919 disposed around the lens is provided. In addition, an infrared light intensity measurement unit 920 that measures the luminance change when the infrared light illumination 919 is irradiated is provided. The infrared light intensity measurement unit 920 is a unit that transmits a luminance fluctuation amount set in advance according to the intensity of infrared light of the infrared light illumination 919 to the luminance change calculation unit 905. The infrared light intensity measurement unit 920 may be a measurement unit that measures infrared light.

  Hereinafter, the processing operation will be described with reference to FIG.

  FIG. 10 is a flowchart of processing operations for preventing the unmoving object detection function from causing erroneous detection when the IRCF 102 operates from the optical path to the optical path.

  Steps s401 to s405 are the same as those in the first embodiment, and a description thereof will be omitted. In the case of an imaging apparatus provided with infrared light illumination, infrared light illumination is applied in step s1006. Therefore, if it is at a short distance, it is possible to take an image that is not inferior to that of daytime. Along with this, a large change in the luminance of the image occurs, so that a gain correction corresponding to the luminance change is performed in step s408. For the luminance change, the value calculated by the luminance change calculation unit 905 that receives the value measured by the infrared light intensity measurement unit 920 is used.

  Thereafter, in step s409, the shooting mode switching unit 116 notifies the reference image switching unit 118 that the shooting mode has been switched.

  When the infrared light illumination 919 is used, the brightness in which infrared light is dominant at night becomes almost the same brightness, so an image for infrared light illumination is created in advance as a reference image.

  An image for infrared light illumination will be described with reference to FIG.

  FIG. 11 is a number line graph showing EV values, IRCF switching threshold values (threshold values for mode switching), and reference image creation EV values. For example, when there is an IRCF switching threshold at a certain EV value, an EV increase due to the infrared illumination 919 provided to the imaging device is set in advance, and the EV increase of the infrared illumination 919 from the IRCF switching threshold is preset. A black and white reference image is created when the brightness reaches around a high EV value. If it is one day, the IRCF 102 switches from on the optical path to the outside of the optical path during the night time zone. A black and white reference image created in the daytime time zone is used as the reference image after switching.

  In step s410, after the shooting mode switching notification is received, the reference image switching unit 118 switches the reference image from a color image to an image during monochrome infrared illumination.

  In step s411, a process of returning the corrected gain to the original level in accordance with the operation of the IRCF 102 is performed. The process for restoring the gain is the same method as shown in FIG.

  As mentioned above, although Example 1 and 2 of this invention were demonstrated, this invention is not limited to these Examples, A various deformation | transformation and change are possible within the range of the summary.

102 Infrared light cut filter (IRCF)
103 Image Sensor 105 Brightness Change Calculation Unit 107 Auto Gain Control (AGC)
109: Non-moving object detection unit 110: Reference image storage unit 111: Difference detection unit 116: Shooting mode switching unit 117: IRCF drive unit 118: Reference image switching unit

Claims (10)

Imaging means for converting a subject image formed by the imaging optical system into an electrical signal;
An infrared cut filter that blocks infrared light out of incident light to the imaging means;
Infrared light cut filter driving means for driving insertion / extraction of the infrared light cut filter on the optical path;
Shooting mode switching means for switching the shooting mode by inserting and removing the infrared light cut filter;
Reference image storage means for storing a reference image for each photographing mode;
Difference detection means for calculating a difference value of luminance between the reference image and the image being shot;
A non-moving object detection unit that performs non-moving object detection based on the difference value, and a luminance change calculation unit that calculates a luminance change due to insertion / extraction of the infrared light cut filter;
Gain control means for controlling the gain of the electrical signal in the imaging means,
The gain control unit adjusts the gain so as to reduce the luminance change calculated by the luminance change calculation unit when the photographing mode is switched.   The imaging apparatus according to claim 1, wherein the luminance change calculation unit predicts an infrared component based on a ratio between a luminance signal and a color difference signal of an image, and calculates a luminance change amount from the predicted value.   3. The imaging apparatus according to claim 1, wherein the luminance change calculated by the luminance change calculating means is a luminance change determined in advance by a preset value of white balance.   The photographing mode switching means switches between two modes of a color mode in which the infrared light cut filter exists on the optical path and a monochrome mode in which the infrared light cut filter does not exist on the optical path. The imaging device according to any one of 1 to 3.   The reference image storage means stores two reference images of a color image and a monochrome image when the shooting mode is the color mode, and stores a reference image of the monochrome image when the shooting mode is the monochrome mode. The imaging device according to claim 4.   The imaging according to any one of claims 1 to 5, wherein the gain control unit returns the gain adjusted so as to reduce the luminance change amount after the photographing mode is switched. apparatus.   6. The gain control unit according to claim 1, wherein the gain control means adjusts the gain so as to keep the brightness of the image being photographed constant during the unmoving object detection after the photographing mode is switched. The imaging device according to any one of the above.   When there is a luminance fluctuation due to white balance gain in addition to the luminance fluctuation due to insertion / extraction of the infrared light cut filter after the shooting mode is switched, the gain control means first calculates the luminance fluctuation due to the white balance gain. The image pickup apparatus according to claim 6, wherein the image pickup device is adjusted by a gain, and then a luminance fluctuation due to insertion / extraction of the infrared light cut filter is adjusted by the gain. Further comprising infrared illumination means,
The imaging apparatus according to claim 1, wherein the infrared light illuminating unit operates when the infrared light cut filter is not present on an optical path. An imaging step of converting a subject image formed by the imaging optical system into an electrical signal;
An infrared light cut filter driving step for driving insertion / extraction of an infrared light cut filter that blocks infrared light out of the incident light onto the optical path;
A shooting mode switching step of switching the shooting mode by inserting and removing the infrared light cut filter;
A reference image storing step for storing a reference image for each photographing mode;
A difference detection step of calculating a difference value of luminance between the reference image and the image being shot;
An unmoving object detection step of performing unmoving object detection based on the difference value;
A luminance change calculating step for calculating a luminance change by insertion / extraction of the infrared light cut filter;
A gain control step for controlling the gain of the electrical signal obtained by converting the subject image in the imaging step,
In the gain control step, when the shooting mode is switched, the gain is adjusted so as to reduce the luminance change calculated in the luminance change calculation step.

Images