JP6525723B2 - Imaging device, control method therefor, program, and storage medium - Google Patents

Description

  The present invention relates to an imaging device used for a surveillance camera or the like.

  When photographing at night by a conventional surveillance camera, visible light such as headlights of a vehicle may fall within the field of view of the surveillance camera. The outline of the car body, the license plate, and the driver's seat of the vehicle are basically dark areas because they are not illuminated by headlights. However, since the headlights become a very bright light source, a halation phenomenon may occur in which a part of the image is white-out due to the influence of entering the angle of view of the camera.

  If halation occurs, the license plate and driver's face etc. will be covered by the overexposure area caused by the headlights and become unrecognizable, or if control is made so that overexposure will not occur, the dark area will remain dark as it is, so it will be clear I can not get an image. In addition, it is possible to acquire a color image by amplifying the image signal read from the image sensor in a time zone such as dawn or evening. However, when there is a vehicle in which the headlights are on, as in the case of photographing at night, overexposure occurs due to the headlights, or control is performed so as not to cause overexposure, and parts other than the headlights become dark. As described above, when a high brightness subject is present in the image, it is necessary to change the setting to an optimal value according to the time zone or the subject.

  In order to cope with such a problem, Patent Document 1 describes a technique for performing photographing at a plurality of shutter speeds when a sudden change in luminance occurs in part of the screen. Further, Patent Document 2 describes a technology in which one optical lens is provided with two image sensors for color and monochrome, and shutter control is individually performed in accordance with an object. There is also a high dynamic range (HDR) technology, etc., in which images are captured by combining exposures on bright and dark objects and combining them to obtain an image with a wide dynamic range.

JP, 2009-302763, A JP, 2010-250710, A

  However, in Patent Document 1 above, halation can not be prevented only by changing the shutter speed, and a clear image can not be obtained for a dark region. Moreover, although the said patent document 2 can obtain a desired image, since several image sensors are needed, it is disadvantageous in terms of cost. In addition, the HDR technology also has a complicated structure and increases memory and the like, which is also disadvantageous in terms of cost.

  The present invention has been made in view of the above problems, and an object thereof is to realize an imaging device capable of automatically photographing an optimum image according to a time zone or a subject while preventing overexposure due to a high luminance subject with an inexpensive and simple configuration. It is to be.

In order to solve the above problems and achieve the object, an imaging device according to the present invention includes an imaging unit configured to capture an image, and a first filter configured to cut an infrared light component of object image light guided to the imaging unit. Means, second filter means for cutting visible light components of object image light guided to the imaging means, object detection means for detecting a predetermined object from the image taken by the imaging means, the imaging means Calculating means for dividing the image picked up by the image into a plurality of areas and calculating an average value of the brightness for each of the divided areas, and determining means for determining whether the average value of the brightness is larger than a predetermined reference value When, in response to said determining means determining result and the object detected by the object detection unit detected result, with respect to the optical path for guiding the subject image light to the image pickup means, said first filter means and said second And a control means for controlling the filter means so as to insertion.

  According to the present invention, it is possible to realize an imaging device capable of automatically photographing an optimal image according to a time zone or a subject while preventing overexposure due to a high luminance subject with an inexpensive and simple configuration.

FIG. 1 is a block diagram showing an apparatus configuration of an embodiment according to the present invention. FIG. 2 is a block diagram showing a configuration of a signal processing unit of the present embodiment. FIG. 6 is a diagram for explaining a high-brightness object detection method according to the present embodiment. FIG. 6 is a view for explaining the movement detection method of the high-brightness subject according to the present embodiment. 6 is a flowchart showing insertion and removal control of the filter of the first embodiment. 6 is a flowchart showing insertion and removal control of the filter of the second embodiment. 7 is a flowchart showing insertion and removal control of the filter of the third embodiment.

  Hereinafter, embodiments of the present invention will be described in detail. The embodiment described below is an example for realizing the present invention, and should be appropriately corrected or changed according to the configuration of the apparatus to which the present invention is applied and various conditions. It is not limited to the embodiment of the invention. Moreover, you may comprise combining suitably one part of each embodiment mentioned later.

  The following describes an embodiment in which the imaging device of the present invention is applied to, for example, a digital video camera (hereinafter, digital camera) that captures still images and moving images, but the present invention is also applicable to other surveillance cameras.

  <Apparatus Arrangement> The outline of the arrangement and functions of the digital camera according to the embodiment of the present invention will be described with reference to FIGS. 1 and 2. FIG.

  In FIG. 1, a digital camera 100 includes a lens unit 101, an image sensor 201, a signal processing unit 301, and an infrared illumination 401.

  An object image light formed by the lens unit 101 is converted into an electric signal by the image sensor 201, and then subjected to signal processing by the signal processing unit 301, and sequentially output to the outside as a moving image or a still image through a network or the like. Be done.

  The lens unit 101 includes a zoom lens 102 as an optical system, a focus lens 103, an aperture 104, a filter 105, and the like, and is controlled based on a control signal from the signal processing unit 301 so as to obtain an optimum image according to the subject. Be done. The filter 105 is provided on the optical path between the lens unit 101 and the image sensor 201 so as to be insertable into and removable from the optical axis 106 of the lens unit 101. The filter 105 includes an infrared cut filter 111 and a visible ray cut filter 112. The infrared cut filter 111 mainly cuts the infrared light component of the light passing through the lens unit 101 and entering the image sensor 201 and transmits the visible light component. The visible ray cut filter 112 mainly cuts a visible light component of the light passing through the lens unit 101 and entering the image sensor 201, and transmits an infrared light component. The insertion and removal of the lens unit 101 on the optical axis 106 can be independently controlled based on the control signal from the signal processing unit 301. When one of the filters is at the insertion position with respect to the optical axis 106, the other filter is exclusively controlled to be at the withdrawal position. In addition, any filter can be controlled in a state where it is not located on the optical axis 106.

  The image sensor 201 is formed of a semiconductor integrated circuit chip in which pixels made of photoelectric conversion elements such as CCDs or CMOSs are arranged in a matrix. The image sensor 201 has high sensitivity mainly to visible light (wavelength 380 to 780 nm), and high sensitivity to any of red (R), green (G) and blue (B) for each pixel. However, it also has some sensitivity to infrared light (780 nm or more). Therefore, for an object bright in infrared light such as a time zone in which sunlight is present and a place illuminated by infrared illumination, the object can be clearly imaged even in the state where the visible light cut filter 112 is inserted. The digital camera 100 according to the present embodiment further includes an infrared illumination 401 for projecting an infrared ray, and can emit an infrared illumination 401 toward the subject to capture an infrared image of the subject.

  <Configuration of Signal Processing Unit> Next, the internal configuration of the signal processing unit 301 will be described with reference to FIG.

  In FIG. 2, an image signal obtained by capturing an object image light by the image sensor 201 is image-processed by the image processing unit 211 of the signal processing unit 301, and a still image file of JPEG format or H.264 is processed. A moving image file encoded by H.264 or the like is generated. The image processing performed by the image processing unit 211 includes, for example, processing such as exposure control, white balance control, and gamma correction, and the signal processing unit 301 performs image processing so as to generate an optimal image according to the subject. The unit 211 is controlled. Further, the signal processing unit 301 controls the zoom lens 102, the focus lens 103, the diaphragm 104, and the filter 105 of the lens unit 101 by the lens unit control unit 212 such that the image processing unit 211 generates an optimal image. . Further, the signal processing unit 301 simultaneously controls the gain of the image sensor 201 and the shutter speed by the imaging control unit 213. For example, when the subject is bright, control is performed to stop the aperture 104, shorten the shutter speed, and reduce the sensor gain. In this case, the infrared cut filter 111 is controlled to a position inserted on the optical axis 106 of the lens unit 101, and by cutting the infrared component of the object image light, the color reproducibility of the captured image is enhanced. When the subject becomes dark, the aperture 104 is opened, and the shutter speed is controlled so as to increase the sensor gain. Furthermore, a brighter image can be obtained by retracting the infrared cut filter 111 from the optical axis 106 of the lens unit 101 and capturing light including infrared light. In this case, when infrared light is incident on the image sensor 201, the color reproducibility of the image is degraded. Therefore, the white balance control by the image processing unit 211 is usually changed to output a black and white image instead of a color image. The signal processing unit 301 determines the insertion / extraction timing of the infrared cut filter 111 by estimating and calculating the illuminance of the object using the control results of the lens unit 101, the image sensor 201, and the image processing unit 211.

  The signal processing unit 301 further includes an area luminance calculation unit 214 and a determination unit 215. The area luminance calculation unit 214 divides the image generated by the image processing unit 211 into a plurality of areas of a predetermined size, and calculates a luminance average value for each of the divided areas. The determination unit 215 compares the luminance average value for each area calculated by the area luminance calculation unit 214 with the reference value L0 (for example, 250), and if it is larger than the reference value L0 determined as the predetermined threshold, It is determined that a high brightness subject exists in the Thereby, it is possible to detect whether or not a high brightness object such as a headlight of a vehicle is present in the image. Detailed operations will be described later.

  The signal processing unit 301 further includes a subject detection unit 216. The subject detection unit 216 performs subject detection processing on the image data generated by the image processing unit 211 by a known method such as pattern matching to extract feature amounts, and detects a predetermined subject present in the image. . The predetermined subject is, for example, a license plate of a vehicle, a face of a person such as a driver of a vehicle or a passerby. The detection results by the subject detection unit 216 are stored as additional information related to the feature amount of the subject in association with the image file, or stored in the memory 219 in the signal processing unit 301.

  The memory 219 may be a recording medium such as a hard disk drive or a memory card which can be built in or externally attached to the digital camera. In addition, the memory 219 may be configured to store a moving image or a still image that is externally output.

  The signal processing unit 301 further includes a communication control unit 217. The communication control unit 217 outputs the image data generated by the image processing unit 211 to the outside via a wireless or wired network. In this case, a luminance average value as a calculation result by the area luminance calculation unit 214, a determination result by the determination unit 215, or a feature amount of a subject as a detection result by the subject detection unit 216 may be output as control information. Furthermore, as the control status of each block of the digital camera, the insertion / removal state of the filter 105 may be output as control information. A control signal of each block of the digital camera may be received from an external device such as a personal computer via the communication control unit 217.

  The signal processing unit 301 further includes a communication control unit 217. The communication control unit 217 outputs the image data generated by the image processing unit 211 to the outside via a wireless or wired network. In this case, the feature amount of the subject as the detection result of the subject detection unit 216 may be simultaneously output.

  The signal processing unit 301 further includes a lighting control unit 218. The illumination control unit 218 performs on / off control and dimming control of the infrared illumination 401. When the infrared illumination 401 is to be turned on, the infrared cut filter 111 is retracted from the optical axis 106 by the lens unit controller 212. The signal processing unit 301 is connected to the above-described units and the memory 219 via a bus 220, exchanges various data, and controls the units.

  Note that one or more of the units illustrated in FIG. 2 may be realized by hardware such as an ASIC or a programmable logic array (PLA), or may be realized by execution of software by a processor such as a CPU or an MPU. Good. Also, it may be realized by a combination of software and hardware. Therefore, in the following, even when different functional blocks are described as the operation subject, the same hardware may be realized as the subject.

  <Region Luminance Calculation Unit and Determination Unit> Next, processing by the region brightness calculation unit 214 and the determination unit 215 of the signal processing unit 301 will be described with reference to FIG.

  Each pixel value of the image data generated by the image processing unit 211 includes light amount information received by each pixel as luminance information. FIG. 3A shows an image of the subject generated by the image processing unit 211 when the vehicle V, which is one of the subjects, turns on the headlights. For this image, the area brightness calculation unit 214 first divides the image into a plurality of areas in an NxM matrix as shown in FIG. 3B, and calculates the average value of the brightness from the pixel values in the areas for each area. Do. N and M are positive integers, respectively, and the divided regions are selected to be square or rectangular. Although the values of N and M vary depending on the size of the image, it is desirable to select one side of the divided area to be several tens to several hundreds of pixels. The values of N and M differ depending on the angle of view of the camera, and when photographing at a wide angle, it is preferable that the numerical value be small because the subject often appears small in the screen. On the other hand, when photographing on the telephoto side, the subject is likely to appear large, so it is preferable that the numerical value be large.

  For each region divided as described above, the luminance average value of the i-th position in the horizontal direction and the j-th position in the vertical direction (i, j) is Li j (i = 1, 2,... N, j = 1, 2,. M). Thereafter, the determination unit 215 compares the luminance average value Lij of each area with a predetermined reference value L0, and if the luminance value of the compared area is brighter than the reference value L0, the position (i, It is determined that a high brightness subject exists in j). As the reference value L0, a value when the luminance value of the pixel in the image is saturated or a value close thereto is used, and when the luminance value is 8 bits, a value of about 200 to 255 in decimal is used. Is preferred.

  When the determination unit 215 determines that a high brightness object is present, the signal processing unit 301 controls the lens unit control unit 212 to insert the visible ray cut filter 112 on the optical axis 106 of the lens unit 101. In this way, it is possible to obtain the image of only the infrared component by cutting the visible light component of the object image light. In the present embodiment, the headlights of the vehicle V assumed as a high brightness object are often used in recent years from the viewpoint of power consumption and light emission efficiency such as LEDs and HID lamps that emit a large amount of visible light. It has become. Such a light source is not as bright as visible light in the infrared wavelength region, and therefore it is difficult for whiteout of the image to occur. In such a situation, when the infrared cut filter 111 is inserted on the optical axis 106 of the lens unit 101, the infrared wavelength region of the object image light is not incident on the image sensor 201. Therefore, the infrared cut filter 111 is retracted from the optical axis 106 substantially simultaneously with the insertion timing of the visible light cut filter 112.

  Note that if the visible ray cut filter 112 is frequently inserted and removed, the brightness and the white balance change each time, and the appearance of the monitoring image is degraded. Therefore, control may be performed so as to insert and remove only when the subject detection unit 216 detects a predetermined subject.

  FIGS. 4 (a1) and 4 (a2) illustrate the case where a light source such as a street lamp G is present in the captured image, and FIGS. 4 (b1) and 4 (b2) illustrate FIGS. 4 (a1) and 4 (a2). A method of determining a high luminance object from the image of FIG. Assuming that the determination unit 215 determines that only the headlights of the vehicles V1 and V2 are high-brightness objects in the images of FIGS. 4 (a1) and 4 (a2), a plurality of images are continuously photographed, The position information of the subject detected from the image of is stored in the memory 219. Then, by detecting whether or not the position information of the subject stored in the memory 219 has changed, for example, in a continuous image, the vehicle V1 of FIG. 4 (a1) and the vehicle V2 of FIG. 4 (a2) Thus, it can be determined that the high brightness subject is moving.

  Next, the operation of the digital camera 100 when the visible ray cut filter 112 is inserted on the optical axis 106 of the lens unit 101 will be described.

  When the visible ray cut filter 112 is inserted on the optical axis 106 of the lens unit 101, only the infrared component is imaged in the object image light, and therefore the image is generally dark. Therefore, it is desirable to light the subject image by turning on the infrared light 401.

  In addition, when the visible light cut filter 112 is inserted, the visible light component is not incident on the image sensor 201, so the brightness in the wavelength region of the visible light of the subject becomes unknown, and it is determined whether a high brightness subject exists. It may not be possible. Therefore, in the present embodiment, when a predetermined time (about several seconds to several hundred seconds) elapses after the visible light cut filter 112 is inserted, the visible light cut filter 112 is retracted again, and the visible light component is sent to the image sensor 201. Control to be incident. Alternatively, the visible ray cut filter 112 may be retracted when a predetermined time has elapsed after a predetermined subject is not detected by the subject detection unit 216. Alternatively, a sensor such as an illuminance sensor capable of detecting the illuminance of the subject may be used.

  <Description of Operation> Next, filter insertion / removal control in the monitoring camera mode of the digital camera 100 according to the present embodiment will be described with reference to FIGS. 1, 2 and 5. FIG. In the digital camera 100 of the present embodiment, the image processing unit 211 generates image data of JPEG format in 1920 × 1080 pixels, and sequentially outputs the image data from the communication control unit 217 to the outside.

  The process of FIG. 5 is realized by reading out the program stored in the memory 219 and executing each part of the signal processing unit 301. The process in FIG. 5 is executed when the digital camera 100 is set to the surveillance camera mode, an image is captured, and the image is output to the outside. The same applies to FIGS. 6 and 7 described later.

  In step S501, the area brightness calculation unit 214 acquires the image data generated by the image processing unit 211.

  In step S502, the area luminance calculation unit 214 divides the image data obtained from the image processing unit 211 into areas of 60 × 60 pixels, and calculates the average luminance value of each area. In addition, the area luminance calculation unit 214 stores the average luminance value of each area in the memory 219.

  In step S503, the determination unit 215 compares the average brightness value for each area calculated by the area brightness calculation unit 214 in step S502 with a reference value L0 which is a predetermined threshold. As a result of comparison, if there is at least one area larger than the reference value L0, the process proceeds to step S504, and if not, the process returns to step S501.

  In step S504, in step S503, the determination unit 215 stores the position of the area where the average luminance value is larger than the reference value L0 in the memory 219 as position information where the high luminance object is present.

  In step S505, when the infrared cut filter 111 is present on the optical axis 106 of the lens unit 101, the lens unit control unit 212 retracts the visible light cut filter 112 from the optical axis 106 of the lens unit 101. Insert on top.

  In steps S506 and S507, the lens unit control unit 212 stands by until a predetermined time (for example, 30 seconds) elapses, and then retracts the visible ray cut filter 112 from the optical axis 106 of the lens unit 101 again. It returns to S501.

  As described above, by controlling the insertion and removal of the infrared cut filter and the visible light cut filter, it is possible to automatically shoot an optimal image according to the time zone and the subject while preventing the whiteout due to the high brightness subject. For example, when photographing at night with a surveillance camera or a car-mounted camera, visible light components contained in light such as headlights are reduced by the visible light cut filter, so the influence of the headlights is suppressed and the license plate is It will be recognizable above.

  Second Embodiment Next, filter insertion / removal control in the monitoring camera mode of the digital camera 100 according to the second embodiment will be described with reference to FIGS. 1, 2 and 6. FIG.

  The configuration of the digital camera 100 for realizing the present embodiment is the same as that of FIG. 1 and FIG. 2 of the first embodiment, and the image processing unit 211 generates image data of JPEG format of 1920 × 1080 pixels. Output externally.

  Steps S601 to S604 in FIG. 6 are the same as steps S501 to S504 in FIG. 5, so the description thereof is omitted, and differences will be mainly described.

  In step S605, the determination unit 215 determines whether or not the position information of the high-brightness subject is stored in the memory 219 for two or more consecutive images. If it is stored, the process proceeds to step S606, and if it is not stored Returns to step S601.

  In step S <b> 606, the determination unit 215 compares position information of high-brightness subjects in two or more images stored in the memory 219. If the position information of the high-brightness object of each of the compared images is adjacent within a predetermined distance range, it is determined that the high-brightness subject is moving, and the process proceeds to step S607 and it is not determined that it is moving. In the case, the process returns to step S601.

  Steps S607 to S609 perform the same processing as steps S505 to S506 in FIG.

  As described above, according to the present embodiment, in addition to the effects of the first embodiment, filter control can be performed only on a moving high-brightness subject.

  [Third Embodiment] Next, filter insertion / removal control in the monitoring camera mode of the digital camera 100 according to the third embodiment will be described with reference to FIGS. 1, 2 and 7. FIG.

  The configuration of the digital camera 100 for realizing the present embodiment is also the same as in FIGS. 1 and 2 of the first embodiment, and the image processing unit 211 generates image data of JPEG format of 1920 × 1080 pixels, and the communication control unit 217 continuously Output externally. Further, it is assumed that the subject detection unit 216 detects a license plate of a vehicle as a predetermined subject.

  Steps S701 to S705 in FIG. 7 are the same as steps S501 to S505 in FIG. 5 and thus the description thereof is omitted, and differences will be mainly described.

  In step S706, the subject detection unit 216 determines whether the license plate of the vehicle is detected as a predetermined subject from the image generated by the image processing unit 211. If it is detected, the process proceeds to step S707 and is not detected. In the case, the process proceeds to step S 708.

  In step S 707, the lens unit control unit 212 stands by until a predetermined time (for example, 30 seconds) elapses.

  In step S708, the lens unit control unit 212 retracts the visible ray cut filter 112 from the optical axis 106 of the lens unit 101 again, and the process returns to step S701.

  As described above, according to the present embodiment, in addition to the effects of the first embodiment, the insertion of the visible ray cut filter is continued while the predetermined high-brightness object is detected, and the detection is no longer performed. It can be evacuated.

Other Embodiments
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. Can also be realized. It can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

DESCRIPTION OF SYMBOLS 101 ... Lens unit, 111 ... Infrared cut filter, 112 ... Visible light ray cut filter, 201 ... Image sensor, 211 ... Image processing part, 212 ... Lens unit control part, 214 ... Area brightness calculation part, 215 ... Determination part, 216 ... Subject detection unit, 219 ... memory, 301 ... signal processing unit

Claims (12)

Imaging means for capturing an image;
First filter means for cutting an infrared light component of subject image light guided to the imaging means;
Second filter means for cutting visible light components of subject image light guided to the imaging means;
A subject detection unit that detects a predetermined subject from an image captured by the imaging unit;
Calculating means for dividing an image captured by the imaging means into a plurality of areas and calculating an average value of luminance for each of the divided areas;
A determination unit that determines whether the average value of the brightness is larger than a predetermined reference value;
Depending on the determination means a determination result and the object detected by the object detection unit detected result, with respect to the optical path for guiding the subject image light to the image pickup means, said first filter means and the second filter An image pickup apparatus comprising: control means for controlling the insertion and removal of the means. Wherein when the average value before Kiteru degree is greater than the predetermined reference value, said first filter means is retracted from the optical path, so as to insert the second filter means in the optical path The image pickup apparatus according to claim 1, wherein the image pickup apparatus is controlled to It further comprises storage means for storing position information of an area where the average value of the brightness is larger than the predetermined reference value,
The control means is configured to perform the first filter means when position information of an area in which it is determined that the average value of luminance is larger than a predetermined reference value is adjacent to each other for two or more consecutive images. 3. The image pickup apparatus according to claim 1, wherein the second filter means is controlled to be inserted into the optical path. The control means is controlled to retract the second filter means from the optical path after a predetermined time has elapsed since the second filter means was inserted into the optical path. The imaging device according to any one of to 3 .   5. The control method according to any one of claims 1 to 4, wherein, when the predetermined object is not detected by the object detection unit, the control unit causes the second filter unit to retract from the light path. An imaging device according to item 5. 6. The image pickup apparatus according to claim 5 , wherein the subject detection unit detects a license plate of a vehicle as the predetermined subject. 6. The image pickup apparatus according to claim 5 , wherein the subject detection unit detects a face of a person as the predetermined subject. The first filter means and the second filter means are detachably provided on an optical path between the imaging means and an optical system for forming an image of a subject. The imaging device according to any one of 7 . The apparatus further comprises communication means for outputting an image captured by the imaging means to an external device,
The image pickup apparatus according to any one of claims 1 to 8 , wherein the communication unit outputs control information including a determination result of the determination unit or a detection result of the subject detection unit together with an image. An imaging means for picking up an image; a first filter means for cutting an infrared light component of the subject image light guided to the imaging means; and a visible light component for the object image light guided to the imaging means A control method of an image pickup apparatus having a second filter unit,
Detecting a predetermined subject from an image captured by the imaging unit;
Dividing an image captured by the imaging unit into a plurality of regions, and calculating an average value of luminance for each of the divided regions;
Determining whether the average value of the brightness is greater than a predetermined reference value;
Depending on the detection result of the result of the determination and before Symbol the Utsushitai, to light path for guiding the subject image light to the imaging means, so as to insertion of said first filter means and said second filter means And controlling the image pickup apparatus. The program for making a computer perform the control method of the imaging device of Claim 10 . A computer readable storage medium storing a program for causing a computer to execute the method of controlling an imaging device according to claim 10 .

Images