WO2019157096A1

WO2019157096A1 - Surveillance camera for day-night transitions and method therefor

Info

Publication number: WO2019157096A1
Application number: PCT/US2019/016912
Authority: WO
Inventors: Richard CROOK
Original assignee: Sensormatic Electronics, LLC
Priority date: 2018-02-08
Filing date: 2019-02-06
Publication date: 2019-08-15

Abstract

The different camera settings and scenes leads to a wide number of parameters that affect exactly when the Day to Night or Night to Day transitions occur, including those that are defined by the scene and not the camera (light level, color of the lighting, etc.). Further, the IR illumination source adds yet another parameter to the configurations. The variability in conditions means that transitions can be difficult to control consistently. The system and method here make on-site day to night and night to day setup easier for the installer/user due to simple controls and is more predictable and consistent compared to existing methods.

Description

SURVEILLANCE CAMERA FOR DAY-NIGHT

TRANSITIONS AND METHOD THEREFOR

RELATED APPLICATIONS

r oooi] This application claims the benefit under 35 U.S.C. § 1 19(e) ofU.S. Patent Application No. 62/627,979, filed on February 8, 2018, which is incorporated herein by reference in its entirety.

BACKGROUND OF THE INVENTION

0002 ] Surveillance cameras, including closed circuit television (CCTV) cameras, can be installed in a wide range of different situations - indoors and outdoors. As a result, they are often required to function in low light conditions. This requires the camera to change from color or Day mode to black and white (b/w) or Night mode.

[ 0003 ] Switching modes often requires the automatic removal of an infrared (IR) cut- filter from the optical path to improve the low light capability. The IR cut-filter blocks infrared light, to which the cameras' image sensors are sensitive, during daytime in order to allow the cameras to provide a more accurate color image when operating in color or Day mode.

[ 0004 ] At the same time, many modem surveillance cameras have illumination systems for illuminating the scenes from which the surveillance cameras capture images. Often, these illumination systems are activated at night so that parking lots or darkened hallways can be monitored even in the absence of other light sources. It is also common that these surveillance camera illumination systems used infrared diodes to illuminate the scene. Diodes are very power efficient. And, the use of infrared light illumination ensures that the cameras do not have a detrimental effect on individuals within the scene. The use of infrared illumination also reduces the obtrusiveness and visibility of the surveillance cameras. As a result, when switching into b/w or Night mode with IR illumination, the IR. cut-filter must be removed so that the image sensors of the surveillance cameras can receive the IR light of the illumination systems.

r o o o 5 j Surveillance cameras can be used to monitor wi dely differing scenes with different overall requirements for each scene. If monitoring quickly moving objects it is likely that the cameras' shutter speeds should not drop below about 1/30 second(s) , or if monitoring a normally static scene it may be preferable to maintain color mode in as low light as possible, possibly even letting the shutter speed drop to l/4s.

SUMMARY OF THE INVENTION

r 0006 ] The different camera settings and scenes leads to a wide number of parameters that affect exactly when the day to night or night to day transitions occur, including those that are defined by the scene and not the camera (light level, color of the lighting, etc.). Further, the IR illumination source state is another parameter for the configurations. The variability in conditions means that different transitions can be difficult to control consistently.

[ 0007 ] As the light level falls in the scene, the transition point to night mode (b/w) should change depending on whether the requirement is for fast shutter speed (reduced motion blur), or best image quality (reduced maximum gain). If a separate light sensor is used to determine the light level (Lux), then the trigger level could be altered depending on the shutter speed requirement, gain requirement and zoom level, but this would require significant development/time to tabulate all the possible combinations and Lux levels. Or if a set Lux level is used, regardless of the camera configurations)! will produce varying image qualities depending on the setup. Neither of these methods is deemed acceptable. Also, the light sensitivity of light sensors is not necessarily the same as the image sensor used in the camera, which can lead to differing responses due to different light types (Halogen is high in IR content, Fluorescent is low in IR content).

[ 0008 ] Thus, the present approach selects the day to night (and night to day) transition point simply based on the image quality parameters defined by the camera configurations. For example, the gain setting of image sensor, shutter speed of the image sensor, and iris setting can be used to control day to night and night to day transitions and color and black and white modes. In addition, analysis of the ratios of red, blue and green gain values from the image sensor provides an estimate color of the light reaching the image sensor and is further used as a basis for the transitions.

r 0009 ] In general, according to one aspect, the invention features a method for controlling day-night Transitions in a surveillance camera. The method comprises analyzing combinati ons of gain setting of an image sensor, shutter speed of the image sensor, and iris setting to control day to night and night to day transitions and color and black and white modes

[ 0010] The method further preferably includes analyzing ratios of red, blue and green gain values from the image sensor to estimate color of the light reaching the image sensor.

[ ooii] By analyzing ratios of red, blue and green gain values, it can be determined if infrared light is present at a level that would require delay of transition back to day mode because color and/or intensity are unsuitable for day mode. The analysi s of ratios of red, blue and green gain values preferably supplements analysis the gain setting, shutter speed, and iris setting.

[ 0012 ] An auxiliary light sensor can be used to validate a level of visible light present. This auxiliary light sensor might be used as a secondary⁷ source after analysis of the gain setting, shutter speed, and iris setting and ratios of red, blue and green gain values.

[ o o 13 ] The method can further compri se the controller changing an intensity infrared illumination generated by the camera when in black and white mode.

[ 0014 ] In general, according to another aspect, the invention features surveillance camera that controls day-night Transitions The camera comprises an image sensor for generating images of a scene, a lens system for forming an image of light from the scene on the image sensor, and a controller that analyzes combinations of gain setting of the image sensor, shutter speed of the image sensor, and iris setting of the lens system to control day to night and night to day transitions and color and black and white modes. r o o 15 j The above and other features of the invention including various novel details of construction and combinations of parts, and other advantages, will now⁷ be more particularly described with reference to the accompanying drawings and pointed out in the claims. It will be understood that the particular method and device embodying the invention are shown by way of illustration and not as a limitation of the invention . The principles and features of this invention may be employed in various and numerous embodiments without departing from the scope of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

[ 0016] In the accompanying drawings, reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale; emphasis has instead been placed upon illustrating the principles of the invention. Of the drawings: [ 0017 ] Fig. 1 is a schematic of a security camera system to which the present invention is applicable;

[ 0018 ] Fig. 2 is a plot of gain-shutter value (GS) value, log scale as a function of luminous flux per unit area (Lux) log scale; and

[ 0019 ] Fig. 3 is a flow diagram illustrating the operation of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[ o 020 ] The invention now will be described more fully hereinafter with reference to the accompanying drawings, in which illustrative embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[ 0021 ] As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items. Further, the singular forms and the articles "a", "an" and "the” are intended to include the plural forms as well, unless expressly stated otherwise. It will be further understood that the terms: includes, comprises, including and/or comprising, when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements,

components, and/or groups thereof. Further, it will be understood that when an element, including component or subsystem, is referred to and/or shown as being connected or coupled to another element, it can be directly connected or coupled to the other element or intervening elements may be present.

[ 0022 ] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consi stent with their meaning in the context of the relevant art and will not be i nterpreted in an idealized or overly formal sense unless expressly so defined herein.

[ 0023 ] Fig. 1 illustrates an exemplary surveillance camera 100 to which the present invention is applicable. [ 0024 ] As is common, the surveillance camera 100 comprises an imaging system 115. This imaging system 1 15 captures frames that contain images of the scene 210 Light from the scene 210 passes through a transparent front cover 117 of the camera system body 125.

[ 0025] The light passes through an infrared (IR) cut-filter 160 that can be moved into and out of the optical path to improve the low light capability by a filter actuator 161

[ 0026] The light is imaged by an imaging lens system 120 on to an imager chip or sensor 122. An iris 125 controls the level of light reaching the chip sensor.

[ 0027 ] As is also common, the surveillance camera 100 often further comprises an infrared illumination system 124. This illumination system 124 comprises one or more infrared light emitting diodes (LEDs) 116 that are typically arrayed around the entrance aperture of the imaging system 115 of the surveillance camera 100.

[ 002 8 ] The frames containing the images of the scene 210 that are generated by the image sensor 122 are provided to a camera controller 110. In some examples, the controller 110 performs video analytics on the images captured by the image sensor 122. Moreover, typically, the controller 110 forwards the frames to a central control or video management system 50. Often, the images or video images are stored on a video management system (VMS) or further analyzed by the central control system 50 and/or processed by video analytics systems and/or viewed by operators such as security guards.

[ 0029] The frames from the surveillance camera 100 are typically transferred to the central control system through a network. In one example, the surveillance camera system 100 is provided with a network interface such as an Ethernet internet protocol (IP) interface 126 that allows the network surveillance camera 100 to be connected to a data network, such as a local area or enterprise network, to allow communication between the central control system 50 and the controller 110 of the surveillance camera 100.

[ 0030 ] Power is provided to the surveillance camera 100, including the controller 1 10, through a power interface 128. In some examples, this is a separate connector located on the body 125 of the surveillance camera 100. In other examples, the power is provided on a power over Ethernet (POE) interface that is combined with the network interface 126.

[ 0031 ] The controller 1 10 controls the power provided to the infrared LEDs 116 of the infrared illumination system 124 via an infrared driver 112 that is also typically powered via the power interface 128. The controller 110 also controls the iris position via an iris control line, a zoom position of the lens system 120 via a zoom control line, the gain of the imager chip 122 and the shutter speed of the imager chip 112 via a gain and shuter control/data transfer line. Further, the controller 110 can also move the IR cut-filter 160 into or out of the opti cal path via the IR filter control line to the fil ter actuator 161.

r 0032 ] In general, the controller 1 10 analyzes the images of the frames generated by the image sensor 122 and controls the power provided to the infrared illumination system 124, along with the zoom position, gain and shutter speed, iris position and IR cut-filter position based on the analysis of those images.

[ 0033 ] In general, day to night changes based simply on Lux (light) level will produce widely varying results depending on the light type, the scene 210, and whether an imaging lens system 120 is at wide angle or telephoto setting. Changes based simply on shutter speed may not give the optimum result if the trigger point is changed from slow to fast shutter speed. The use of a number that uses data from gain, shutter and iris position and is independent of the individual gain, shutter or iris configurations will produce a consistent and predictable transition regardless of how the camera is configured.

[ 0034 ] The present camera 100 does not rely on attempting to change from Day mode to Night mode at a specific luminous flux per unit area (Lux) level. Instead, the camera 100 can change at a defined image quality. The live values for gain, shutter and iris can be combined into a single number. To handle different requirements image motion or frame to frame changes in the captured images, the shutter speed may be held higher, than otherwise necessary, to reduce blurring in the images from motion - a shutter speed of 1/100s will have less motion blur than for a shutter speed of l/10s for exactly the same scene, lighting and motion.

[ 0035 ] However, at 1/100s there will need to be either more light to account for the faster shutter, or more gain, or the iris needs to open more. The use of shutter speed as a time in milliseconds, gain as a multiply factor and iris as a percentage or fraction of the fully open position allow the numbers to simply be multiplied together to give a single value (l/25s=40 milliseconds (ms)) gain of 6db= x2 multiply factor, iris wide open =

100%.

[ 0036 ] For any particular light level and scene there are a wide number of different combinations of gain, shuter and iris that will result in the same acceptable exposure. For this particular light level all of the combinations of gain, shutter and iris result in the same number. For example:

l/25s (40ms), gain xlO = 400

l/50s (20ms), gain x20 = 400

l/lOOs (10ms), gain x40 = 400

[ 0037 ] Therefore, the use of this gain, shutter and iris number (GSI) allows day to night and night to day changes to be at a consistent, expected image quality regardless of the actual combination of gain, shutter and iris used. Transition will be at the same point even if the user alters the controls governing the way that the exposure algorithm works.

[ 0038 ] Night to day transitions can use exactly the same process, although at a different GSI number. The use of Luma levels, which is the weighted sum of gamma- compressed Red, Green, and Blue components of the image from the image sensor 122 and measurements of the Red, Green and Blue levels in the image can be used to determine if the light level is suitable to allow the change back to color mode. This is essential for night to day transition because different lighting types have different IR and visible light properties and the transition back to color mode when there is insufficient visible light present will result in a very dark or black image for a few seconds while the exposure algorithm changes back to night mode

[ o o 39 ] The power (intensity) of infrared illuminati on from the LEDs 1 16 built into the camera 100 can also be adjusted dynamically to help maintain the image quality, instead of being just on or off

[ 0040] In any event, the present approach uses the combination of the gain, shutter and iris together in the transition decision both for day to night and night to day. In addition the inclusion of dynamically variable infrared illumination intensity or porver in black and white exposure management and the Luma value and RGB values are used in the decision process for when to change back from night to day mode.

[ 0041] The internal firmware executed by the controller 110 of the camera 100 can show the live parameters that the camera is using at that specific time - gain, shutter speed, iris position, IR light power output (if present). Exposure algorithms will alter some or all of these parameters to control the image exposure. These parameters will change depending on the light level, light type, zoom position. Therefore, comparison of the live values to defined thresholds can then result in consistent image quality for Day to Night or Night to Day transition. Also, if the scene 210 changes in any way (zoom level or whatever the camera is looking at) then the gain, shutter speed, iris position and IR values will be changed by the controller 1 10 to accommodate and the Day to Night or Night to Day transitions, while probably at different ambient Lux levels, will remain at consistent camera parameters.

[ 0042 ] The live gain and shutter speed parameters issued by the controller 1 10 and actually being used (and iris position) show how‘hard’ the auto-exposure algorithm is having to work to produce an accepted image brightness. Because this is from the live scene it is independent of any ambient lighting and is also independent of zoom level - when a lens is moved from wide angle to telephoto there is a decreasing amount of light reaching the image sensor 122 and so gain must increase or shutter speed get slower or the iris must open more to accommodate the change in received light level.

[ 0043 ] For example, at a given light level, a camera may use gain of xlO (not a dB number) and shutter speed of 1 /25s. Converting shutter speed into an exposure time in milliseconds, l/25s becomes 40 milliseconds (ms). Multiplying gain by shutter speed gives 400 in this example. If the exposure algorithm was to use l/50s (20ms), for exactly the same scene and light level, the gain would become x20 (half of the exposure time requiring a doubling of the gain) to maintain the same image brightness. So, the gain-shutter value is essentially independent of the actual gain and shutter values used by the Auto Exposure algorithm (depending on the camera exposure mode). The following numbers represent different ways for the Exposure Algorithm to show the same scene brightness assuming that the iris does not change

l/25s (40ms), gain xlO = 400

l/50s (20ms), gain x20 = 400

1/lOOs (10ms), gain x4Q = 400

[ 0044 ] If the received light level increases then the auto exposure (AE) algorithm wall alter gain and shutter (and potentially iris position) to maintain the same displayed image brightness. For example, an increase in lighting level could cause the AE algorithm to reduce gain (l/25s at xlO to l/25s at x5). This results in a different (lower) gain-shutter value (GS) for the new scene (40ms and gain x5 = 200). [ 0045] If the received light level decreases then AE may cause gain to increase or it could cause the shutter speed to be slower or the iris to open more.

l/25s (40ms) at gain xlO could become

l/25s (40ms) at gain x20, (GS=800) or

1/12.5s (80ms) at gain xlO, (GS=800)

[ o o 46] As the received light level decreases the GS value increases and as the received light level increases the GS value decreases.

[ 0047 ] In the brightest condition that the camera can handle, shutter speed will be as fast as the camera is capable (for example l/5000s = 0.2ms) and gain will be as low' as possible (OdB) xl giving a GS value of 0.5. At the other extreme, in total darkness the shutter speed will be as slow as allowed and the gain as high as allowed, for example l/4s (250ms) and 48dB gain (x256) giving a GS value of 64000.

[ 0048 ] Fig. 2 is a graph that plots the GS number (gain as multiplier times shutter speed in milliseconds) for light levels from about 500 Lux to O.OlLux for both halogen (see line 282) and fluorescent (see line 280) lighting. It shows a predictable increase of GS as the light level falls. It show's that at a set GS value there is a specific Lux level, so altering the ratios of gain and shutter, w'hile still maintaining the same GS value

(40xI0^:=^:20x20==5x80), will always be at the same light level as perceived by the camera 100

[ 004 9 ] The iris 125 is another integral part of this. The above descriptions and calculations assume that the iris is not changing. This is also applicable if the camera/lens has no iris (equivalent to a fixed iris). When present, the iris adds another factor to the exposure equation. A steady exposure can be maintained in varying light levels if gain and shutter remain fixed provided an iris can open or close appropriately to keep the amount of light reaching the image sensor at a constant level.

[ 0050 ] Define iris open (or fixed iris) as 100% and use this as a third factor in the equation. A moveable iris could then close towards a minimum practical opening size (e.g., 30%). This percentage opening is the third factor.

l/25s (40ms), gain xlO, iris 100(%) ^::: 40000

I /50s (20ms), gain x20, iris 100(%) = 40000 l/25s (40ms), gain x20, iris 50(%) = 40000

[ 0051 ] The GSI value achieved can be plotted from darkness to very bright and for any specific brightness the combination of gain, shutter and iris will be the same regardless of what shutter speed is used, what gain is allowed or the iris position - assuming the same exposure (luma) level.

[ 00 52 ] This process is principally targeted at day to night and night to day transitions. As these are most commonly in overall low lighting conditions it is likely that any moveable iris will be in the max open position (100(%)) and so the factors can be simplified to just g and s. However, as the iris position (opening) affects the depth of field it is conceivable that the iris is not guaranteed to be wide open.

[ 0053] Day to Night transition point

[ 0054 ] If a camera is limited to a slowest possible shutter speed of l/25s (40ms) and a maximum gain of 42db (xT28) then whenever a decreasing light level causes the exposure to reach l/25s and 42dB any further decrease in light level will cause the image to become darker because the exposure algorithm can no longer adjust shutter speed any slower or gain any higher to maintain the desired image brightness.

[ 0055] Therefore, if Day to Night transition is triggered by a selected GSI value then for a fixed light level the change will happen at that same light level regardless of whether shutter is l/25s or l/50s. Use of the camera’s selected maximum gain and slowest shutter parameters can immediately define a GSI value beyond which the image will simply get darker as light level falls. Triggering Day to Night transition when GSI reaches this value will ensure that the camera only changes to night mode (black and white) when it cannot maintain a color image without becoming darker based on the shutter, gain and iris configurations.

r 0056 ] Selection of a GSI number that is not the maximum possible for the camera will raise the light level at which the transition to night mode will happen. Depending on the set-up of the exposure algorithm this can mean a lower gain level at the transition point or a faster shutter speed

[ 0057 ] If the camera/1 ens has an IR-cut-filter 160, it will be removed whenever the Day to Night transition happens. This will generally increase the received light level on the image sensor 122 (altering the gain and shutter values) which alters the GSI value for that light level. Account must be taken for the GSI potential increase immediately after changing to night mode. The increase is partly from the removal of the filter 160 simply letting more visible light through but also from the spectral content of the light - higher IR content (Halogen lighting) would cause a larger GSI value change than lower IR. content (Fluorescent).

[ 0058 ] A color image at l/25s (40ms) at gain xlO (GS=400) may change to l/25s (40ms) at gain x5 (GS=200) when the cut filter 160 is removed and the image is changed to black and white or Night mode.

[ 0059] As the light level falls further, the GSI value will continue to increase until the settings are at the maxima allowed after which the black and white image will start to get darker. This is the point beyond which the camera cannot adjust the exposure any more.

[ o o 6 o ] Built-in IR illumination

[ 0061 ] It is not necessarily always the case that when a camera changes to black and white mode that IR is immediately required - consider a slowly decreasing light level (daylight). If the camera has built in IR illumi nation a secondary GSI value can be selected, only for use in black and white, at which point the IR illumination can be turned on.

[ 0062 ] With power controlled IR illumination, the controller 110 via the IR driver 1 12, the level can be adjusted to maintain a selected GSI value as the light level decreases. This could be done either by turning on all the IR illumination to maximum power and then decreasing it until the GSI value increased past the value just before the IR was turned on. Another alternative is to slowly increase the IR output from zero until the desired GSI value is regained. Beginning by turning on at maximu power would make a Day to Night transition faster if the local lighting was just turned off (internal scene in a room where the lights are turned off).

[ o 063 ] As the received light level increases and the GSI value decreases, the IR output could be decreased to maintain the desired GSI value until all IR illumination was off. At this point the GSI value would need to drop by a determined amount (increasing light level) before the camera would consider changing back from black and white to colour.

This amount would have to take into account the GSI difference for IR cut filter in place (colour) compared to IR cut filter removed (black and white) and also an additional light level increase to prevent changing back to night mode (hysteresis). [ 0064 ] In cameras without built in IR illumination the process for changing back from black and white to color (Night to Day transition) is exactly the same as for cameras with IR illumination whenever the GSI value is low enough to warrant changing back from night to day

r 0065 ] In night mode, in can be challenging to know how to determine if the light received is IR or visible. A high amount of IR received from another source (separate IR illuminator) or something that emits a lot of IR content (the sun or Halogen lighting) may cause the internal IR to be off and the GSI value to indicate an increase in ambient lighting triggering a change back to color when there may be insufficient visible light. An IR-only light sensor would help as it could determine if the GSI value is because of a high amount of external IR. light or not. A visible-only light sensor could be used provided its visible light sensitivity matched (or was close to) that of the image sensor. Below, two different methods are outlined for the b/w to color change. Method 1 requires movement of the IR cut-fdter 160 and a quick reading of the Luma level, while Method 2 uses analysis of the red, green and blue values from the image sensor to determine if the light level is from elevated IR or not.

[ 0066] Method 1

[ 0067 ] Whenever the transition happens, the GSI value will always rise when changing from black and white mode to color mode, or the received Luma value will always decrease (or y-mean). If the immediate GSI rise or Luma drop is beyond some acceptable margin then the scene does not have enough visible light and it should immediately be changed back to black and white or Night mode. An additional GSI decrease would be added to force the received light to a higher level before change to color was considered again. The use of instantaneous Luma value after change from black and white to color should be a much quicker determination than waiting for the Exposure Algorithm to produce a settled GSI value. This requires separate control of the IR cut-filter 160. Using a change from‘force b/w’ to‘force color’ may or may not be quick enough. By remaining in b/w and moving the IR cut-filter 160 to IN (in the optical path) will block IR while letting visible light through. Monitoring of the Luma change after the IR cut-filter 160 is in the optical path will indicate if there is enough light for color mode or not. If the Luma increase is sufficient the camera could stay with IR cut-filter 160 in the optical path and change to color mode. If the Luma increase is not enough or actually decreases then there is not enough color light and the IR cut-filter 160 can be changed back to OUT (out of the optical path or b/w mode). Movement of IR cut-filter 160 to IN and back to OUT can be done in much less than 1 second.

[ 0068 ] Method 2

[ 0069] In b/w mode, the image sensor 122 actually still generates color images. The color saturation is reduced to 0 to remove unwanted color noise in low light or color changes from IR illumination. It is possible to read the Auto White Balance (AWB) red, green, blue (RGB) gain values even in b/w mode to determine the general color content of the light. Generally, a pure IR illumination will cause RGB gains to be equal as image sensors red, green and blue pixels are equally sensitive to pure IR illumination. Very red light (low color temperature) would appear as low Red gain, blue light would appear as low Blue gain.

[ 0070] High IR content lighting will generally have a low color temperature and therefore RED gain will be lower than low IR content lighting. In some cases, changing AWB to Wide while in B/W will give the best RGB gain values. If Red value is lower than Green and Blue values then the process allows the GSI value to fall lower (brighter scene) before considering change to color. There is a need to determine the GSI value change if Red gain is low.

[ 0071] Summary':

[ 0072 ] Use of GSI value means that day to night transition is based on the effort required from the exposure algorithm to maintain a desired image brightness and is independent on the scene or zoom level. Day to Night transition at max zoo will probably happen at a higher Lux level reaching the camera due to the reduced amount of light reaching the image sensor compared to at max wide for the same scene. However, the day to night transition will be at the same gain and shutter combination and therefore the same general image quality.

[ 0073] Fig. 3 is a flow diagram illustrating the surveillance camera's day-night transitions according to the invention.

[ 0074 ] It relies on a Day to Night Threshold. This threshold is a single number that is derived from a combinati on of a shutter-speed selection, a max gain sel ection and an iris selection. As the light level falls, the gain, shutter and iris values are changed by the camera’s Auto Exposure algorithm. When the combination of gain, shutter and iris reaches this number it triggers the camera to change from Day (color) mode to Night (black and white) mode.

[ 0075] The IR band is an exposure window (gain shuter and iris) that the firmware tries to maintain as the light level changes (up or down) by controlling the power of the IR illumination within the camera. The window is derived from the Day to Night transition configurations. However, not all cameras have built in IR illumination (124) and cameras with built in IR illumination may have it disabled by the user.

[ 0076] According to Method 2 above, Night to Day threshold is a trigger level that is derived from the Day to Night Threshold. While monitoring the camera parameters for Day to Night transition is easy, determining if the parameters have reached the Night to Day threshold is not so easy. The same gain, shutter and iris parameters can have different amounts of visible light depending on the light type. Thus, additional data is needed to determine if night to day transition can be done safely, without leaving the color image too dark, which could cause a change back to night mode (potential for day to night and night to day oscillation). In one embodiment, additional data could come from the Red, Green and Blue color data from the image sensor 122 (preferred solution).

[ 0077 ] In another method, additional data could come from an IR or visible light sensor (least desirable solution) but these sensors tend to be inaccurate and have a wide value tolerance at the relatively low light levels involved with Night to Day change.

[ 0078 ] In another solution, Method 1 above, the light IR cut- filter that is integral to most CCTV cameras is quickly changed. (Movement of the filter is an electromechanical process that can be triggered by software - filter removed for night mode and replaced for day mode). This is accomplished by briefly replacing the filter while still in Night mode, reading the image luminance value and remove the filter again if the Luminance is too low. This would be done while leaving the camera in Night mode (black and white). This could be achieved in less than 1 second but may still result in a brief change in the images from the camera.

[ 0079] In more detail, in step 212, the live gain, shutter, and iris information is obtained by the controller 110. The controller then determines in step 214 whether the system is operating below' the day to night threshold in step 214 That is, does the single number derived from a combination of a shutter-speed selection, a max gain selection and an iris selection selected by the camera’s Auto Exposure algorithm indicate a transition to Night (black and white) mode.

[ 008 0] If it is not, then step 212 is repeated.

[ 008 1] On the other hand, if it has fallen below the threshold as determined in step 214, then the controller 110 changes the camera to Night mode in step 216. In night mode, the image sensor 125 does not produce color images and the IR cut-filter is removed from in front of the image sensor.

[ 0082 ] Then the controller 110 determines whether IR is enabled in step 218.

[ o o 83 ] If IR is enabled, then the control ler turns IR illumination on at its maximum setting in step 230. This is accomplished by the controller activating the IR driver 112 to provide the maximum rated current and voltage to the infrared light emitting diodes (LEDs) 116.

[ 0084 ] The controller also then obtains the live gain, shutter, and iris information. The controller determines whether or not it is above the IR band in step 234. As mentioned previously, IR band is an exposure window calculated by the controller 110 based on the gain setting of the image sensor 122, shutter speed of the image sensor, and iris setting of the iris 125 that the controller tries to maintain as the light level changes (up or down) by controlling the power delivered by the IR driver 122 to the LEDs 116 of the IR

illumination within the camera. The window is derived from the Day to Night transition configurations.

r o o 85] If it i s not above the IR band, then contro!l er determines whether thi s exposure window is below the IR band in step 236.

[ 008 6] If it is not below the IR band in step 236, the processing returns to step 232.

[ 0087 ] If it is below the IR band in step 236, then in step 238, the controller 110 determines whether or not IR is at a maximum. That is, the controller determines if the LEDs are at their maximum rated light output. If no then the IR is increased in step 240 so that the IR driver 112 drives the infrared light emitting diodes (LEDs) 116 to produce more light. Processing then also returns back to step 232.

[ o o 88 ] Returning back to step 218, if IR is not enabled, then the controller 110 determines whether it is above the night to day threshold in step 220. Again, the controller 110 calculates a number derived from combination of the shutter-speed selection, the max gain selection and the iris selection selected by the camera’s Auto Exposure algorithm

[ 0089] According to step 220, the controller 110 repeatedly analyzes the ratios of Red, Blue and Green gain values from the image sensor 122 to estimate the color of the light reaching the camera. The controller determines if infrared light is present at a level that would require delay of transition back to day mode (color) in increasing light level. This color analysis supplements the analysis of gain, shutter and iris to help prevent transition back to day mode when lighting (color, type and/or intensity) is unsuitable for day mode.

[ 0090] In more detail, especially in color mode (day), auto white balance (AWB) is operating to control the relative gain between the red blue and green pixels of the image sensor. This is a common feature in virtually every camera. It automatically adjusts the strength of red, green and blue gain controls, attempting to give a visually correct color image regardless of the color of light illuminating the scene. For example, this process makes white surfaces and objects look white regardless of the color of the light present. r o o 91 j When in night mode (black and white image) the image sensor 122 is still producing Red, Green and Blue gain data and the Auto White balance feature is still functioning. Preferably, the controller 110 uses the red, green and blue gain numbers from the still functioning white balance algorithm to estimate the actual color of light present in the scene while operating in night mode. Analysis of the ratio of the Red, Green and Blue gain numbers help the controller assess the color of the detected light. For example, if there is red light present then the red gain value will be lower than green and blue, if blue light then blue gain will be lower than red and green. If the light is not significantly red or green or blue then the gain levels will be very_' similar. Strong IR light would appear as reduced Red gain.

[ 0092 ] If there are several modes available for the cameras existing auto white balance algorithm, then the mode with the widest possible range should be used in night mode.

[ 0093 ] According to some embodiments, a separate auxiliary light sensor 163 is employed by the controller 110 as an additional backup data source, if present in camera design. The controller 110 uses the detected light level auxiliary' light sensor 163 to validate if there is or is not any visible light present. Nevertheless, the light sensor 163 is not used as the primary data source for day to night or night to day transitions. The primary sources are gain, shutter, iris data and red, green and blue data

[ 0094 ] If the night to day threshold is exceeded as determined in step 220, then the controller changes to day mode in step 222. The image sensor 125 is converted to produce color images and the IR cut-filter is removed from in front of the image sensor and processing returns to step 212.

[ o o 95 ] If not above the night to day threshold in step 220, then processing moves to step 232.

[ 0096] The controller 1 10 is continually monitoring the live gain/shutter/iris exposure numbers in step 232.

r 0097 j If the exposure has risen above a desired level, then light from the LEDs 1 16 needs to be reduced by control of the IR driver 112 in step 252, if the LED's are on as determined in step 250.

[ 0098 ] On the other hand, if the LEDs 116 are off as determined in step 250, then the process moves to step 220 to consider changing back to day mode.

[ 0099] In general, it is challenging to maintain exactly one gain/shutter/iris exposure level so the thresholding of step 234 and step 236 define the lower and upper limits (band) for how much the exposure value can change without requiring IR power adjustment.

[ o o l o o ] If the exposure level remains steady, it means that the light level is steady and the algorithm would repeat steps 234, 236, 232, not above the upper limit and not below lo er limit. The range between the upper limit and the lower limit is the IR control band. r ooioi] On the other hand, if the light level drops below the lower limit, processing moves from step 236 to step 238.

[ 00102 ] While this invention has been particularly shown and described with references to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the invention encompassed by the appended claims.

1

Claims

What is claimed is:

1. A method for controlling transitions in a surveillance camera, the method comprising:

analyzing combinations of gain setting of an image sensor, shuter speed of the image sensor, and iris setting; and

controlling day to night and night to day transitions and color and black and white modes based on the analysis of the gain setting of the image sensor, shutter speed of the image sensor, and iris setting.

2. The method of claim 1, further compri sing additionally analyzing ratios of red, blue and green gain values from the image sensor to estimate color of the light reaching the image sensor.

3. The method of claim 2, further comprising analyzing the ratios of red, blue and green gain values from the image sensor during a black and white mode.

4. The method of claim 3, further comprising analyzing the ratios of red, blue and green gain values to determine if infrared light is present.

5. The method of claim 3, wherein the analysis of the ratios of red, blue and green gain values is used to determine whether delay in a transition back to day mode is required because color and/or intensity are unsuitable for day mode.

6. The method of claim 2, wherein the analysis of ratios of red, blue and green gain values supplements analysis the gain setting, shutter speed, and iris setting.

7. The method of any of the previous claims, further comprising using an auxil iary light sensor to validate a level of visible light present.

8. The method of claim 7, wherein the auxiliary light sensor is used as a secondary source after analysis the gain setting, shutter speed, and iris setting and ratios of red, blue and green gain values.

9. The method of claim 1, further comprising the controller changing an intensity infrared illumination generated by the camera when in black and white mode.

10. A surveillance camera that controls Day -Night Transitions, the camera comprising:

image sensor for generating images of a scene;

a lens system for forming an image of light from the scene on the image sensor; and

a controller that analyzes combinations of gain setting of the image sensor, shutter speed of the image sensor, and iris setting of the lens system to control day to night and night to day transitions and color and black and white modes.

11. The camera of claim 10, wherein the controller additionally analyzes ratios of red, blue and green gain values from the image sensor to estimate color of the light reaching the image sensor.

12. The camera of claim 11, wherein the controller analyzes the ratios of red, blue and green gain values from the image sensor during a black and white mode.

13. The camera of claim 12, wherein the controller analyzes the ratios of red, blue and green gain values to determine if infrared light is present.

14. The camera of claim 13, wherein the controller analyzes the ratios of red, blue and green gain values to determine whether delay in a transition back to day mode is required because color and/or intensity are unsuitable for day mode.

15. The camera of any of claims 10-14, wherein the controller analysis of ratios of red, blue and green gain values supplements analysis of the gain setting, shutter speed, and iris setting.

16. The camera of any of claims 10-15, further comprising an auxiliary light sensor, wherein the controller uses the response from the auxiliary light sensor to validate a level of vi sibl e light present.

17. The camera of claim 16, wherein the controller uses the auxiliary light sensor as a secondary' source after analysis of the gain setting, shutter speed, and iris setting and ratios of red, blue and green gain values.

18. The camera of any of claims 10-15, wherein the controller changes an intensity infrared illumination generated by the camera when in black and white mode.