EP1685543B1 - Digital video surveillance - Google Patents

Digital video surveillance Download PDF

Info

Publication number
EP1685543B1
EP1685543B1 EP04811354A EP04811354A EP1685543B1 EP 1685543 B1 EP1685543 B1 EP 1685543B1 EP 04811354 A EP04811354 A EP 04811354A EP 04811354 A EP04811354 A EP 04811354A EP 1685543 B1 EP1685543 B1 EP 1685543B1
Authority
EP
European Patent Office
Prior art keywords
alarm
camera
images
cameras
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Not-in-force
Application number
EP04811354A
Other languages
German (de)
French (fr)
Other versions
EP1685543A2 (en
Inventor
Gene A. Grindstaff
Sheila G. Whitaker
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Intergraph Corp
Original Assignee
Intergraph Software Technologies Co
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Intergraph Software Technologies Co filed Critical Intergraph Software Technologies Co
Priority to PL04811354T priority Critical patent/PL1685543T3/en
Priority to EP08075701A priority patent/EP2019380A1/en
Publication of EP1685543A2 publication Critical patent/EP1685543A2/en
Application granted granted Critical
Publication of EP1685543B1 publication Critical patent/EP1685543B1/en
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19678User interface
    • G08B13/1968Interfaces for setting up or customising the system
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19639Details of the system layout
    • G08B13/19641Multiple cameras having overlapping views on a single scene
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19654Details concerning communication with a camera
    • G08B13/19656Network used to communicate with a camera, e.g. WAN, LAN, Internet
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19678User interface
    • G08B13/19682Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19678User interface
    • G08B13/19691Signalling events for better perception by user, e.g. indicating alarms by making display brighter, adding text, creating a sound
    • G08B13/19693Signalling events for better perception by user, e.g. indicating alarms by making display brighter, adding text, creating a sound using multiple video sources viewed on a single or compound screen
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19695Arrangements wherein non-video detectors start video recording or forwarding but do not generate an alarm themselves
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19697Arrangements wherein non-video detectors generate an alarm themselves

Definitions

  • the present invention relates to a video surveillance system operating in a network having an associated bandwidth and to a method for controlling such a video surveillance system.
  • Trigger sensors are sensors that coupled to a camera that indicate that an event has occurred.
  • a trigger sensor may be a contact switch that is attached to a door, that when the door is opened causes an alarm.
  • Having to rewire all of the cameras and all of the trigger sensors in a surveillance system can be extremely time consuming and costly especially on large scale systems. Further, expansion of the system once it is wired is difficult, if not impossible. Adding additional cameras and sensors can require the entire system to be rewired.
  • prior art surveillance systems do not provide for continuous monitoring such that the system will automatically shift between a normal condition state and a configurable alarm condition state wherein images from different cameras are recorded and stored depending upon the state. Still further, the prior art systems do not allow the user to define the alarm conditions which will trigger the storage of image information. Prior art systems may include trigger sensors, but do not allow the user to define the threshold parameters that will cause the trigger sensors to set off an alarm. Nor do these prior art systems allow the user to combine different trigger sensors and different parameters for the trigger sensors in combination in order to set an alarm condition. Further, users of the prior art systems cannot create an alarm condition by selecting trigger sensors and selecting parameters for the trigger sensors.
  • prior art video surveillance systems are not modular and are not designed for a distributed architecture. For example, if the video display section of a prior art video surveillance system fails then the system would also fail to record images.
  • the present invention provides a control system with two or more cameras and one or more sensors to form a video surveillance system, the control system receiving a request for images from one or more of the cameras over a network having an associated bandwidth, the control system comprising: a display module for receiving the request for images and providing an output of the requested images from one or more of the cameras, wherein the display module monitors available network bandwidth and adjusts the output based on the available network bandwidth by reducing resolution of one or more images from one or more of the cameras, but giving priority to and providing a higher resolution image from any camera that is associated with an alarm state; characterized in that the system further comprises: an alarm processing module for receiving at least one sensor signal from one or more sensors, each of the two or more cameras being associated with one or more of the one or more sensors, wherein the alarm processing module determines if there is an alarm state for a camera based in part on the at least one sensor signal, and wherein the display module also receives input from the alarm processing module which indicates if a camera is in an alarm state.
  • the present invention also provides a method for controlling a video surveillance system operating in a network having an associated bandwidth, the method comprising: receiving a request originating from a location associated with a user interface in the network for transmission of images from one or more of a plurality of cameras from the video surveillance system; obtaining the images from the one or more of the cameras wherein the images are composed of digital data; determining if the digital data for the images to be transmitted exceeds bandwidth constraint; determining if a camera is in an alarm state; reducing the digital data until the digital data no longer exceeds bandwidth constraint by decreasing the resolution of images, but giving priority and providing a higher resolution to the images that are associated with an alarm state; and transmitting the images to the location associated with the user interface, wherein determining if a camera is in an alarm state comprises: receiving at least one sensor signal from one or more sensors, each of the plurality of cameras being associated with one or more of the one or more sensors, and determining if there is an alarm state for a camera based in part on the at least one sensor signal
  • module shall apply to both software embodiments and hardware embodiments, as well as, combinations of software and hardware, unless the context indicates otherwise.
  • Fig. 1 is block diagram showing the hardware employed in one embodiment of the digital video surveillance system 100.
  • the digital video surveillance system 100 includes one or more cameras 105 that are networked together and that incorporate one or more trigger sensors (not shown).
  • the digital video surveillance system 100 may be combined with a legacy system without having to rewire the network or requiring new contacts for the trigger sensors.
  • the pre-existing legacy network may be used.
  • the system includes sixteen digital cameras 105 that are connected together over a 100 Base FX Ethernet/fiber network which is part of a legacy system.
  • a multi-mode switch such as an Alcatel Omni Stack 6100 fiber switch 120 is employed for routing the signals between different networks.
  • the legacy system includes a 10/100 Base FX network.
  • Newly added to the legacy system is a 1000 Base FX connections. Both networks are coupled to a digital video recorder 130 and also to computer 140 which receives in the images and allows a user of the system to control the display of the cameras on a display device 150.
  • the computer 140 also has software that is added for receiving in the digital images over the network, processing the digital images, allowing the user to display the images and monitoring one or more trigger sensors that are associated with the cameras.
  • the present invention may also integrate with an analog monitoring system, however each camera would be coupled to an analog-to-digital converter for converting the output images to a digital data stream or the multi-mode switch may have an integrated analog to digital converter. Further, the system would account for the time division multiplexing of the analog images of the legacy system.
  • a phone network may be used either as the new network or as the legacy network, but the network is preferably partitioned such that the data and voice connections are separate.
  • the video recorder and the computer should be coupled to the backbone of the newly added high speed network in order to compensate for digital data bandwidth issues from the multiple cameras.
  • Fig. 1 is exemplary and may be expanded, as the system is modular to include more cameras and more computers that are networked together.
  • Fig. 2 is a block diagram showing the modules and communication between modules employed in one embodiment of the digital video surveillance system 100.
  • the system is very modular and the modules may be distributed across many computers that are networked together.
  • the digital video surveillance system includes an administration module 210 which allows a user to set-up the system. The administrator can changes parameters within a database file which is identified as the video database 220 in Fig. 2 .
  • the database 220 is coupled to a network database interface 225 such as a Microsoft DAO (data access object) interface or a Sequel Server object interface.
  • the video database 220 includes a file or set of files that are utilized by an administration program 210.
  • the database 220 includes a listing of the cameras, a listing of each camera's location, and an alarm setting for the camera.
  • the database 220 may also include a table of access rights for operators of the system.
  • the administrator may select the cameras that are to be displayed. For example, if the system includes twenty-five cameras, the administrator may assign privileges such that only sixteen cameras are available for a particular user.
  • the administrator may also set conditions for an alarm. Alarm conditions can be simple conditions such as the closing of a switch for a particular trigger sensor that is associated with a camera, or alarm conditions may be a complex concatenation of events.
  • a camera may have multiple sensors associated with the camera including a temperature sensor, a contact sensor for a door, and a light sensor. As an example, the administrator may indicate that an alarm will occur when the contact sensor for the door is released (the door is opened) and also when the temperature within the room reaches 72 degrees Fahrenheit. Thus, if only one of the conditions occur, there will be no alarm.
  • each of the cameras is coupled to one or more trigger sensors.
  • Standard within the video surveillance industry is the use of a PLC (programmable logic control) module 230.
  • the PLC module 230 monitors the status of the trigger sensors.
  • the present invention accesses the PLC module's information. This information is retrieved by an alarm processing module 240.
  • Either the alarm processing module or the PLC module includes an interface that can translate the sensor signal status into a predefined software protocol.
  • the alarm processing software module 240 also accesses the video database 220 through the network database interface 225.
  • the video database 220 may be located on a remote computer.
  • each of the various modules is designed for a distributed architecture and therefore, the modules may be located on different computer systems.
  • the alarm processing module 240 accesses the database 220, the alarm processing module 240 obtains the alarm conditions associated with each of the cameras and the associated trigger sensors. When the status of a trigger sensor changes, the alarm processing module 240 compares the condition set for an alarm with the change in the trigger sensor's status that is obtained from the PLC monitor module. If an alarm condition arises, the alarm processing module 240 sends an alarm message to the display software module 250.
  • the display software module 250 interfaces with a camera interface module 255.
  • the camera interface module 255 is capable of integrating with a pre-existing camera interface and is also capable of receiving camera information directly from the cameras. In one embodiment, the camera interface 255 is an Active X control, but different manufacturers use different interface protocols and the interface protocols may vary.
  • the camera information is passed to the display module 250.
  • the display module 250 displays the images captured by the camera on the user's display in a window.
  • the user may control the number of cameras that are displayed, whether to record each camera, and the mode of operation.
  • the display module 250 also receives input from the alarm processing module 240 which will indicate if a camera is in an alarm state. If the camera is in an alarm state, the display module 250 will cause the display to show the images from the camera with indicia that there is an alarm, such as, a border around the camera window.
  • the window for the camera may flash or other indicia may be displayed to indicate that there is an alarm condition.
  • an alarm may cause the display of the camera to be redrawn into a larger window on the display device.
  • the display module can also change the resolution of the images depending on whether the images are associated with an alarm state or are in a normal state. If an alarm state is detected, the display module can send the images at a higher resolution than for the cameras in a normal state. Further, a user may also use the display module to select other cameras in a normal state that the user wishes to have displayed at a higher resolution and thus, the display module will transmit the images to the display at the higher resolution.
  • Each of the modules within the system includes the ability to couple to the other modules within the system through a network as shown in Fig. 2A .
  • each module is equipped to transmit and receive data communications using a networking protocol, such as TCP/IP.
  • TCP/IP networking protocol
  • the ability to distribute the modules in a network allows the modules to be placed in different locations, but still access information from the other modules.
  • different users can access images from different locations by having the user interface controls 205 situated on a computer associated with each user.
  • the system may have a plurality of monitoring stations 206A at remote locations and each user can independently select the cameras 260A, 261A, 262A that the user wishes to view.
  • the modules are designed as a system, and the system monitors the network bandwidth.
  • the system can automatically give priority to images from cameras that are associated with an alarm condition so that a user viewing a monitoring station 206A does not loose any information about an alarm due to bandwidth constraints in the network 270A.
  • bandwidth constraints can be accommodated by reducing the resolution of the images that are transmitted, but providing a higher resolution image from any camera 260A that is associated with an alarm state.
  • the display module 250 can stop the transmission of images that are not in the alarm state to accommodate network bandwidth limitations.
  • the bandwidth of the system is predetermined and thus there is no need to actively monitor the bandwidth.
  • the display module 250 still accounts for the bandwidth of the images that are being sent out into the network and monitor the percentage of bandwidth being utilized. Thus, based on the percentage of bandwidth that is utilized, the display module can determine the resolution of the images that are to be transmitted. The system thus, automatically and dynamically adjusts to the bandwidth conditions based on the resolution of the images being transmitted, the number of requested cameras, the number of users that are accessing images and the known network bandwidth.
  • command and control of the system through the administration module can be maintained at a remote location.
  • the display system and the recording system are independent. If the display system fails, the digital video surveillance system is still capable of recording information. There are separate data paths to both the recording software and the display software.
  • the video recording software 260 is legacy recording software.
  • the invention disclosed is not limited to such an embodiment.
  • a new recording program may be integrated into a legacy system.
  • the camera used in the system is shown as an analog camera.
  • the camera is coupled to a video encoder which includes a A/D converter 270.
  • the A/D converter is the only hardware that is present. All of the other modules are software components.
  • the video encoder 270 also packetizes the data and converts the image data into a format capable of HTTP display. Further, the encoder accounts for any time division multiplexing of image information.
  • the video encoder is a video encoder manufactured by Axis Communications.
  • the video encoder includes at least two sensor inputs, two control inputs, a control output and a sensor output.
  • the control output is fed back into a sensor input.
  • the alarm processing software generates a control signal indicating that an alarm condition has occurred to the control input of the video encoder.
  • the legacy recording software program expects a sensor signal as an indication to begin recording.
  • the control signal from the alarm processing software will cause a sensor signal to be output and received by the recording software and recording will begin.
  • This jumper allows recording to be controlled without having to create an additional protocol signal. Additionally, the control signal to the video encoder can cause the analog video signal to be sampled and converted to different sized digital formats, since the video recorder expects a sensor signal output from the video encoder to change formats.
  • the user of the system can control and change the status of a camera from an alarmed status to normal status by using the user interface 205 and indicating acknowledgement of the alarm condition.
  • Fig. 3A-3B shows a flow chart of how the software modules operate.
  • the record set includes the names and locations of each of the cameras along with the alarm conditions for each of the cameras.
  • Alarm conditions may include relative evaluations such as greater than, less than and equal to.
  • an alarm condition may include a settable parameter, such as a threshold level.
  • the alarm processing module retrieves the record set.
  • the cameras are initialized 310. Each of the cameras is turned on and is set to a normal setting.
  • the triggers sensors are also read and the alarm processing software module compares the signal from the trigger sensor(s) with the record set information.
  • the system then waits for user interaction or for an alarm to occur 320.
  • the system checks to see if it was an alarm 325. If it is an alarm, the camera state is updated 330.
  • the display software indicates that an alarm has occurred by including some indicia on the display. For example, the window displaying the camera images may be enclosed with a different colored border.
  • a status variable for the camera is also updated to indicate that an alarm has occurred. If it is an alarm 335, text is added to the alarm list indicating the type of alarm 340.
  • the trigger sensors may be temperature sensors, contact sensors, smoke detectors etc. The alarm may indicate that there is a fire, a rise in temperature, an intruder or other type of alarm. The system then determines if the alarm is occurring at present. A check of the status of the trigger sensors is performed 345.
  • the system checks whether the event is user input 350. If the answer is no the system either exits if the event is an exit request or returns to point B. If the event is user input, the system checks to see if the user has requested to increase the size of a view of a particular camera 352. If the answer is yes the system changes a variable and marks the camera as enlarged 355. The system then redraws the camera into a larger sized window and updates and redraws the labels 357. The system then continues onto point B.
  • the system inquires if the user desires to remove the images of the camera from the display (turn off the camera) 360. If the user does want to remove the images of the camera, the system inquires whether the camera is in an alarmed state 362. If the camera is in an alarmed state, the user is notified that the user cannot remove the images from the display of an alarmed camera 365. The system then returns to point B. If the camera is not alarmed, the camera is marked as being off and the camera is removed from the display window and all of the remaining camera windows are resized to fit within the display 367. If the user has not requested to turn off the images from the camera, the system inquires whether the user has requested to change the status of an alarm 370.
  • the visual display marks the alarmed camera window with indicia that the alarm has been acknowledged, but the alarm is still active 372.
  • the system inquires if all of the camera alarms have been acknowledged 375: If the answer is yes the outline (border) of the window for the camera is changed from the color associated with an alarm condition to that of a normal condition, but the indicia such as an asterisk remains to indicate that the camera/trigger sensors are still in an alarmed state 377. The system then resizes and redraws the windows 357 and the system returns to point B. If all of the alarms have not been acknowledged by the user, the system returns to Point B.
  • the system checks to see if the request by the user is for reconfiguring the database 380.
  • the system at this point may check the status of the user to see if the user has been granted the privilege to change the database. If the answer to this inquiry is yes, the system looks to see what the user wishes to change 382.
  • the user can change the alarm settings, including the alarm constraints, the name of a camera, and which cameras are being displayed, for example. If the user does not change a setting in the database the system returns to point B. If the user does make a change to the database then the data base is updated 385. A command is sent to the alarm processing software module which indicates that the database has been updated and the recordset is again downloaded to the alarm processing software module 387.
  • an administrator or user with the proper privileges can set the alarm such that an alarm requires a combination of events to trigger an alarm. For example, an alarm may be sounded only if a fire occurs and a flood occurs. In such a case, a temperature sensor may have to reach a particular temperature and a water level indicator may need to reach a particular level before an alarm occurs. An authorized user has control to change whether one of or more sensors will cause an alarm and the threshold parameter for each sensor.
  • the user may acknowledge the alarm. If the user acknowledges the alarm 347, the system removes text from the alarm list display 348. The system then inquires whether the camera is still alarmed 349. If the camera is still in an alarmed state (the trigger sensors are set such that an alarm condition exists), then the system returns to point B. If the camera is no longer in an alarmed state the display software redraws the windows displaying the camera's images on the display screen along with updated labeling 390, i.e., returning to labeling indicative of the normal state.
  • Fig. 4 shows the interaction of the alarm and normal conditions.
  • the system operates under a normal condition state for much of the time.
  • the software routine as explained with respect to Figs. 3A and 3B continues to monitor the trigger sensors for an alarm condition.
  • the system automatically goes into an alarm condition state.
  • the system will remain in that state until the user acknowledges the state.
  • the user of the system is provided with a graphical interface on a display device that shows one or more windows in which the images from a camera are displayed.
  • the graphical window is updated with text that indicates that an alarm has occurred along with a border around the window, such as a red border.
  • a new window will be provided for that camera.
  • the user can then acknowledge that an alarm is occurring. This takes the system out of the alarm mode and resets the system back into the normal condition mode.
  • the window for the camera which was having the alarm will be reset, such that the text will no longer indicate that an alarm is occurring, and the border around the window is changed back to the normal condition color.
  • an asterisk or other indicator will be marked on the graphical user interface which indicates that the sensors are still in an alarmed condition state, however the system will return to a normal condition state.
  • Fig. 5 shows the three modes of operation for the normal condition.
  • a user can determine which cameras the user wants displayed and what information that the user would like stored. For example, in manual mode 510, the user selects the cameras to have displayed. The user may select one or multiple cameras to display. In manual mode 510, the user monitors the system and in the normal condition none of the images from any of the cameras are recorded and stored to memory. If the user selects the cycle mode 520, the user can select one or more cameras to have permanently displayed on the display, but at least one of the window cycles through the remaining cameras. For example, if there are eight possible cameras, the user may select to permanently display cameras 1-3 (521) and in the fourth window 525 the system will cycle through cameras 4-8.
  • the system does not record and store information to a memory storage device.
  • the third mode which is marked S.S. 530 the display operates much like the manual mode, but all the information from all of the cameras is recorded to memory 540 in digital form.
  • the number of cameras shown in the Figs are provided for illustrative purposes and the system is not limited to a fixed number of cameras.
  • Fig. 6 shows the display screen 610 A,B,C and storage 620 A,B,C when different numbers of cameras are used under an alarm condition.
  • the display 610A changes such that a textual indicator is provided, for example "alarm #1" and the window for the camera changes its border color.
  • Other indicia might be used to indicate that an alarm has occurred such as having the picture flash or the border flash.
  • each camera that has an associated alarm condition will have its images stored to memory 620A.
  • the display 610B will show each of the cameras and when an alarm occurs for one or more of the cameras one or more indicators that an alarm is occurring will be provided.
  • cameras 1-3 (612B) are each having an alarm.
  • a textual message is provided and each of the camera windows for which there is an alarm has a changed border color.
  • the system automatically begins to record the camera images. If there are more than four cameras and only four cameras are capable of being displayed due to sizing of the windows on the display device 610C, one or more of the windows 615C will rotate.
  • a window will be initiated which will display the camera and will have some indicia that an alarm condition is occurring. If more than four cameras experience alarm conditions at the same time, the display 610C will rotate the images in different windows so that a user can view each alarmed camera.
  • the number of cameras that are referenced in the Figs. is for illustrative purposes and the system does not have a fixed number of cameras nor is the camera limited to the number of camera images that can be displayed on a display device.
  • the digital images from the camera are stored in memory 620A, 620B, and 620C.
  • Figs. 5 and 6 show the video images being cycled through if there are more images than space to display the images on the display device, however there are other methods for displaying the images in both the alarm and normal conditions.
  • the user may be allowed to page through a group of cameras.
  • the current images from a camera may be displayed based upon the cameras that have been either most frequently requested in the past or a camera that has been requested over a certain past time period.
  • the software also allows a user to prioritize the cameras that will be displayed, so that a camera that is marked with a high priority will be the last image to be removed from the display in the event of an alarm condition. It should be understood by one skilled in the art that other variations for displaying the images from a camera can be used without deviating from the scope of the invention.
  • Fig. 7 is a screen shot showing the user graphical interface.
  • sixteen cameras are visible at the same time and one of the cameras AC #2 (710) in the upper right hand corner is experiencing an alarm. The alarm condition is identifiable because the border around the window for camera AC #2 (710) has changed.
  • To the right of the screen each camera of the sixteen possible cameras that is available is listed along with a check box (720), which when checked indicates that images from the camera will be displayed. In this fig. "select all cameras” is selected and all sixteen cameras are displayed at the same time.
  • Fig. 8 shows a different screen shot. This screen shot shows that only six of the sixteen possible cameras are selected and displayed to the user.
  • AC #2 is experiencing an alarm condition.
  • the screen 710 for AC#2 is increased in size indicating that it is in an alarm condition. By increasing the size of the image, the user can better identify the cause of the alarm.
  • AC#2 can also be identified as experiencing an alarm condition because the border around its window 710 is changed to a different color than the border around the windows of camera's operating in a normal state.

Abstract

A control system for use with one or more video surveillance cameras forming a video surveillance system. The control system includes a video database module wherein a database entry includes at least one sensor condition defining an alarm state for each camera. Additionally, there is an alarm condition module that receives one or more sensor signals from one or more sensors associated with a camera and retrieves the sensor condition defining an alarm state for the camera. The alarm condition module outputs an alarm signal if the alarm condition module determines that the one or more sensor signals meets the one or more sensor conditions. The control system is modular and may be distributed in a network environment. The control system also monitors the network bandwidth and can adjust the throughput of digital data representing the digital images in order to avoid any bandwidth limitations.

Description

    Technical Field and Background Art
  • The present invention relates to a video surveillance system operating in a network having an associated bandwidth and to a method for controlling such a video surveillance system.
  • Prior art surveillance systems that are to be installed with legacy surveillance systems require complete network rewiring. This is especially true if the legacy system includes analog cameras and the new system is digital. In addition to rewiring all of the cameras, the sensors that trigger the cameras need to be rewired. Trigger sensors are sensors that coupled to a camera that indicate that an event has occurred. For example, a trigger sensor may be a contact switch that is attached to a door, that when the door is opened causes an alarm. Having to rewire all of the cameras and all of the trigger sensors in a surveillance system can be extremely time consuming and costly especially on large scale systems. Further, expansion of the system once it is wired is difficult, if not impossible. Adding additional cameras and sensors can require the entire system to be rewired.
  • Additionally, prior art surveillance systems do not provide for continuous monitoring such that the system will automatically shift between a normal condition state and a configurable alarm condition state wherein images from different cameras are recorded and stored depending upon the state. Still further, the prior art systems do not allow the user to define the alarm conditions which will trigger the storage of image information. Prior art systems may include trigger sensors, but do not allow the user to define the threshold parameters that will cause the trigger sensors to set off an alarm. Nor do these prior art systems allow the user to combine different trigger sensors and different parameters for the trigger sensors in combination in order to set an alarm condition. Further, users of the prior art systems cannot create an alarm condition by selecting trigger sensors and selecting parameters for the trigger sensors.
  • Additionally, prior art video surveillance systems are not modular and are not designed for a distributed architecture. For example, if the video display section of a prior art video surveillance system fails then the system would also fail to record images.
  • Reference may be made, inter alia, to US 2003/0095042 A1 (Ebata et al ) , which is entitled "Surveillance system and network system".
  • Summary of Invention
  • The present invention provides a control system with two or more cameras and one or more sensors to form a video surveillance system, the control system receiving a request for images from one or more of the cameras over a network having an associated bandwidth, the control system comprising: a display module for receiving the request for images and providing an output of the requested images from one or more of the cameras, wherein the display module monitors available network bandwidth and adjusts the output based on the available network bandwidth by reducing resolution of one or more images from one or more of the cameras, but giving priority to and providing a higher resolution image from any camera that is associated with an alarm state; characterized in that the system further comprises: an alarm processing module for receiving at least one sensor signal from one or more sensors, each of the two or more cameras being associated with one or more of the one or more sensors, wherein the alarm processing module determines if there is an alarm state for a camera based in part on the at least one sensor signal, and wherein the display module also receives input from the alarm processing module which indicates if a camera is in an alarm state.
  • The present invention also provides a method for controlling a video surveillance system operating in a network having an associated bandwidth, the method comprising: receiving a request originating from a location associated with a user interface in the network for transmission of images from one or more of a plurality of cameras from the video surveillance system; obtaining the images from the one or more of the cameras wherein the images are composed of digital data; determining if the digital data for the images to be transmitted exceeds bandwidth constraint; determining if a camera is in an alarm state; reducing the digital data until the digital data no longer exceeds bandwidth constraint by decreasing the resolution of images, but giving priority and providing a higher resolution to the images that are associated with an alarm state; and transmitting the images to the location associated with the user interface, wherein determining if a camera is in an alarm state comprises: receiving at least one sensor signal from one or more sensors, each of the plurality of cameras being associated with one or more of the one or more sensors, and determining if there is an alarm state for a camera based in part on the at least one sensor signal.
  • Brief Description of the Drawings
  • The foregoing features of the invention will be more readily understood by reference to the following detailed description, taken with reference to the accompanying drawings, in which:
    • Fig. 1 is block diagram showing the hardware employed in one embodiment of the digital video surveillance system;
    • Fig. 2 is a block diagram showing the modules and communication between modules of the software employed in one embodiment of the digital video surveillance system;
    • Fig. 2A is a block diagram showing one embodiment of the modular system in a networked environment;
    • Fig. 3A-3B shows a flow chart of how the software modules operate;
    • Fig. 4 shows the interaction of the alarm and normal conditions;
    • Fig. 5 shows one embodiment in which there are three modes of operation for the normal condition;
    • Fig. 6 shows an embodiment in which different numbers of cameras are in an alarm condition;
    • Fig. 7 is a screen shot of the graphical user interface wherein one of the cameras that is displayed is in an alarm state; and
    • Fig. 8 is another screen shot of the graphical user interface wherein one of the cameras that is displayed is in an alarm state.
    Detailed Description of Specific Embodiments
  • In the following disclosure and the appended claims the term "module" shall apply to both software embodiments and hardware embodiments, as well as, combinations of software and hardware, unless the context indicates otherwise.
  • Fig. 1 is block diagram showing the hardware employed in one embodiment of the digital video surveillance system 100. The digital video surveillance system 100 includes one or more cameras 105 that are networked together and that incorporate one or more trigger sensors (not shown). The digital video surveillance system 100 may be combined with a legacy system without having to rewire the network or requiring new contacts for the trigger sensors. The pre-existing legacy network may be used. In exemplary Fig. 1, the system includes sixteen digital cameras 105 that are connected together over a 100 Base FX Ethernet/fiber network which is part of a legacy system. A multi-mode switch such as an Alcatel Omni Stack 6100 fiber switch 120 is employed for routing the signals between different networks. The legacy system includes a 10/100 Base FX network. Newly added to the legacy system is a 1000 Base FX connections. Both networks are coupled to a digital video recorder 130 and also to computer 140 which receives in the images and allows a user of the system to control the display of the cameras on a display device 150. The computer 140 also has software that is added for receiving in the digital images over the network, processing the digital images, allowing the user to display the images and monitoring one or more trigger sensors that are associated with the cameras.
  • The present invention may also integrate with an analog monitoring system, however each camera would be coupled to an analog-to-digital converter for converting the output images to a digital data stream or the multi-mode switch may have an integrated analog to digital converter. Further, the system would account for the time division multiplexing of the analog images of the legacy system.
  • Different networking options may also be employed other than fiber channels. For example, a phone network may be used either as the new network or as the legacy network, but the network is preferably partitioned such that the data and voice connections are separate.
  • Preferably in the video surveillance system, the video recorder and the computer should be coupled to the backbone of the newly added high speed network in order to compensate for digital data bandwidth issues from the multiple cameras.
  • It should be understood by one of ordinary skill in the art that the system shown in Fig. 1 is exemplary and may be expanded, as the system is modular to include more cameras and more computers that are networked together.
  • Fig. 2 is a block diagram showing the modules and communication between modules employed in one embodiment of the digital video surveillance system 100. The system is very modular and the modules may be distributed across many computers that are networked together. The digital video surveillance system includes an administration module 210 which allows a user to set-up the system. The administrator can changes parameters within a database file which is identified as the video database 220 in Fig. 2. The database 220 is coupled to a network database interface 225 such as a Microsoft DAO (data access object) interface or a Sequel Server object interface. The video database 220 includes a file or set of files that are utilized by an administration program 210. The database 220 includes a listing of the cameras, a listing of each camera's location, and an alarm setting for the camera. The database 220 may also include a table of access rights for operators of the system. The administrator may select the cameras that are to be displayed. For example, if the system includes twenty-five cameras, the administrator may assign privileges such that only sixteen cameras are available for a particular user. The administrator may also set conditions for an alarm. Alarm conditions can be simple conditions such as the closing of a switch for a particular trigger sensor that is associated with a camera, or alarm conditions may be a complex concatenation of events. For example, a camera may have multiple sensors associated with the camera including a temperature sensor, a contact sensor for a door, and a light sensor. As an example, the administrator may indicate that an alarm will occur when the contact sensor for the door is released (the door is opened) and also when the temperature within the room reaches 72 degrees Fahrenheit. Thus, if only one of the conditions occur, there will be no alarm.
  • As previously stated, each of the cameras is coupled to one or more trigger sensors. Standard within the video surveillance industry is the use of a PLC (programmable logic control) module 230. The PLC module 230 monitors the status of the trigger sensors. The present invention as shown in this embodiment, accesses the PLC module's information. This information is retrieved by an alarm processing module 240. Either the alarm processing module or the PLC module includes an interface that can translate the sensor signal status into a predefined software protocol. The alarm processing software module 240 also accesses the video database 220 through the network database interface 225. As such, the video database 220 may be located on a remote computer. Similarly, each of the various modules is designed for a distributed architecture and therefore, the modules may be located on different computer systems. When the alarm processing module 240 accesses the database 220, the alarm processing module 240 obtains the alarm conditions associated with each of the cameras and the associated trigger sensors. When the status of a trigger sensor changes, the alarm processing module 240 compares the condition set for an alarm with the change in the trigger sensor's status that is obtained from the PLC monitor module. If an alarm condition arises, the alarm processing module 240 sends an alarm message to the display software module 250. The display software module 250 interfaces with a camera interface module 255. The camera interface module 255 is capable of integrating with a pre-existing camera interface and is also capable of receiving camera information directly from the cameras. In one embodiment, the camera interface 255 is an Active X control, but different manufacturers use different interface protocols and the interface protocols may vary. The camera information (images) is passed to the display module 250. The display module 250 then displays the images captured by the camera on the user's display in a window. The user may control the number of cameras that are displayed, whether to record each camera, and the mode of operation. The display module 250 also receives input from the alarm processing module 240 which will indicate if a camera is in an alarm state. If the camera is in an alarm state, the display module 250 will cause the display to show the images from the camera with indicia that there is an alarm, such as, a border around the camera window. In some embodiments, the window for the camera may flash or other indicia may be displayed to indicate that there is an alarm condition. For example, an alarm may cause the display of the camera to be redrawn into a larger window on the display device. The display module can also change the resolution of the images depending on whether the images are associated with an alarm state or are in a normal state. If an alarm state is detected, the display module can send the images at a higher resolution than for the cameras in a normal state. Further, a user may also use the display module to select other cameras in a normal state that the user wishes to have displayed at a higher resolution and thus, the display module will transmit the images to the display at the higher resolution.
  • Each of the modules within the system includes the ability to couple to the other modules within the system through a network as shown in Fig. 2A. Thus, each module is equipped to transmit and receive data communications using a networking protocol, such as TCP/IP. The ability to distribute the modules in a network allows the modules to be placed in different locations, but still access information from the other modules. Additionally, different users can access images from different locations by having the user interface controls 205 situated on a computer associated with each user. The system may have a plurality of monitoring stations 206A at remote locations and each user can independently select the cameras 260A, 261A, 262A that the user wishes to view.
  • Further, the modules are designed as a system, and the system monitors the network bandwidth. By monitoring the network bandwidth, the system can automatically give priority to images from cameras that are associated with an alarm condition so that a user viewing a monitoring station 206A does not loose any information about an alarm due to bandwidth constraints in the network 270A. Further, bandwidth constraints can be accommodated by reducing the resolution of the images that are transmitted, but providing a higher resolution image from any camera 260A that is associated with an alarm state. Under extreme conditions, the display module 250 can stop the transmission of images that are not in the alarm state to accommodate network bandwidth limitations. In certain embodiments, the bandwidth of the system is predetermined and thus there is no need to actively monitor the bandwidth. However, the display module 250 still accounts for the bandwidth of the images that are being sent out into the network and monitor the percentage of bandwidth being utilized. Thus, based on the percentage of bandwidth that is utilized, the display module can determine the resolution of the images that are to be transmitted. The system thus, automatically and dynamically adjusts to the bandwidth conditions based on the resolution of the images being transmitted, the number of requested cameras, the number of users that are accessing images and the known network bandwidth.
  • Since the system is modular, command and control of the system through the administration module can be maintained at a remote location.
  • Also due to the system's modularity, the display system and the recording system are independent. If the display system fails, the digital video surveillance system is still capable of recording information. There are separate data paths to both the recording software and the display software. In the embodiment that is shown in Fig. 2, the video recording software 260 is legacy recording software. However, the invention disclosed is not limited to such an embodiment. A new recording program may be integrated into a legacy system. Further, the camera used in the system is shown as an analog camera. The camera is coupled to a video encoder which includes a A/D converter 270. In Fig. 2, the A/D converter is the only hardware that is present. All of the other modules are software components. The video encoder 270 also packetizes the data and converts the image data into a format capable of HTTP display. Further, the encoder accounts for any time division multiplexing of image information.
  • In the embodiment that is shown, the video encoder is a video encoder manufactured by Axis Communications. The video encoder includes at least two sensor inputs, two control inputs, a control output and a sensor output. In order to use the legacy video recording software, the control output is fed back into a sensor input. The alarm processing software generates a control signal indicating that an alarm condition has occurred to the control input of the video encoder. The legacy recording software program expects a sensor signal as an indication to begin recording. Thus, by using a control signal and feeding the control signal in a loop-back fashion as the input of a sensor input, the control signal from the alarm processing software will cause a sensor signal to be output and received by the recording software and recording will begin. This jumper allows recording to be controlled without having to create an additional protocol signal. Additionally, the control signal to the video encoder can cause the analog video signal to be sampled and converted to different sized digital formats, since the video recorder expects a sensor signal output from the video encoder to change formats.
  • The user of the system can control and change the status of a camera from an alarmed status to normal status by using the user interface 205 and indicating acknowledgement of the alarm condition.
  • Fig. 3A-3B shows a flow chart of how the software modules operate. First the database is accessed and the record set is retrieved 300. The record set includes the names and locations of each of the cameras along with the alarm conditions for each of the cameras. Alarm conditions may include relative evaluations such as greater than, less than and equal to. Further, an alarm condition may include a settable parameter, such as a threshold level. The alarm processing module retrieves the record set. Next the cameras are initialized 310. Each of the cameras is turned on and is set to a normal setting. The triggers sensors are also read and the alarm processing software module compares the signal from the trigger sensor(s) with the record set information. The system then waits for user interaction or for an alarm to occur 320. When an event occurs, the system checks to see if it was an alarm 325. If it is an alarm, the camera state is updated 330. The display software indicates that an alarm has occurred by including some indicia on the display. For example, the window displaying the camera images may be enclosed with a different colored border. A status variable for the camera is also updated to indicate that an alarm has occurred. If it is an alarm 335, text is added to the alarm list indicating the type of alarm 340. For example, the trigger sensors may be temperature sensors, contact sensors, smoke detectors etc. The alarm may indicate that there is a fire, a rise in temperature, an intruder or other type of alarm. The system then determines if the alarm is occurring at present. A check of the status of the trigger sensors is performed 345. If the alarm is still going on the system returns to point B on the flow chart. If the event is not an alarm then the system goes to point A on Fig. 3B. The system then queries whether the event is user input 350. If the answer is no the system either exits if the event is an exit request or returns to point B. If the event is user input, the system checks to see if the user has requested to increase the size of a view of a particular camera 352. If the answer is yes the system changes a variable and marks the camera as enlarged 355. The system then redraws the camera into a larger sized window and updates and redraws the labels 357. The system then continues onto point B. If the user input is not to enlarge the view of a camera, the system inquires if the user desires to remove the images of the camera from the display (turn off the camera) 360. If the user does want to remove the images of the camera, the system inquires whether the camera is in an alarmed state 362. If the camera is in an alarmed state, the user is notified that the user cannot remove the images from the display of an alarmed camera 365. The system then returns to point B. If the camera is not alarmed, the camera is marked as being off and the camera is removed from the display window and all of the remaining camera windows are resized to fit within the display 367. If the user has not requested to turn off the images from the camera, the system inquires whether the user has requested to change the status of an alarm 370. If the answer is yes, the visual display marks the alarmed camera window with indicia that the alarm has been acknowledged, but the alarm is still active 372. The system then inquires if all of the camera alarms have been acknowledged 375: If the answer is yes the outline (border) of the window for the camera is changed from the color associated with an alarm condition to that of a normal condition, but the indicia such as an asterisk remains to indicate that the camera/trigger sensors are still in an alarmed state 377. The system then resizes and redraws the windows 357 and the system returns to point B. If all of the alarms have not been acknowledged by the user, the system returns to Point B.
  • If the user has not selected to change the status of the alarm, the system then checks to see if the request by the user is for reconfiguring the database 380. The system at this point may check the status of the user to see if the user has been granted the privilege to change the database. If the answer to this inquiry is yes, the system looks to see what the user wishes to change 382. The user can change the alarm settings, including the alarm constraints, the name of a camera, and which cameras are being displayed, for example. If the user does not change a setting in the database the system returns to point B. If the user does make a change to the database then the data base is updated 385. A command is sent to the alarm processing software module which indicates that the database has been updated and the recordset is again downloaded to the alarm processing software module 387. The system then returns to point C. It should be understood that an administrator or user with the proper privileges can set the alarm such that an alarm requires a combination of events to trigger an alarm. For example, an alarm may be sounded only if a fire occurs and a flood occurs. In such a case, a temperature sensor may have to reach a particular temperature and a water level indicator may need to reach a particular level before an alarm occurs. An authorized user has control to change whether one of or more sensors will cause an alarm and the threshold parameter for each sensor.
  • Returning to Fig. 3A, if the event is an alarm 335, the user may acknowledge the alarm. If the user acknowledges the alarm 347, the system removes text from the alarm list display 348. The system then inquires whether the camera is still alarmed 349. If the camera is still in an alarmed state (the trigger sensors are set such that an alarm condition exists), then the system returns to point B. If the camera is no longer in an alarmed state the display software redraws the windows displaying the camera's images on the display screen along with updated labeling 390, i.e., returning to labeling indicative of the normal state.
  • It should be clear that the flow charts of Fig. 3A and 3B operate as a continuous loop.
  • Fig. 4 shows the interaction of the alarm and normal conditions. The system operates under a normal condition state for much of the time. The software routine as explained with respect to Figs. 3A and 3B continues to monitor the trigger sensors for an alarm condition. When an alarm condition occurs, the system automatically goes into an alarm condition state. When the system is in an alarm condition state, the system will remain in that state until the user acknowledges the state. The user of the system is provided with a graphical interface on a display device that shows one or more windows in which the images from a camera are displayed. When an alarm occurs, the graphical window is updated with text that indicates that an alarm has occurred along with a border around the window, such as a red border. If the images from a camera are not presently displayed and that the sensors for that camera experience an alarm condition, a new window will be provided for that camera. The user can then acknowledge that an alarm is occurring. This takes the system out of the alarm mode and resets the system back into the normal condition mode. When the acknowledgement occurs, the window for the camera which was having the alarm will be reset, such that the text will no longer indicate that an alarm is occurring, and the border around the window is changed back to the normal condition color. If the user acknowledges an alarm, and the alarm condition still exists an asterisk or other indicator will be marked on the graphical user interface which indicates that the sensors are still in an alarmed condition state, however the system will return to a normal condition state.
  • Fig. 5 shows the three modes of operation for the normal condition. A user can determine which cameras the user wants displayed and what information that the user would like stored. For example, in manual mode 510, the user selects the cameras to have displayed. The user may select one or multiple cameras to display. In manual mode 510, the user monitors the system and in the normal condition none of the images from any of the cameras are recorded and stored to memory. If the user selects the cycle mode 520, the user can select one or more cameras to have permanently displayed on the display, but at least one of the window cycles through the remaining cameras. For example, if there are eight possible cameras, the user may select to permanently display cameras 1-3 (521) and in the fourth window 525 the system will cycle through cameras 4-8. In the cycle mode, the system does not record and store information to a memory storage device. In the third mode, which is marked S.S. 530 the display operates much like the manual mode, but all the information from all of the cameras is recorded to memory 540 in digital form. The number of cameras shown in the Figs are provided for illustrative purposes and the system is not limited to a fixed number of cameras.
  • Fig. 6 shows the display screen 610 A,B,C and storage 620 A,B,C when different numbers of cameras are used under an alarm condition. When a single camera is used and an alarm condition arises the display 610A changes such that a textual indicator is provided, for example "alarm #1" and the window for the camera changes its border color. Other indicia might be used to indicate that an alarm has occurred such as having the picture flash or the border flash. During the alarm condition state, each camera that has an associated alarm condition will have its images stored to memory 620A. If the display window 610B is sized such that only four cameras are capable of being shown and there are no more than four cameras in the system, the display 610B will show each of the cameras and when an alarm occurs for one or more of the cameras one or more indicators that an alarm is occurring will be provided. For example, in the Fig., cameras 1-3 (612B) are each having an alarm. A textual message is provided and each of the camera windows for which there is an alarm has a changed border color. When the alarm condition occurs, the system automatically begins to record the camera images. If there are more than four cameras and only four cameras are capable of being displayed due to sizing of the windows on the display device 610C, one or more of the windows 615C will rotate. If an alarm occurs for one of the cameras that is not being displayed, a window will be initiated which will display the camera and will have some indicia that an alarm condition is occurring. If more than four cameras experience alarm conditions at the same time, the display 610C will rotate the images in different windows so that a user can view each alarmed camera. Again it should be understood that the number of cameras that are referenced in the Figs. is for illustrative purposes and the system does not have a fixed number of cameras nor is the camera limited to the number of camera images that can be displayed on a display device. For each camera that experiences an alarm condition, the digital images from the camera are stored in memory 620A, 620B, and 620C.
  • Figs. 5 and 6 show the video images being cycled through if there are more images than space to display the images on the display device, however there are other methods for displaying the images in both the alarm and normal conditions. For example, the user may be allowed to page through a group of cameras. In other embodiments, the current images from a camera may be displayed based upon the cameras that have been either most frequently requested in the past or a camera that has been requested over a certain past time period. The software also allows a user to prioritize the cameras that will be displayed, so that a camera that is marked with a high priority will be the last image to be removed from the display in the event of an alarm condition. It should be understood by one skilled in the art that other variations for displaying the images from a camera can be used without deviating from the scope of the invention.
  • Fig. 7 is a screen shot showing the user graphical interface. In this embodiment, sixteen cameras are visible at the same time and one of the cameras AC #2 (710) in the upper right hand corner is experiencing an alarm. The alarm condition is identifiable because the border around the window for camera AC #2 (710) has changed. To the right of the screen each camera of the sixteen possible cameras that is available is listed along with a check box (720), which when checked indicates that images from the camera will be displayed. In this fig. "select all cameras" is selected and all sixteen cameras are displayed at the same time.
  • Fig. 8 shows a different screen shot. This screen shot shows that only six of the sixteen possible cameras are selected and displayed to the user. In this screen 710 AC #2 is experiencing an alarm condition. The screen 710 for AC#2 is increased in size indicating that it is in an alarm condition. By increasing the size of the image, the user can better identify the cause of the alarm. AC#2 can also be identified as experiencing an alarm condition because the border around its window 710 is changed to a different color than the border around the windows of camera's operating in a normal state.
  • Although various exemplary embodiments of the invention have been disclosed, it should be apparent to those skilled in the art that various changes and modifications can be made which will achieve some of the advantages of the invention without departing from the true scope of the invention. These and other obvious modifications are intended to be covered by the appended claims.

Claims (4)

  1. A control system with two or more cameras and one or more sensors to form a video surveillance system, the control system receiving a request for images from one or more of the cameras over a network having an associated bandwidth, the control system comprising:
    a display module (150,250) for receiving the request for images and providing an output of the requested images from one or more of the cameras (105), wherein the display module monitors available network bandwidth and adjusts the output based on the available network bandwidth by reducing resolution of one or more images from one or more of the cameras, but giving priority to and providing a higher resolution image from any camera that is associated with an alarm state;
    characterized in that the system further comprises:
    an alarm processing module (240) for receiving at least one sensor signal from one or more sensors (230), each of the two or more cameras (105) being associated with one or more of the one or more sensors (230), wherein the alarm processing module (240) determines if there is an alarm state for a camera based in part on the at least one sensor signal, and wherein the display module (150,250) also receives input from the alarm processing module (240) which indicates if a camera is in an alarm state.
  2. A control system according to claim 1 wherein the display module (150,250) and the alarm processing module (240) are networked together so that each module can be located at a different remote location.
  3. A control system according to claim 1 wherein it further comprises:
    a video recorder module (130,260);
    wherein the alarm processing module (240) operates in a normal state until an alarm condition arises based upon the receipt of one or more sensor signals (230) associated with a camera (105), and then operates in an alarm state causing video images from the camera associated with the alarm to be recorded and saved to associated memory by the video recorder module (130,260).
  4. A method for controlling a video surveillance system operating in a network having an associated bandwidth, the method comprising:
    receiving a request originating from a location associated with a user interface in the network for transmission of images from one or more of a plurality of cameras from the video surveillance system;
    obtaining the images from the one or more of the cameras wherein the images are composed of digital data;
    determining if the digital data for the images to be transmitted exceeds bandwidth constraint;
    determining if a camera is in an alarm state;
    reducing the digital data until the digital data no longer exceeds bandwidth constraint by decreasing the resolution of images, but giving priority and providing a higher resolution to the images that are associated with an alarm state;
    and
    transmitting the images to the location associated with the user interface, wherein determining if a camera is in an alarm state comprises: receiving at least one sensor signal from one or more sensors, each of the plurality of cameras being associated with one or more of the one or more sensors, and determining if there is an alarm state for a camera based in part on the at least one sensor signal.
EP04811354A 2003-11-18 2004-11-17 Digital video surveillance Not-in-force EP1685543B1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PL04811354T PL1685543T3 (en) 2003-11-18 2004-11-17 Digital video surveillance
EP08075701A EP2019380A1 (en) 2003-11-18 2004-11-17 Digital video surveillance

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US52096703P 2003-11-18 2003-11-18
PCT/US2004/038626 WO2005050582A2 (en) 2003-11-18 2004-11-17 Digital video surveillance

Related Child Applications (1)

Application Number Title Priority Date Filing Date
EP08075701A Division EP2019380A1 (en) 2003-11-18 2004-11-17 Digital video surveillance

Publications (2)

Publication Number Publication Date
EP1685543A2 EP1685543A2 (en) 2006-08-02
EP1685543B1 true EP1685543B1 (en) 2009-01-21

Family

ID=34619544

Family Applications (2)

Application Number Title Priority Date Filing Date
EP04811354A Not-in-force EP1685543B1 (en) 2003-11-18 2004-11-17 Digital video surveillance
EP08075701A Withdrawn EP2019380A1 (en) 2003-11-18 2004-11-17 Digital video surveillance

Family Applications After (1)

Application Number Title Priority Date Filing Date
EP08075701A Withdrawn EP2019380A1 (en) 2003-11-18 2004-11-17 Digital video surveillance

Country Status (12)

Country Link
US (1) US20050162268A1 (en)
EP (2) EP1685543B1 (en)
JP (2) JP2007516521A (en)
CN (2) CN1898707B (en)
AT (1) ATE421739T1 (en)
AU (1) AU2004292297B2 (en)
DE (1) DE602004019243D1 (en)
ES (1) ES2320005T3 (en)
HK (1) HK1094073A1 (en)
IL (1) IL175751A (en)
PL (1) PL1685543T3 (en)
WO (1) WO2005050582A2 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111354174A (en) * 2018-12-24 2020-06-30 中移(杭州)信息技术有限公司 Alarm method, device, server and readable storage medium

Families Citing this family (56)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060004698A1 (en) * 2004-06-30 2006-01-05 Nokia Corporation Automated prioritization of user data files
US7728871B2 (en) * 2004-09-30 2010-06-01 Smartvue Corporation Wireless video surveillance system & method with input capture and data transmission prioritization and adjustment
US9189934B2 (en) * 2005-09-22 2015-11-17 Rsi Video Technologies, Inc. Security monitoring with programmable mapping
TWI303938B (en) * 2005-10-05 2008-12-01 Avermedia Information Inc Video surveillance system
TWI297118B (en) * 2005-10-24 2008-05-21 Avermedia Tech Inc Method for executing data compression with surveillance host
US7613360B2 (en) * 2006-02-01 2009-11-03 Honeywell International Inc Multi-spectral fusion for video surveillance
JP4442571B2 (en) * 2006-02-10 2010-03-31 ソニー株式会社 Imaging apparatus and control method thereof
JP4890880B2 (en) * 2006-02-16 2012-03-07 キヤノン株式会社 Image transmitting apparatus, image transmitting method, program, and storage medium
ES2677274T3 (en) * 2006-03-24 2018-07-31 Rsi Video Technologies, Inc. Security monitoring with programmable mapping
US20070268367A1 (en) * 2006-05-11 2007-11-22 Eran Agmon Video Surveillance With Satellite Communication Access
US7778445B2 (en) * 2006-06-07 2010-08-17 Honeywell International Inc. Method and system for the detection of removed objects in video images
US20080126533A1 (en) * 2006-11-06 2008-05-29 Microsoft Corporation Feedback based access and control of federated sensors
AU2006252090A1 (en) * 2006-12-18 2008-07-03 Canon Kabushiki Kaisha Dynamic Layouts
CN101266711B (en) * 2007-03-15 2011-02-23 中兴通讯股份有限公司 A video monitoring system and alarm recording network storage method
US8970703B1 (en) 2007-04-16 2015-03-03 The United States Of America As Represented By The Secretary Of The Navy Automatically triggered video surveillance system
US8605151B2 (en) * 2007-09-21 2013-12-10 Utc Fire & Security Americas Corporation, Inc. Methods and systems for operating a video surveillance system
JP4893649B2 (en) * 2008-02-08 2012-03-07 富士通株式会社 Bandwidth control server, bandwidth control program, and monitoring system
US20100058248A1 (en) * 2008-08-29 2010-03-04 Johnson Controls Technology Company Graphical user interfaces for building management systems
ITPO20080010A1 (en) * 2008-09-02 2010-03-03 Roberto Casati I SEE YOU (VI SURVEILLANCE) COMPLETE SYSTEM OF VIDEO SURVEILLANCE ACTIVATION SURVEILLANCE, ALERT, ON-LINE CHECK AND INTERACTION WITH DEMOTICS.
JP4674627B2 (en) * 2008-10-07 2011-04-20 富士ゼロックス株式会社 Information processing apparatus, remote instruction system, and program
CN101588490B (en) * 2009-06-24 2011-02-02 杭州华三通信技术有限公司 Method, system and apparatus for adjusting video quality grade
US8294560B2 (en) * 2009-07-20 2012-10-23 The United States Of America As Represented By The Secretary Of The Army Method and apparatus for identifying threats using multiple sensors in a graphical user interface
AU2010291859A1 (en) * 2009-09-01 2012-03-22 Demaher Industrial Cameras Pty Limited Video camera system
TWI407791B (en) * 2009-12-30 2013-09-01 Vtc Electronics Corp Intellectual monitoring system and monitoring method thereof
CN101883271B (en) * 2010-06-04 2014-02-05 中兴通讯股份有限公司 Method and system for controlling data code rate in wireless video conferencing system
US9449482B2 (en) * 2010-07-14 2016-09-20 Honeywell International Inc. Method and apparatus for activating and deactivating video cameras in a security system
CN102447884A (en) * 2010-10-14 2012-05-09 鸿富锦精密工业(深圳)有限公司 Automatic adjustment system and method of resolution for network camera
TW201308265A (en) * 2011-08-10 2013-02-16 Chunghwa Telecom Co Ltd Network monitoring method and system
MY177404A (en) * 2011-12-05 2020-09-14 Mimos Berhad Method and system for prioritizing displays of surveillance system
WO2013161345A1 (en) * 2012-04-24 2013-10-31 日産自動車株式会社 Monitoring system and monitoring method
TWI555407B (en) * 2012-07-18 2016-10-21 晶睿通訊股份有限公司 Method for setting video display
CN103632505A (en) * 2012-10-12 2014-03-12 韶关市奥诚电子技术有限公司 Event-driven network video alarm
CN103024348B (en) * 2012-11-06 2016-08-17 前卫视讯(北京)科技发展有限公司 The operation management system of video monitoring
MY188914A (en) * 2013-01-15 2022-01-13 Mimos Berhad A system and a method for determining priority of visuals
CN104079866B (en) * 2013-03-26 2018-06-29 株式会社日立制作所 Video monitoring data communication front-end equipment and video monitoring data front end processing method
JP6323985B2 (en) * 2013-04-18 2018-05-16 古野電気株式会社 Display device and information collecting method using the same
US9449491B2 (en) * 2013-04-23 2016-09-20 Canary Connect, Inc. Notifying a community of security events
KR101964230B1 (en) * 2013-12-16 2019-04-02 한화테크윈 주식회사 System for processing data
JP6415061B2 (en) * 2014-02-19 2018-10-31 キヤノン株式会社 Display control apparatus, control method, and program
KR102066944B1 (en) * 2014-07-09 2020-01-16 한화정밀기계 주식회사 Robot control system
EP3073702B1 (en) 2015-03-27 2017-09-06 Axis AB Method and devices for negotiating bandwidth in a peer-to-peer network
US10311307B2 (en) 2015-06-23 2019-06-04 Raytheon Company Methods and apparatus for video wall with feed indicators
CN105471850A (en) * 2015-11-17 2016-04-06 浙江宇视科技有限公司 Video uploading method and video uploading device
CN107221133B (en) 2016-03-22 2018-12-11 杭州海康威视数字技术股份有限公司 A kind of area monitoring alarm system and alarm method
CN107959872A (en) * 2016-10-18 2018-04-24 杭州海康威视系统技术有限公司 A kind of video switching method, device and video patrol system
WO2018119683A1 (en) * 2016-12-27 2018-07-05 Zhejiang Dahua Technology Co., Ltd. Methods and systems of multi-camera
JP6486397B2 (en) * 2017-02-06 2019-03-20 セコム株式会社 Data processing system and data processing method
US10760321B2 (en) * 2017-04-27 2020-09-01 Rob J. Evans Thermal door release system
KR20190121680A (en) * 2018-04-18 2019-10-28 엔쓰리엔 주식회사 Apparatus and method to transmit data by extracting data in shop floor image, apparatus and method to receive data extracted in shop floor image, and system to transmit and receive data extracted in shop floor image
CN111385463B (en) * 2018-12-29 2021-12-28 华为技术有限公司 Method for controlling camera, control device, network equipment and camera
CN110006471B (en) * 2019-04-02 2022-03-11 浙江宇视科技有限公司 Method and system for dynamically displaying sensor data
CN111540072B (en) * 2020-04-23 2021-04-02 深圳智优停科技有限公司 Parking space management service method, equipment and system
CN111784987B (en) * 2020-07-13 2021-03-23 深圳智乐未来机器人技术有限公司 Intelligent security alarm system
CN113992833A (en) * 2021-11-16 2022-01-28 浙江黄氏建设工程有限公司 Camera data acquisition system applied to future community
CN116112645B (en) * 2023-04-11 2023-11-21 重庆华悦生态环境工程研究院有限公司深圳分公司 Multi-image transmission method and device for reservoir environment
CN116486585B (en) * 2023-06-19 2023-09-15 合肥米视科技有限公司 Production safety management system based on AI machine vision analysis early warning

Family Cites Families (45)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US3652790A (en) * 1970-01-28 1972-03-28 Itt Search and tracking television system and method
US3891798A (en) * 1971-03-19 1975-06-24 Rockwell International Corp Tracker unit
FR2302004A5 (en) * 1973-08-31 1976-09-17 Thomson Csf AUTOMATIC TARGET TRACKING SYSTEM, PROCESSED BY VIDEO CONTRAST ANALYSIS
US3988533A (en) * 1974-09-30 1976-10-26 Video Tek, Inc. Video-type universal motion and intrusion detection system
US4120004A (en) * 1976-01-26 1978-10-10 Coutta John M Surveillance system
US4405940A (en) * 1977-05-31 1983-09-20 Westinghouse Electric Corp. Apparatus and method for preprocessing video frame signals
NL7801727A (en) * 1978-02-16 1979-08-20 Hollandse Signaalapparaten Bv THRESHOLD SELECTION CIRCUIT.
JP2528789B2 (en) * 1985-06-26 1996-08-28 中央電子 株式会社 Video information management device
KR920010034B1 (en) * 1988-03-31 1992-11-13 Toshiba Co Ltd Image track display apparatus
KR910004009A (en) * 1989-07-27 1991-02-28 강진구 Video camera automatic shooting device
JPH03130900A (en) * 1989-10-17 1991-06-04 Nippon Telegr & Teleph Corp <Ntt> Remote monitoring device
US5175694A (en) * 1990-02-08 1992-12-29 The United States Of America As Represented By The Secretary Of The Navy Centroid target tracking system utilizing parallel processing of digital data patterns
KR100204101B1 (en) * 1990-03-02 1999-06-15 가나이 쓰도무 Image processing apparatus
US5243418A (en) * 1990-11-27 1993-09-07 Kabushiki Kaisha Toshiba Display monitoring system for detecting and tracking an intruder in a monitor area
US5164827A (en) * 1991-08-22 1992-11-17 Sensormatic Electronics Corporation Surveillance system with master camera control of slave cameras
JPH06284148A (en) * 1993-03-30 1994-10-07 Hitachi Ltd Moving picture communication control method and communication controller
GB2321358A (en) * 1997-01-20 1998-07-22 O E M Project Management Ltd Video security system coupled to a fax modem
GB2329542B (en) * 1997-09-17 2002-03-27 Sony Uk Ltd Security control system and method of operation
GB2329541B (en) 1997-09-17 2002-05-29 Sony Uk Ltd Security control system
US6363380B1 (en) * 1998-01-13 2002-03-26 U.S. Philips Corporation Multimedia computer system with story segmentation capability and operating program therefor including finite automation video parser
JP3581566B2 (en) * 1998-06-10 2004-10-27 株式会社日立製作所 Monitoring system
CN1120455C (en) * 1998-07-10 2003-09-03 赵治平 Omnibearing electronic household guard and alarm method and system
JP3682179B2 (en) * 1998-08-31 2005-08-10 株式会社日立製作所 Monitoring system and network system
US6292098B1 (en) * 1998-08-31 2001-09-18 Hitachi, Ltd. Surveillance system and network system
US6583813B1 (en) * 1998-10-09 2003-06-24 Diebold, Incorporated System and method for capturing and searching image data associated with transactions
JP2000270500A (en) * 1999-03-18 2000-09-29 Toyo Commun Equip Co Ltd Insulation monitor system
US20040075738A1 (en) * 1999-05-12 2004-04-22 Sean Burke Spherical surveillance system architecture
US6476858B1 (en) * 1999-08-12 2002-11-05 Innovation Institute Video monitoring and security system
US6917288B2 (en) * 1999-09-01 2005-07-12 Nettalon Security Systems, Inc. Method and apparatus for remotely monitoring a site
JP4061821B2 (en) * 2000-05-24 2008-03-19 松下電工株式会社 Video server system
JP2002183845A (en) * 2000-12-15 2002-06-28 Toko Electric Corp Crime preventing device
JP2002252715A (en) * 2001-02-23 2002-09-06 Hitachi Commun Syst Inc Remote image supervisory system
DE20104329U1 (en) * 2001-03-14 2001-05-31 Vtq Videotronik Gmbh Monitoring device with sensors
TWI236294B (en) * 2001-05-11 2005-07-11 Ulead Systems Inc Method and device for capturing digital video
JP2003067875A (en) * 2001-08-23 2003-03-07 Radias:Kk System for managing vehicle and vehicle
JP4038355B2 (en) * 2001-09-26 2008-01-23 株式会社日立製作所 Monitoring system and recording device
JP2003125384A (en) * 2001-10-15 2003-04-25 Hitachi Building Systems Co Ltd Automatic door system
DE60213746T2 (en) * 2001-11-28 2007-08-16 Matsushita Electric Industrial Co., Ltd., Kadoma Security system for a house
JP2003289528A (en) * 2002-03-28 2003-10-10 Toshiba Corp Remote monitoring system and monitoring control method for the same
AU2003241143A1 (en) * 2002-06-25 2004-01-06 Quix Technologies Ltd. Image processing using probabilistic local behavior assumptions
US7244230B2 (en) * 2002-11-08 2007-07-17 Siemens Medical Solutions Usa, Inc. Computer aided diagnostic assistance for medical imaging
JP2004236235A (en) * 2003-01-31 2004-08-19 Canon Inc Imaging device
US20040184523A1 (en) * 2003-02-25 2004-09-23 Dawson Thomas Patrick Method and system for providing reduced bandwidth for picture in picture video transmissions
CN1445730A (en) * 2003-03-14 2003-10-01 长春卫通科技有限责任公司 Image storage alarming method and system
JP2005073218A (en) * 2003-08-07 2005-03-17 Matsushita Electric Ind Co Ltd Image processing apparatus

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111354174A (en) * 2018-12-24 2020-06-30 中移(杭州)信息技术有限公司 Alarm method, device, server and readable storage medium

Also Published As

Publication number Publication date
AU2004292297A1 (en) 2005-06-02
IL175751A (en) 2010-06-30
CN1898707B (en) 2011-08-10
CN101715110A (en) 2010-05-26
JP2011041311A (en) 2011-02-24
EP1685543A2 (en) 2006-08-02
EP2019380A1 (en) 2009-01-28
HK1094073A1 (en) 2007-03-16
IL175751A0 (en) 2006-09-05
ATE421739T1 (en) 2009-02-15
WO2005050582A3 (en) 2005-08-04
US20050162268A1 (en) 2005-07-28
JP2007516521A (en) 2007-06-21
ES2320005T3 (en) 2009-05-18
AU2004292297B2 (en) 2009-11-19
CN1898707A (en) 2007-01-17
PL1685543T3 (en) 2009-07-31
AU2004292297A2 (en) 2005-06-02
AU2004292297A9 (en) 2008-10-02
DE602004019243D1 (en) 2009-03-12
WO2005050582A2 (en) 2005-06-02

Similar Documents

Publication Publication Date Title
EP1685543B1 (en) Digital video surveillance
MXPA06013936A (en) Method and system for wide area security monitoring, sensor management and situational awareness.
US20160283074A1 (en) Infinite recursion of monitors in surveillance applications
KR100442170B1 (en) Remote Control and Management System
US6002995A (en) Apparatus and method for displaying control information of cameras connected to a network
US20060171453A1 (en) Video surveillance system
US20030117500A1 (en) Network video recording system
CN101341753A (en) Method and system for wide area security monitoring, sensor management and situational awareness
US7106364B1 (en) Camera control system
WO2008021052A2 (en) Rack interface pod with intelligent platform control
US11488458B2 (en) Systems and methods for providing an immersive experience of a facility control room using virtual reality
KR20090001699A (en) Operating system for integrated control room with versatile multivision
KR100792240B1 (en) Multi vision materialization method and system for the same
KR200492429Y1 (en) Integrated server for remote controlling
KR20020001234A (en) A Remote Security Device Using Internet And Method Thereof
KR20090056423A (en) Virtual matrix system and control method
KR101078885B1 (en) Method and apparatus for controlling surveillance keyboard based on network for IP(Internet Protocol) surveillance system
KR20240021551A (en) Server and facility integrated control method
KR20240041461A (en) Method and system for providing integrated control service for equipment with enhanced security
JP2009301408A (en) Plant monitoring system
KR20040103649A (en) Remote Management Service Method Using Digital Move Video Recorder
US20090244287A1 (en) Video monitoring system
KR100512946B1 (en) Closed circuit television capable of displaying a system ID and a method thereof
JP2005252818A (en) Surveillance camera program
WO2008020897A2 (en) Rack interface pod with intelligent platform control

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20060518

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LU MC NL PL PT RO SE SI SK TR

DAX Request for extension of the european patent (deleted)
17Q First examination report despatched

Effective date: 20060821

RIC1 Information provided on ipc code assigned before grant

Ipc: G08B 13/196 20060101AFI20070129BHEP

REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 1094073

Country of ref document: HK

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: INTERGRAPH SOFTWARE TECHNOLOGIES COMPANY

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LU MC NL PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REF Corresponds to:

Ref document number: 602004019243

Country of ref document: DE

Date of ref document: 20090312

Kind code of ref document: P

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2320005

Country of ref document: ES

Kind code of ref document: T3

REG Reference to a national code

Ref country code: HK

Ref legal event code: GR

Ref document number: 1094073

Country of ref document: HK

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090121

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090121

REG Reference to a national code

Ref country code: PL

Ref legal event code: T3

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090421

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090622

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090521

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090121

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090121

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090121

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090121

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090121

26N No opposition filed

Effective date: 20091022

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090421

REG Reference to a national code

Ref country code: NL

Ref legal event code: V1

Effective date: 20100601

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20091130

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20091117

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090422

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20091117

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20091130

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20100601

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20091130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20091117

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20091117

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20091117

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090722

REG Reference to a national code

Ref country code: PL

Ref legal event code: LAPE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090121

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20090121

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20151127

Year of fee payment: 12

Ref country code: GB

Payment date: 20151127

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20151126

Year of fee payment: 12

Ref country code: AT

Payment date: 20151103

Year of fee payment: 12

Ref country code: FR

Payment date: 20151117

Year of fee payment: 12

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602004019243

Country of ref document: DE

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 421739

Country of ref document: AT

Kind code of ref document: T

Effective date: 20161117

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20161117

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20170731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161117

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161117

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170601

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161118

REG Reference to a national code

Ref country code: ES

Ref legal event code: FD2A

Effective date: 20181121