WO2006107430A1 - Event-packaged video sequence - Google Patents

Event-packaged video sequence Download PDF

Info

Publication number
WO2006107430A1
WO2006107430A1 PCT/US2006/005943 US2006005943W WO2006107430A1 WO 2006107430 A1 WO2006107430 A1 WO 2006107430A1 US 2006005943 W US2006005943 W US 2006005943W WO 2006107430 A1 WO2006107430 A1 WO 2006107430A1
Authority
WO
WIPO (PCT)
Prior art keywords
event
video
interest
parameter
video clip
Prior art date
Application number
PCT/US2006/005943
Other languages
French (fr)
Inventor
Keith L. Curtner
Kwong Wing Au
Saad J. Bedros
Original Assignee
Honeywell International Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Honeywell International Inc. filed Critical Honeywell International Inc.
Priority to EP06735551A priority Critical patent/EP1864496A1/en
Publication of WO2006107430A1 publication Critical patent/WO2006107430A1/en

Links

Classifications

    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/19Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier
    • G11B27/28Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/73Querying
    • G06F16/732Query formulation
    • G06F16/7335Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/783Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/7837Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using objects detected or recognised in the video content
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19602Image analysis to detect motion of the intruder, e.g. by frame subtraction
    • G08B13/19606Discriminating between target movement or movement in an area of interest and other non-signicative movements, e.g. target movements induced by camera shake or movements of pets, falling leaves, rotating fan
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19602Image analysis to detect motion of the intruder, e.g. by frame subtraction
    • G08B13/19613Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19665Details related to the storage of video surveillance data
    • G08B13/19669Event triggers storage or change of storage policy
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19665Details related to the storage of video surveillance data
    • G08B13/19671Addition of non-video data, i.e. metadata, to video stream
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19678User interface
    • G08B13/1968Interfaces for setting up or customising the system
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19678User interface
    • G08B13/19682Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system
    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/19Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier
    • G11B27/28Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording
    • G11B27/32Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording on separate auxiliary tracks of the same or an auxiliary record carrier
    • G11B27/322Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording on separate auxiliary tracks of the same or an auxiliary record carrier used signal is digitally coded
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/18Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources

Definitions

  • the present invention relates generally to the field of video image processing. More specifically, the present invention pertains to methods and systems for packaging video sequences based on user-specified events.
  • Video surveillance systems are used in a variety of applications for monitoring objects within an environment.
  • security applications for example, such systems are sometimes employed to track individuals or vehicles entering or leaving a building facility or security gate, or to monitor individuals within a store, office building, hospital, or other such setting where the health and/or safety of the occupants may be of concern.
  • aviation industry for example, such systems have been used to monitor the presence of individuals at key locations within an airport such as at a security gate or parking garage.
  • the video surveillance system may be tasked to record video image data for later use in determining the occurrence of a particular event.
  • video images are typically stored as either analog video streams or as digital image data on a hard drive, optical drive, videocassette recorder (VCR), or other suitable storage means.
  • An illustrative method of event-packaging a video sequence may include the steps of acquiring a video stream containing an event of interest, extracting various event information from the video stream and storing one or more event parameters relating to the event within an event database, extracting a video clip from the video stream containing the event of interest, associating the event parameters to the video clip, and exporting the video clip containing the event of interest and associated event parameters to an external agent such as a terminal station or networked storage device.
  • a video image-processing appliance manager and/or one or more appliance modules can be provided to automatically extract event information from the acquired video stream, and/or to manage the indexing and storage of event parameters within the event database.
  • a graphical user interface may be provided to permit the event information extracted from the video stream to be compared against a set of user-specified event parameters.
  • the user-specified event parameters may comprise a set of semantic parameters that can be used to detect and analyze various events occurring within one or more regions of interest.
  • Other features such as a means for adjusting the pre-record and/or post record duration of the extracted video clip and , a means for providing annotation information along with the extracted video clip can be further provided on the graphical user interface, if desired.
  • Figure 1 is a block diagram showing an illustrative network-based video monitoring system for use in monitoring one or more regions of interest within an environment;
  • FIG. 8 Figure 2 is a block diagram showing the architecture of a video image-processing appliance manager in accordance with an exemplary embodiment of the present invention
  • Figure 3 is a flow chart showing an illustrative method of defining and analyzing events occurring within a region of interest of a video monitoring system in accordance with an exemplary embodiment of the present invention
  • Figure 4 is a pictorial view showing an illustrative video frame including multiple regions of interest
  • Figure 5 is another pictorial view of the illustrative video frame of Figure 4 showing the juxtaposition of a minimum boundary rectangle of an object within a region of interest;
  • Figure 6 is a flow chart showing an illustrative method of event-packaging a video sequence in accordance with an exemplary embodiment of the present invention.
  • Figure 7 is a pictorial view showing an illustrative graphical user interface in accordance with an exemplary embodiment of the present invention.
  • FIG. 1 is a block diagram showing an illustrative network-based video monitoring system 10 employing a number of digital video system sensors (DVSS's).
  • System 10 illustratively a video surveillance system for use in detecting and monitoring objects within an environment, may comprise a network 12 such as a wired or wireless local area network (LAN) or wide area network (WAN) that can be used to connect one or more DVSS's 14 to a network server 16 or other such host application.
  • LAN local area network
  • WAN wide area network
  • a DVSS can be understood generally as a subsystem that provides sequences of digital video images or other desired data to the monitoring system 10.
  • Examples of DVSS's that can be employed by the system 10 may include, but are not limited to, one or more digital cameras, analog cameras equipped with a digitizer, frame grabbers, digital video recorders, files of stored video data in a database, microphones for audio signal capture, accelerometers for seismic data capture, pressure sensors, temperature sensors, level and/or flow sensors, proximity sensors, contact sensors ⁇ e.g. window or door switches), and/or multi-spectral sensors.
  • a number of browsers or terminal stations 1 8 equipped with a user interface 20 can be connected to the network 12 and tasked to control the operation of the DVSS's 14 in a particular manner.
  • a terminal monitor 22 e.g. a closed-circuit television monitor
  • the browsers or terminal stations 1 8 can be configured to interact with a host application software program 24 that can be used to task the DVSS's 14 in a particular manner.
  • the host application software program 24 can be used to change operational parameters of the DVSS's 14 (e.g. PTZ settings, ROI settings, resolution, frame rate, etc.), and/or to task a video image processing appliance manager 28 to perform certain user- defined tasks such as motion detection, motion tracking, etc. If, for example, the user desires to increase the resolution of images received , by one or more of the DVSS's 14 in order to perform facial recognition of an individual within a region of interest, the host application software program 24 can be configured to accept a command causing one of the DVSS's 14 to zoom-in on the subject's face and increase the image capture rate.
  • operational parameters of the DVSS's 14 e.g. PTZ settings, ROI settings, resolution, frame rate, etc.
  • DVSS's 14 are shown connected to the network 12, each of which can be tasked to acquire video and/or still images within a respective field of view (FOV) represented generally by dashed lines 26. It should be understood, however, that a greater or lesser number of DVSS's 14 may be employed, if desired. As shown in Figure 1 , the DVSS's 14 can be connected to the network 1 2 directly, through the host application software program 24, or both, as desired.
  • the host application software program 24 can be configured to call a video image- processing appliance manager 28 that can be used to facilitate processing of video images received from the DVSS's 14 using a number of plug-in appliance modules 30.
  • the appliance manager 28 can be adapted to interface with the host application software program 24 as well as other components within the system 10.
  • the video images and/or other information acquired by the DVSS's 14 can be fed through the appliance manager 28, which tasks the appropriate appliance modules 30 to determine the occurrence of any events configured within the system 10. If an event is triggered, the appliance manager 28 can be configured to output a response ⁇ e.g. via the user interface 20) indicating that an event has occurred within a region of interest.
  • the video images triggering the event can be stored on a hard-drive, magnetic tape, or other storage medium, allowing the video images to be replayed and/or subsequently processed.
  • the video images acquired from the DVSS 14 can also be displayed in real-time on the terminal monitor 22.
  • FIG. 2 is a block diagram showing the architecture of a video image-processing appliance manager 32 in accordance with an exemplary embodiment of the present invention.
  • the appliance manager 32 may provide an interface to a number of host applications 34, which as is described above with respect to Figure 1 , can include various software algorithms and /or functionality for operating a number of DVSS components.
  • the host applications 34 may include, for example, an existing host application software , program configured to interact with one or more video cameras (e.g. DVSS's) each adapted to acquire video images within a field of view (FOV).
  • video cameras e.g. DVSS's
  • FOV field of view
  • the appliance manager 32 can also be interfaced with other types of DVSS's including, but not limited to, frame grabbers, digital video recorders, files of stored video data in a database, microphones for audio signal capture, accelerometers for seismic data capture, pressure sensors, temperature sensors, level and/or flow sensors, proximity sensors, contact sensors (e.g. window or door switches), and/or multi-spectral sensors, as desired.
  • the host applications 34 can comprise separate components from the DVSS's (e.g. a stand-alone software package), or can be formed integral with one or more of the DVSS's and provided as a single component, if desired.
  • one or more of the DVSS's may comprise a physically separate video camera that is connected to an existing software- based host application adapted to run on the Internet, an intranet connection, and/or on an individual workstation equipped with a user interface 36.
  • each of the associated DVSS's can be connected to their associated host application 34 using an application program interface (API) or other suitable interface.
  • API application program interface
  • the host applications 34 may comprise one or more existing host application software programs contained on a network server, browser, terminal station, or other platform.
  • the functionality provided by the existing host applications 34 will typically vary depending on their intended use. If, for example, the host applications 34 are adapted to interface with network-based control, access, and security systems/products, the host applications 34 may include an existing security software program that can be used to task a number of DVSS's to pan, tilt, and zoom to a tracked motion within a region of interest.
  • Examples of other types of host applications 34 may include, but are not limited to, building management applications (e.g. HVAC control), life safety applications (e.g. fire protection, medical care, etc.), asset location applications, and energy management applications.
  • the appliance manager 32 will typically comprise a separate module from the host applications 34, allowing the appliance manager 32 to be linked with the user's existing system without having to significantly modify or reprogram the existing software to accommodate new DVSS clients as they are added to the system. It should be understood, however, that the appliance manager 32 and host applications 34 could be incorporated together as a single, standalone module, if desired.
  • the appliance manager 32 can be configured to maintain one or more business objects, which can include various information about the region or regions of interest to be monitored, any events configured by a configurator 44, as well as various configuration information about the host applications 34 connected to the system.
  • the appliance manager 32 can be configured to maintain a camera structure list and an event structure list containing information about the type of DVSS's employed and the type of events configured within the system.
  • Such lists can be stored within a memory unit or database ⁇ e.g. database 38) and recalled each time the appliance manager 32 receives an initialization call 40 or detection call 42 from one or more of the host applications 34.
  • the video monitoring system may include a configurator 44 that can be used in the direct manipulation and configuration of images or other data received by the host applications 34.
  • a tuning request/call 46 received from one or more of the host applications 34 can be used by the configurator 44 to tune the appliance manager 32 and/or other desired system components to function in a particular manner.
  • the host application 34 can be configured to send a tuning request or call 46 to the appliance manager 32 that can be used by the configurator 44 to coordinate such a change. Once the change has been made, the appliance manager 32 can then be configured to pass the newly configured video stream through.
  • the configurator 44 can also be configured to send a response to the host application 34 and /or to the user indicating whether the tuning invocation succeeded or failed.
  • the appliance manager 32 can be connected to a database 38 that can be configured to store information received from the DVSS's as well as parameters received by the configurator 44, as directed by i the appliance manager 32 and/or user via the user interface 36.
  • the database 38 can be two separate databases residing at different servers, wherein one database can be linked to the host application 34 and the other database can be linked to the appliance manager 32.
  • the database 38 may comprise a single database, or multiple databases existing on a single server.
  • the appliance manager 32 can be configured to interact with a number of plug-in appliance modules each adapted to run various video image-processing algorithms or routines that can be used to perform certain user-defined image processing functions.
  • the appliance manager 32 is shown operatively connected to a video motion detection (VMD) module 48, a video motion tracking (VMT) module 50, an object classification (OC) module 52, an event detection module 54, and an action dispatcher module 56.
  • VMD video motion detection
  • VMT video motion tracking
  • OC object classification
  • the various appliance modules 48,50,52,54,56 can be provided as either software appliances adapted to be run on a network or terminal server, or as separate hardware units that can be plugged into the appliance manager vis-a-vis a network bridge or other suitable connection.
  • These modules 48,50,52,54,56 can be upgraded, when needed, to enhance performance and/or meet specific host application requirements.
  • the appliance manager 32 can be configured to permit the modular incorporation of future appliance modules 58, as desired. If, for example, the user desires to add a facial detection module or rapid eye detection module to the video monitoring system, the appliance manager 32 can be configured to accept a software call, network socket, physical port ⁇ e.g. a USB port, Firewire/IEEE 1 394 port, parallel/serial port, etc.), and/or wireless port that can be used to add the additional modules. Since an appliance manager 32 is provided to interface with the host applications 34, the addition of future plug-in modules does not require the user to re-code or re-formulate the existing host applications 34. In addition, the appliance manager 32 may provide the user with the ability to change the implementation and/or features of existing functionality without significantly affecting the operation of the DVSS's.
  • the appliance manager 32 can be configured to run a video image-processing algorithm or routine that continuously monitors the camera structure list and configured events list to determine whether an event has been detected by one or more of the DVSS's. When an event contained within the event list is detected, the appliance manager 32 can be configured to transmit a result 60 back to the host application 34 along with an action request 62 tasking one or more of the DVSS's to operate in a particular manner.
  • the appliance manager 32 can be configured to provide a result 60 to the appropriate host application 34 informing it that the object is being tracked along with an action request 62 tasking one or more associated DVSS's to track the object.
  • FIG. 3 an illustrative method 64 of defining and analyzing events occurring within a video monitoring system will now be described in the context of the illustrative appliance manager 32 of Figure 2.
  • method 64 may begin at block 66, wherein a call is received by the appliance manager 32, causing the appliance manager 32 to initialize an algorithm or routine therein that can be used in defining, detecting, analyzing, indexing, and/or retrieving one or more events of interest.
  • appliance manager 32 may be called when a user and/or host application desires to task one or more DVSS's 14 to detect the occurrence of an event within a field of view.
  • the appliance manager 32 can be configured to task one or more of the appliance modules to extract information regarding one or more objects located within a region of interest, as indicated generally by reference to block 68.
  • Information that can be extracted at this step may include, for example, information about an object's motion, trajectory, orientation, size, aspect ratio, color, lighting, temperature, and/or information about an object's type or classification (e.g. "human”, “animal”, “vehicle”, “animate”, “inanimate”, etc.).
  • Such extracted information can be obtained using one or more of the appliance modules 48,50,52,54 described above with respect to Figure 2.
  • the appliance manager 32 tasks the video motion detection module 48 and video motion tracking module 50 to run separate algorithms or routines that can be used to perform such tasks.
  • Information regarding the classification of the object can be determined by invoking the object classification module 52 and running an algorithm or routine therein that determines whether an object is a vehicle.
  • the appliance manager 32 can be configured to task a video face detection module and/or video face tracking module to run separate algorithms or routines that can be used to gather information to perform facial recognition on individuals.
  • the types of information extracted by the monitoring system will typically vary depending on the types of video cameras employed, the location of the video cameras, the particular appliance module(s) available to the system, as well as other factors.
  • each event of interest can be specified by a set of conditions and/or sub-conditions that identify the object and semantically describe its state and/or physical features relative to a particular region of interest.
  • each condition and/or sub-condition can be quantified by a set of semantic parameters that can be compared against a corresponding set of semantic parameters programmed within the event detection module 54. Examples of such semantic parameters may include, but are not limited to, information regarding the region of interest (e.g.
  • semantic parameters relating to the type or classification of object detected e.g. “human”, “animal”, “vehicle”, “animate”, “inanimate”, “other”, “any”, etc.
  • semantic parameters relating to the type of motion ⁇ e.g. walk, run, high speed, low speed, etc.
  • Semantic parameters relating to the physical appearance e.g. color and size of object, single or group of people
  • for a particular detected face e.g. "happy, "neutral”, “sad”, etc.
  • such user-specified semantic parameters can be stored as event parameters within an image database (e.g. database 38), and can be provided to the event detection module 54 through the appliance manager 32 via a graphical user interface or other suitable interface. Indexing and/or subsequent retrieval of such events can also be accomplished in a similar manner using such semantic parameters, if desired.
  • the appliance manager 32 invokes the action dispatcher module 56 to set up the appropriate action request or requests and perform an action response, as indicated generally by reference to block 74.
  • the action dispatcher module 48 can be configured to trigger an alarm or other such response to notify a user when a configured event has been detected.
  • the appliance manager 32 can also be configured to record a video clip containing the detected event and/or send a video feed to a terminal station, browser, network server, or other such location for further analysis by a user and/or host application.
  • the video feed may contain one or more supporting event parameters.
  • Other action responses such as that described below with respect to the illustrative graphical user interface 1 22 of Figure 6 can also be performed, if desired.
  • Figure 4 is a pictorial view showing an illustrative video frame 76 including multiple regions of interest.
  • video frame 76 may define a pixel area 78 having a first region of interest 80 and a second region of interest 82.
  • the first and second regions of interest 80,82 may represent, for example, regions of interest within a security zone or other such area in which the monitoring of individuals, vehicles, or other objects of interest is desired.
  • the first region of interest 80 may represent an entranceway of a building or structure whereas the second region of interest 82 may represent a parking garage or carport.
  • regions of interest 80,82 are specifically depicted in the illustrative view of Figure 4, it should be understood that a greater or lesser number of regions of interest could be defined. Moreover, while the two regions of interest 80,82 are shown having a substantially rectangular shape in Figure 4, it should be understood that the regions of interest 80,82 may assume other polygonal shapes within the video frame 76. If, for example, a particular region of interest to be monitored comprises a parking lot, then a more complex polygonal shape approximating the boundaries of that region may be utilized.
  • the first and second regions of interest 80,82 may be shaped as any closed polygon in the image defined by the vertices of the polygon.
  • the first region of interest 80 is shown as a polygon ⁇ e.g. a rectangle) having a left side boundary 84, a top side boundary 86, a right side boundary 88, and a lower side boundary 90, which define a first pixel area 92 within the video frame 76.
  • the second region of interest 82 is shown as a polygon having a left side boundary 94, a top side boundary 96, a right side boundary 98, and a lower side boundary 100, which define a second pixel area 102 within the video frame 76.
  • the pixel areas 92,102 defined by each respective region of interest 80,82 are non- overlapping such that none of the image pixels contained in the first pixel area 92 overlap with any of the image pixels contained in the second pixel area 1 02. It should be understood, however, that other , embodiments are envisioned wherein portions of the regions of interest 80,82 overlap one another.
  • other polygons having a non-rectangular shape may be defined as the ROI.
  • the major and minor axes of the ROI are determined and a rectangular region defined by the major and minor axes can be used for evaluating the event conditions, as described below.
  • the first and second regions of interest 80,82 may each define an ROI boundary area 104,106, which as indicated by shading, can be located about the outer periphery of each respective pixel area 92,1 02.
  • Each ROI boundary area 104, 106 can be defined as a percentage or fraction of the total number of pixels contained within the pixel area 92, 1 02.
  • the first and second regions of interest 80,82 may have respective ROI boundary areas 104,1 06 defined as percentage values x ⁇ % and X2%, respectively, wherein each percentage value xi%, xz% is expressed as a percentage of the pixel area of the ROI boundary area 104,106 relative to their corresponding pixel area 92,102.
  • the size, shape and locations of the ROI boundary areas 1 04,1 06 may vary depending on the type of region selected by the user, the characteristics of the objects located within that scene, as well as other factors.
  • ROI boundary areas 1 04, 1 06 are specifically depicted in the illustrative embodiment Figure 5, it should be understood that other types of ROI boundary areas 1 04,1 06 can be utilized.
  • the ROI to be monitored is a building or structure having an irregular shape
  • the ROI boundary area 104,1 06 used to define that ROI may have also have an irregular shape to better approximate the ROI.
  • Such irregular shape may be defined, for example, by selecting a greater number of reference points on the video frame 76, using curved lines to define the boundary area, and/or by some other suitable technique.
  • the number of ROI's located within the video frame 76 may vary depending on the number of ROI's to be monitored.
  • Figure 5 is another pictorial view of the illustrative video frame 76 of Figure 4 showing the juxtaposition of a minimum boundary rectangle 1 08 of an object within the first region of interest 80.
  • the minimum boundary rectangle 108 may in 1 clude an outer boundary 1 10 defining an area 1 12 containing an object OBJ such as an individual or vehicle.
  • the minimum boundary rectangle 108 is shown covering a lower-left portion of the ROI boundary area 1 04, intersecting the left side boundary 84 and bottom side boundary 90 of the first region of interest 80.
  • a number of events relating to the object's motion relative to the first region of interest 80 can be determined by comparing the coordinates of the ROI boundary area 104 with a corresponding set of coordinates of the minimum boundary rectangle 108.
  • an upper-left location 1 14 and lower-right location 1 16 of the ROI boundary area 104 can be assigned a set of coordinates of (RXI .RYI) and (R ⁇ 2,R ⁇ 2), respectively.
  • an upper-left location 1 18 and lower-right location 120 of the minimum boundary rectangle 108 can be assigned a corresponding set of coordinates (OXI ,OYI) and (O ⁇ 2, O ⁇ 2), respectively.
  • a similar coordinate scheme can be utilized for other objects and regions of interest situated within the video frame 76, as desired.
  • the sets of upper-left and lower-right coordinates (RXI ,RYI),(RX2,RY2) and (O ⁇ i,O ⁇ ),(O ⁇ 2,O ⁇ 2) can be used to define various event conditions (i.e. semantic parameters) relating to the object's location and direction relative to the ROI boundary area 104 by comparing the corresponding coordinate values.
  • the equation 0 ⁇ O ⁇ 2 - Rxi ⁇ xi%(R ⁇ 2 - Rxi) will be satisfied, thus indicating that a "Left" event has occurred.
  • the equation 0 ⁇ R ⁇ 2 - Oxi ⁇ xi%(R ⁇ 2 - Rxi) will be satisfied, indicating that a "Right" event condition has occurred.
  • event conditions ⁇ e.g. "Start Track”, “Enters Region”, “Stop Track”, “Exits Region”, etc.
  • other event conditions ⁇ e.g. "Start Track”, “Enters Region”, “Stop Track”, “Exits Region”, etc.
  • the monitoring system can be configured to combine the two semantic parameters into a string of semantic parameters using a suitable Boolean operand such as "and" or "+”. Once the event conditions ⁇ i.e. semantic parameters) are combined, the monitoring system can then be configured to match these event conditions with a corresponding set of event conditions specified by the user.
  • method 1 22 may begin at block 124, wherein a DVSS or network of DVSS's is/are tasked to acquire a video stream containing an event of interest (block 1 26).
  • the acquired video stream may comprise a digital video stream obtained from an analog camera having a digital IP-addressed streamer, or, in the alternative, a digital camera having a straight digital IP-addressed streamer.
  • method 122 may include the step of extracting various event information in a manner similar to that described above with respect to Figure 3, wherein information regarding an object located within a region of interest is compared and matched against, a set of event parameters specified by a user.
  • event parameters may include metadata such as the date and time of the event, a header identifying the configured event, the camera or cameras tasked to acquire the video stream containing the event, the location of the event, the type of event detected, the lighting conditions detected by the camera or cameras, the classification of the object triggering the event (e.g. individual, vehicle, animal, animate, inanimate, etc.), the direction of object motion, the color of the object (e.g. red, blue, dark, light, etc.), and/or facial information.
  • event information can be extracted using the video image processing appliance manager 32 and/or one or more of the appliance modules 40,42,44,46 described herein. Once extracted, the event information can then be indexed and stored as event parameters within an event database (e.g. database 38) for subsequent use.
  • a video clip containing the desired event may be extracted from the video stream and then optionally stored within a video database, as indicated generally by reference to block 130.
  • a user-selectable duration of the video stream can be recorded as a video clip within the video database, and can be associated with the one or more of the event parameters ⁇ e.g. event description, date, time, camera number, etc.) representing the event. Other characteristics describing the object or objects triggering the event as well as each , object's interaction with the region of interest may also contained within the event database, if desired.
  • a user interface ⁇ e.g.
  • a GUI can be employed to permit the user to adjust the pre-record and/or post-record duration of the video clip, the space or memory allocated to the video clip, the format of the video clip ⁇ e.g. MPEC, ivlJPEG, etc.), the aspect ratio of the video frames within the video clip, as well as other settings.
  • Annotation information such as object labels, motion trails, motion direction arrows, highlighting of detected objects, etc. can also further provided along with the event- packaged video clip, if desired.
  • the video clip and associated metadata can then be exported to an external agent, as indicated generally by reference to block 132.
  • the event-packaged video clip may be sent to the external agent either directly or when prompted by the user via the user interface.
  • the event parameters that supported the detection can be associated and sent with the video clip.
  • the databases containing the event information and associated video clips may then be searched using semantic parameters similar to that described above with respect to Figures 4-5. If, for example, the user desires to search the event database for those events containing an individual entering a particular region of interest, the user may input a command into a user interface causing any video clips associated with that particular event to be retrieved from the video database.
  • the video clip containing the event as well as the event parameters used to define the event can then be exported to an external agent for further analysis, if desired.
  • event-packaged video clip can be exported to the external agent via a network connection, electronic mail, PDA, phone line, or other suitable transmission means.
  • the event-packaged video clip can also be sent to other locations within the monitoring system such as a terminal station, network storage device or other system appliance.
  • the search time usually required to locate an event within a video sequence may be reduced significantly.
  • such event-packaged video clips may reduce the time required for an operator to respond to an event, allowing quicker response times to be achieved in some applications.
  • Graphical user interface 1 34 may include a display screen 1 36 configured to display various information related to events configured within the appliance manager 32 as well as the various appliance modules ⁇ e.g. the event detection module 54).
  • the graphical user interface 1 34 may include a REGIONS OF INTEREST section 1 38 that permits the user to identify various regions of interest within the field of view (FOV) of a DVSS in the monitoring system.
  • FOV field of view
  • the REGION OF INTEREST section 1 38 may include a DVS ID menu button 140 that can be used to specify where the region of interest is location, and a REGION NAME menu button 142 that can be used to name a particular region of interest (e.g. "Carpark A") in which to detect events.
  • the region name uniquely associates with an area in the FOV of the DVSS whose identity is specified by the DVS ID menu button 140. This area can be displayed and annotated in the display screen 1 36 of the graphical user interface 134.
  • the graphical user interface 134 can be configured to display a list of those regions of interest available for detection within the FOV of the DVS ID 140 by the monitoring system.
  • the graphical user interface 1 34 may include a PARAMETER SETTING section 144 that permits the user to specify the parameter values of the various appliance modules.
  • a SCENARIO menu button 146 within the PARAMETER SETTING section 144 can be provided to permit the user to adjust the sensitivity at which events are detected. If, for example, the selected region of interest is located indoors where lighting conditions are typically bright, the user may select "Indoors (High)" or other appropriate text using the SCENARIO menu button 146.
  • a SUB-SAMPLING menu button 148 and MINIMUM OBJECT SIZE text box 1 50 may also be provided to 1 permit the user to adjust the sub-sampling rate of images captured and/or to set the minimum size of objects to be detected.
  • a CLASSIFIER menu button 1 52 may be provided to permit the user to specify which classifier and its associated parametric values to be used in classifying the object.
  • Other parameters in addition to those specifically depicted in Figure 7 may also be provided to facilitate operation and/or configuration of the monitoring system, if desired.
  • a VIDEO MONITOR section 1 54 of the graphical user interface 1 34 can be configured to display video images and/or still images acquired by one or more of the DVSS's within the monitoring system.
  • the DVS ID which captures the video image, may also be superimposed on the VIDEO MONITOR section 1 54.
  • the graphical user interface 1 34 is shown displaying a first region of interest 1 56 in an upper-left portion of the VIDEO MONITOR section 1 54, and a second region of interest 1 58 in a lower-right portion of the VIDEO MONITOR ' section 1 54.
  • Each region of interest 1 56, 1 58 can be demarcated on the display screen 1 36 using a dashed boundary box 1 60,1 62 or other suitable visual indicator.
  • the corresponding region name e.g. "Carpark A”, “Security Zone 1 ", etc.
  • 1 64,1 66 may also be superimposed on each region of interest 1 56, 1 58 displayed on the display screen 1 36, if desired.
  • An EVENTS Tu DETECT WITHIN REGION OF INTEREST section 168 of the graphical user interface 1 34 can be provided to permit the user to select those event conditions to be detected within the particular region of interest selected via the REGION NAME menu button 142.
  • a list of event conditions currently programmed within the monitoring system can be displayed within a text box 1 70 located within section 168.
  • An ADD icon button 1 72 located to the right of the text box 1 70 can be provided to permit the user to add additional event conditions to be detected by the monitoring system.
  • the current event condition selected within the text box 170 can be highlighted by blinking text, italics, inverted text, or other suitable visual means.
  • the ADD icon button 1 72 the user can be prompted to enter the desired event conditions.
  • An EVENT CONDITION DETAILS section 1 76 of the graphical user interface 1 34 can be configured to permit user entry as well as display those details associated with each event condition configured within the monitoring system.
  • an OBJECT TYPE menu button 1 78 and a CONDITION TO DETECT menu button 1 80 can be provided to permit the user to select the type of object and event condition(s) to be detected. Jf, for example, the user desires to monitor vehicles entering a carpark, the user may select "vehicle" using the OBJECT TYPE menu button 1 78 and "enters region" using the CONDITON TO DETECT menu button 1 80.
  • Examples of other object types or classifications that can be selected using the OBJECT TYPE menu button 1 78 may include “individual”, “animal”, “animate object”, “inanimate object”, “any object”, “unknown object”, etc.
  • Examples of other types of event conditions that can be selected using the CONDITION TO DETECT menu button 180 may include “start track”, “stop track”, “exits region”, “movement detected”, etc.
  • a DIRECTION OF MOTION menu button 182 can be provided to permit the user to select the direction or directions in which an object triggers an event. If, for example, the user desires to detect only those events in which an object travels through a region of interest from the left, the user may select "From Left" using the DIRECTION OF MOTION menu button 1 82, causing the monitoring system to perform an action response only when the object is traveling from this direction.
  • the user may select “From Right”, “From Above”, and/or “From Below” or other appropriate text using the DIRECTION OF MOTION menu button 1 82, causing the monitoring system to perform an action response only when the object is traveling from these directions.
  • the graphical user interface 1 34 may permit the user to select multiple directions (e.g. "From Left” and “From Below)” in which to detect and/or track object motion.
  • 1 34 can be configured to permit the user to specify more complex motion to be detected and tracked. If, for example, the user desires to detect when an object traverses a serpentine-like path within a particular region of interest, the user may select "Serpentine Path" or other appropriate text using the DIRECTION OF MOTION menu button 1 82, causing the monitoring system to perform an action response when the object travels in such pattern. Other user-specified directions and/or paths can be further implemented in similar fashion via the DIRECTION OF MOTION menu button 1 82, if desired.
  • a VIDEO CAMERA menu button 1 84 can be provided to permit the user to select those video cameras to be tasked in detecting events. If, for example, the user desires to separately task one or more video cameras (e.g. "Camera 1 ", “Camera 2", “Cameras 1 & 2", etc.) to detect object motion within a particular region of interest, the user may select the appropriate camera name using the VIDEO CAMERA menu button 1 84. Alternatively, and as shown in Figure 7, if the user desires to task all of the video cameras to detect a particular event, the user may select "Any Camera” or other appropriate text using the VIDEO CAMERA menu button 1 84. In certain embodiments, other components (e.g. motion sensors, temperature sensors, etc.) used by the video monitoring system to detect events may be tasked in similar fashion using the VIDEO CAMERA menu button 1 84, if desired.
  • other components e.g. motion sensors, temperature sensors, etc.
  • a DURATION OF CONDITON menu button 1 86 can be provided on the graphical user interface 1 34 to permit the user to select the period of time necessary to trigger an event. If, for example, the user desires to trigger only those events lasting for a duration of two minutes or more, the user may select "2 Min” or other appropriate text , using the DURATION OF CONDITON menu button 1 86. Other durations (e.g. "1 Min”, “30 Min”, “1 Hr.”, “4 Hrs.”, "1 Day”, etc.) may also be provided via the DURATION OF CONDITION menu button 186 to permit the user to select other time periods, as desired. In some embodiments, the graphical user interface 1 34 can be configured to accept a user-specified duration if the user desires to set a time period different than that contained in memory.
  • a DOMINANT COLOR menu button 1 88 and LICENSE NUMBER menu button 1 90 can be provided to permit the user to select the color and/or license number of any vehicles to be detected by the monitoring system. If, for example, the user desires to detect only those vehicles that are of a particular color (e.g. “red”, “green”, “blue”, “white”, “light”, “dark”, etc.) or of a particular license plate number or type (e.g. "AIK388", ⁇ Minnesota>, ⁇ Blacklisted>, etc.), the user may enter such information into the , graphical user interface 134 via the DOMINANT COLOR and LICENSE NUMBER menu buttons 1 88,190.
  • a particular color e.g. "red”, “green”, “blue”, “white”, “light”, “dark”, etc.
  • AIK388 ⁇ Minnesota>, ⁇ Blacklisted>, etc.
  • a set of selection boxes 192,194 can be selected on the display screen 136 to task the monitoring system to attempt to read license plate numbers and/or to count the number of event conditions satisfied.
  • box 194 can be selected if the user desires to count the number of vehicles, individuals, and/or other objects passing in and out of a region of interest such as a security checkpoint.
  • a WHEN CONDITION IS DETECTED section 196 of the graphical user interface 1 34 can be provided to permit the user to select those actions to be associated with a particular event.
  • An ALARM selection box 198 can be selected to generate an alarm when an event is detected by the monitoring system, or when an error or other user- specified condition has occurred.
  • an ALARM LEVEL menu button 200 can be provided to vary the level of the alarm (e.g. "High”, “Medium”, “Low”, etc).
  • an ALAiRM MODE menu button 202 can be provided to toggle the mode (e.g. "Audible Only”, “Audible/Visual”, “Visual Only”) of the alarm, when activated.
  • a START RECORDING selection box 204 can be selected to activate a recording when one or more events are detected by the monitoring system.
  • a PRE-RECORD TIME menu button 206 and RECORD TIME menu button 208 within section 1 96 can be provided to permit the user select the amount of time to be allocated to recording , before and after the detected event. In the illustrative view depicted in Figure 7, for example, "No Pre-recording" and "30 seconds" are shown selected using the PRE-RECORD TIME and RECORD TIME menu buttons 206,208, causing the monitoring system to record for a period of 30 seconds after the detection of an event. If the user desires to increase or decrease the time allocated for recording, or if the user desires to pre-record video prior to the occurrence of an event, the user may select menu buttons 206,208 and make the appropriate adjustments, as desired.
  • a RECORD FRAME RATE menu button 21 0 can be further provided to permit the user to adjust the frame rate of each video camera tasked by the monitoring system. If, for example, the user desires to record at a frame rate of 25 fps, the user may select "25 frames per second" or other appropriate text using the RECORD FRAME RATE menu button 21 0.
  • the graphical user interface 1 34 can be configured to display an alphanumeric message 21 2 informing the user of the maximum image bandwidth to be delivered by each video camera tasked by the monitoring system.
  • a DELETE AFTER menu button 214 can also be provided to permit the user to specify a time period ⁇ e.g. "1 day”, “2 days”, “1 week”, “1 month”, “indefinitely”, etc.) in which to store the recorded video clip prior to being deleted.
  • a SEND VIDEO TO STATION(S) selection button 21 6 can be selected to output video feeds to selected stations upon the detection of an event by the monitoring system. If, for example, the user desires to output video to a particular terminal or network station, the user may select a STATION NUMBER icon button 21 8 on the graphical user interface 1 34 and then enter the particular station or stations in which to feed the video using text box 220. Alternatively, if the user desires to output video to all stations tasked to monitor a particular area, the user may select the ALL STATIONS IN AREA icon button 222 on the graphical user interface 1 34 and select the appropriate area ⁇ e.g. "Region A", "Zone B", etc.) using menu button 224.

Abstract

Methods and systems for packaging video sequences based on user-specified events are described herein. An illustrative method of event-packaging a video sequence may include the steps of acquiring a video stream containing an event of interest, extracting various event information from the video stream and storing one or more event parameters within an event database, extracting a video clip from the video stream containing the event of interest, associating metadata representing the event parameters to the video clip, and . exporting the video clip containing the event of interest and associated metadata to an external agent. In certain embodiments, a video image- processing appliance manager and/or one or more appliance modules can be provided to automatically extract event information from the acquired video stream, and to manage the indexing and storage of event parameters within the event database. A graphical user interface may also be provided to permit the event information extracted from the video stream to be compared against a set of user-specified event parameters.

Description

INVENTION TITLE
EVENT PACKAGED VIDEO SEQUENCE
DESCRIPTION
Field
[Para 1 ] The present invention relates generally to the field of video image processing. More specifically, the present invention pertains to methods and systems for packaging video sequences based on user-specified events.
Background
[Para 2] Video surveillance systems are used in a variety of applications for monitoring objects within an environment. In security applications, for example, such systems are sometimes employed to track individuals or vehicles entering or leaving a building facility or security gate, or to monitor individuals within a store, office building, hospital, or other such setting where the health and/or safety of the occupants may be of concern. In the aviation industry, for example, such systems have been used to monitor the presence of individuals at key locations within an airport such as at a security gate or parking garage.
[Para 3] In certain applications, the video surveillance system may be tasked to record video image data for later use in determining the occurrence of a particular event. In forensic investigations, for example, it is common to task one or more video cameras within the system to indiscriminately record entire video clips that can later be analyzed to detect the occurrence of an event such as a robbery or theft. Such video images are typically stored as either analog video streams or as digital image data on a hard drive, optical drive, videocassette recorder (VCR), or other suitable storage means.
[Para 4] To permit prompt action to be taken upon the. occurrence of an event, it is sometimes desirable to bind portions of the video sequence into a "package" containing the event, and then transmit such package to an external agent for further analysis. In certain security applications, for example, it may be desirable for a security guard to record a video clip containing facial images of an individual, and then transmit such video clip to another agent (e.g. a facial image database) for further analysis. To accomplish this task, many systems require the operator to manually scan the entire video stream until the desired event is found. In some cases, the system may require the operator to determine the time and/or date of the event as well as the particular camera or cameras used to detect the event. The lack of an automated means for providing video clips directly to an operator can thus result in increased search times and slower response times.
Summary
[Para 5] The present invention pertains to methods and systems for packaging video sequences based on user-specified events. An illustrative method of event-packaging a video sequence may include the steps of acquiring a video stream containing an event of interest, extracting various event information from the video stream and storing one or more event parameters relating to the event within an event database, extracting a video clip from the video stream containing the event of interest, associating the event parameters to the video clip, and exporting the video clip containing the event of interest and associated event parameters to an external agent such as a terminal station or networked storage device.
[Para 6] In certain embodiments, a video image-processing appliance manager and/or one or more appliance modules can be provided to automatically extract event information from the acquired video stream, and/or to manage the indexing and storage of event parameters within the event database. A graphical user interface may be provided to permit the event information extracted from the video stream to be compared against a set of user-specified event parameters. In one illustrative embodiment, the user-specified event parameters may comprise a set of semantic parameters that can be used to detect and analyze various events occurring within one or more regions of interest. Other features such as a means for adjusting the pre-record and/or post record duration of the extracted video clip and , a means for providing annotation information along with the extracted video clip can be further provided on the graphical user interface, if desired.
Brief Description of the Drawings
[Para 7] Figure 1 is a block diagram showing an illustrative network-based video monitoring system for use in monitoring one or more regions of interest within an environment;
[Para 8] Figure 2 is a block diagram showing the architecture of a video image-processing appliance manager in accordance with an exemplary embodiment of the present invention;
[Para 9] Figure 3 is a flow chart showing an illustrative method of defining and analyzing events occurring within a region of interest of a video monitoring system in accordance with an exemplary embodiment of the present invention;
[Para 1 0] Figure 4 is a pictorial view showing an illustrative video frame including multiple regions of interest;
[Para 1 1 ] Figure 5 is another pictorial view of the illustrative video frame of Figure 4 showing the juxtaposition of a minimum boundary rectangle of an object within a region of interest;
[Para 1 2] Figure 6 is a flow chart showing an illustrative method of event-packaging a video sequence in accordance with an exemplary embodiment of the present invention; and
[Para 1 3] Figure 7 is a pictorial view showing an illustrative graphical user interface in accordance with an exemplary embodiment of the present invention.
Detailed Description
[Para 1 4] The following description should be read with reference to the drawings, in which like elements in different drawings are numbered in like fashion. The drawings, which are not necessarily to scale, depict selected embodiments and are not intended to limit the scope of the invention. Although examples of algorithms and processes are illustrated for the various elements, those skilled in the art will recognize that many of the examples provided have suitable , alternatives that may be utilized.
[Para 1 5] Figure 1 is a block diagram showing an illustrative network-based video monitoring system 10 employing a number of digital video system sensors (DVSS's). System 10, illustratively a video surveillance system for use in detecting and monitoring objects within an environment, may comprise a network 12 such as a wired or wireless local area network (LAN) or wide area network (WAN) that can be used to connect one or more DVSS's 14 to a network server 16 or other such host application. A DVSS can be understood generally as a subsystem that provides sequences of digital video images or other desired data to the monitoring system 10. Examples of DVSS's that can be employed by the system 10 may include, but are not limited to, one or more digital cameras, analog cameras equipped with a digitizer, frame grabbers, digital video recorders, files of stored video data in a database, microphones for audio signal capture, accelerometers for seismic data capture, pressure sensors, temperature sensors, level and/or flow sensors, proximity sensors, contact sensors {e.g. window or door switches), and/or multi-spectral sensors.
[Para 1 6] A number of browsers or terminal stations 1 8 equipped with a user interface 20 (e.g. a graphical user interface) can be connected to the network 12 and tasked to control the operation of the DVSS's 14 in a particular manner. In some embodiments, a terminal monitor 22 (e.g. a closed-circuit television monitor) can also be provided in addition to, or in lieu of, the browsers or terminal stations 1 8 to view images acquired from one or more of the DVSS's 14. The browsers or terminal stations 1 8 can be configured to interact with a host application software program 24 that can be used to task the DVSS's 14 in a particular manner. Based on user input via the user interface 20, for example, the host application software program 24 can be used to change operational parameters of the DVSS's 14 (e.g. PTZ settings, ROI settings, resolution, frame rate, etc.), and/or to task a video image processing appliance manager 28 to perform certain user- defined tasks such as motion detection, motion tracking, etc. If, for example, the user desires to increase the resolution of images received , by one or more of the DVSS's 14 in order to perform facial recognition of an individual within a region of interest, the host application software program 24 can be configured to accept a command causing one of the DVSS's 14 to zoom-in on the subject's face and increase the image capture rate. In the illustrative embodiment of Figure 1 , three DVSS's 14 are shown connected to the network 12, each of which can be tasked to acquire video and/or still images within a respective field of view (FOV) represented generally by dashed lines 26. It should be understood, however, that a greater or lesser number of DVSS's 14 may be employed, if desired. As shown in Figure 1 , the DVSS's 14 can be connected to the network 1 2 directly, through the host application software program 24, or both, as desired.
[Para 1 7] As can be further seen in Figure 1 , the host application software program 24 can be configured to call a video image- processing appliance manager 28 that can be used to facilitate processing of video images received from the DVSS's 14 using a number of plug-in appliance modules 30. The appliance manager 28 can be adapted to interface with the host application software program 24 as well as other components within the system 10. The video images and/or other information acquired by the DVSS's 14 can be fed through the appliance manager 28, which tasks the appropriate appliance modules 30 to determine the occurrence of any events configured within the system 10. If an event is triggered, the appliance manager 28 can be configured to output a response {e.g. via the user interface 20) indicating that an event has occurred within a region of interest. If desired, the video images triggering the event can be stored on a hard-drive, magnetic tape, or other storage medium, allowing the video images to be replayed and/or subsequently processed. In some embodiments, the video images acquired from the DVSS 14 can also be displayed in real-time on the terminal monitor 22.
[Para 1 8] Figure 2 is a block diagram showing the architecture of a video image-processing appliance manager 32 in accordance with an exemplary embodiment of the present invention. As shown in Figure 2, the appliance manager 32 may provide an interface to a number of host applications 34, which as is described above with respect to Figure 1 , can include various software algorithms and /or functionality for operating a number of DVSS components. The host applications 34 may include, for example, an existing host application software , program configured to interact with one or more video cameras (e.g. DVSS's) each adapted to acquire video images within a field of view (FOV). The appliance manager 32 can also be interfaced with other types of DVSS's including, but not limited to, frame grabbers, digital video recorders, files of stored video data in a database, microphones for audio signal capture, accelerometers for seismic data capture, pressure sensors, temperature sensors, level and/or flow sensors, proximity sensors, contact sensors (e.g. window or door switches), and/or multi-spectral sensors, as desired.
[Para 1 9] The host applications 34 can comprise separate components from the DVSS's (e.g. a stand-alone software package), or can be formed integral with one or more of the DVSS's and provided as a single component, if desired. In certain embodiments, for example, one or more of the DVSS's may comprise a physically separate video camera that is connected to an existing software- based host application adapted to run on the Internet, an intranet connection, and/or on an individual workstation equipped with a user interface 36. In such applications, each of the associated DVSS's can be connected to their associated host application 34 using an application program interface (API) or other suitable interface.
[Para 20] The host applications 34 may comprise one or more existing host application software programs contained on a network server, browser, terminal station, or other platform. The functionality provided by the existing host applications 34 will typically vary depending on their intended use. If, for example, the host applications 34 are adapted to interface with network-based control, access, and security systems/products, the host applications 34 may include an existing security software program that can be used to task a number of DVSS's to pan, tilt, and zoom to a tracked motion within a region of interest. Examples of other types of host applications 34 may include, but are not limited to, building management applications (e.g. HVAC control), life safety applications (e.g. fire protection, medical care, etc.), asset location applications, and energy management applications. [Para 21 ] The appliance manager 32 will typically comprise a separate module from the host applications 34, allowing the appliance manager 32 to be linked with the user's existing system without having to significantly modify or reprogram the existing software to accommodate new DVSS clients as they are added to the system. It should be understood, however, that the appliance manager 32 and host applications 34 could be incorporated together as a single, standalone module, if desired.
[Para 22] The appliance manager 32 can be configured to maintain one or more business objects, which can include various information about the region or regions of interest to be monitored, any events configured by a configurator 44, as well as various configuration information about the host applications 34 connected to the system. In certain embodiments, for example, the appliance manager 32 can be configured to maintain a camera structure list and an event structure list containing information about the type of DVSS's employed and the type of events configured within the system. Such lists can be stored within a memory unit or database {e.g. database 38) and recalled each time the appliance manager 32 receives an initialization call 40 or detection call 42 from one or more of the host applications 34.
[Para 23] The video monitoring system may include a configurator 44 that can be used in the direct manipulation and configuration of images or other data received by the host applications 34. A tuning request/call 46 received from one or more of the host applications 34 can be used by the configurator 44 to tune the appliance manager 32 and/or other desired system components to function in a particular manner. If, for example, a user desires to increase the frame capture rate of one of the DVSS's 14 (e.g. a video camera) field of view, the host application 34 can be configured to send a tuning request or call 46 to the appliance manager 32 that can be used by the configurator 44 to coordinate such a change. Once the change has been made, the appliance manager 32 can then be configured to pass the newly configured video stream through. In some embodiments, the configurator 44 can also be configured to send a response to the host application 34 and /or to the user indicating whether the tuning invocation succeeded or failed. [Para 24] The appliance manager 32 can be connected to a database 38 that can be configured to store information received from the DVSS's as well as parameters received by the configurator 44, as directed by i the appliance manager 32 and/or user via the user interface 36. In certain embodiments, the database 38 can be two separate databases residing at different servers, wherein one database can be linked to the host application 34 and the other database can be linked to the appliance manager 32. In other embodiments, the database 38 may comprise a single database, or multiple databases existing on a single server.
[Para 25] The appliance manager 32 can be configured to interact with a number of plug-in appliance modules each adapted to run various video image-processing algorithms or routines that can be used to perform certain user-defined image processing functions. In the illustrative embodiment of Figure 2, for example, the appliance manager 32 is shown operatively connected to a video motion detection (VMD) module 48, a video motion tracking (VMT) module 50, an object classification (OC) module 52, an event detection module 54, and an action dispatcher module 56. The various appliance modules 48,50,52,54,56 can be provided as either software appliances adapted to be run on a network or terminal server, or as separate hardware units that can be plugged into the appliance manager vis-a-vis a network bridge or other suitable connection. These modules 48,50,52,54,56 can be upgraded, when needed, to enhance performance and/or meet specific host application requirements.
[Para 26] The appliance manager 32 can be configured to permit the modular incorporation of future appliance modules 58, as desired. If, for example, the user desires to add a facial detection module or rapid eye detection module to the video monitoring system, the appliance manager 32 can be configured to accept a software call, network socket, physical port {e.g. a USB port, Firewire/IEEE 1 394 port, parallel/serial port, etc.), and/or wireless port that can be used to add the additional modules. Since an appliance manager 32 is provided to interface with the host applications 34, the addition of future plug-in modules does not require the user to re-code or re-formulate the existing host applications 34. In addition, the appliance manager 32 may provide the user with the ability to change the implementation and/or features of existing functionality without significantly affecting the operation of the DVSS's.
[Para 27] The appliance manager 32 can be configured to run a video image-processing algorithm or routine that continuously monitors the camera structure list and configured events list to determine whether an event has been detected by one or more of the DVSS's. When an event contained within the event list is detected, the appliance manager 32 can be configured to transmit a result 60 back to the host application 34 along with an action request 62 tasking one or more of the DVSS's to operate in a particular manner. If, for example, the video motion tracking module 50 detects that an object is moving in a particular direction within a region of interest, the appliance manager 32 can be configured to provide a result 60 to the appropriate host application 34 informing it that the object is being tracked along with an action request 62 tasking one or more associated DVSS's to track the object.
[Para 28] Turning now to Figure 3, an illustrative method 64 of defining and analyzing events occurring within a video monitoring system will now be described in the context of the illustrative appliance manager 32 of Figure 2. As shown in Figure 3, method 64 may begin at block 66, wherein a call is received by the appliance manager 32, causing the appliance manager 32 to initialize an algorithm or routine therein that can be used in defining, detecting, analyzing, indexing, and/or retrieving one or more events of interest. In the context of the illustrative video monitoring system 10 described above with respect to Figure 1 , for example, such appliance manager 32 may be called when a user and/or host application desires to task one or more DVSS's 14 to detect the occurrence of an event within a field of view.
[Para 29] Once invoked, the appliance manager 32 can be configured to task one or more of the appliance modules to extract information regarding one or more objects located within a region of interest, as indicated generally by reference to block 68. Information that can be extracted at this step may include, for example, information about an object's motion, trajectory, orientation, size, aspect ratio, color, lighting, temperature, and/or information about an object's type or classification (e.g. "human", "animal", "vehicle", "animate", "inanimate", etc.). Such extracted information can be obtained using one or more of the appliance modules 48,50,52,54 described above with respect to Figure 2. If, for example, the user wishes to define an event that detects and tracks vehicle motion within a parking garage, the appliance manager 32 tasks the video motion detection module 48 and video motion tracking module 50 to run separate algorithms or routines that can be used to perform such tasks. Information regarding the classification of the object, in turn, can be determined by invoking the object classification module 52 and running an algorithm or routine therein that determines whether an object is a vehicle. In some embodiments, the appliance manager 32 can be configured to task a video face detection module and/or video face tracking module to run separate algorithms or routines that can be used to gather information to perform facial recognition on individuals. The types of information extracted by the monitoring system will typically vary depending on the types of video cameras employed, the location of the video cameras, the particular appliance module(s) available to the system, as well as other factors.
[Para 30] Once the information is extracted within a region of interest, the appliance manager 32 tasks the event detection module 54 to combine this information in a meaningful manner to detect the occurrence of an event of interest, as indicated generally by reference to block 70. Each event of interest can be specified by a set of conditions and/or sub-conditions that identify the object and semantically describe its state and/or physical features relative to a particular region of interest. In certain embodiments, for example, each condition and/or sub-condition can be quantified by a set of semantic parameters that can be compared against a corresponding set of semantic parameters programmed within the event detection module 54. Examples of such semantic parameters may include, but are not limited to, information regarding the region of interest (e.g. "entranceway", "parking garage", "security zone", "conveyor belt", etc.), actions relating to the object (e.g. "start", "stop", "enter", "exit", etc.), and information regarding the direction of the object (e.g. "left", "right", "top" bottom", etc.). In some embodiments, semantic parameters relating to the type or classification of object detected (e.g. "human", "animal", "vehicle", "animate", "inanimate", "other", "any", etc.) may also be provided. In other embodiments, semantic parameters relating to the type of motion {e.g. walk, run, high speed, low speed, etc.) may also be provided. Semantic parameters relating to the physical appearance (e.g. color and size of object, single or group of people) or for a particular detected face (e.g. "happy, "neutral", "sad", etc.) may also be provided, if desired.
[Para 31 ] Once the semantic parameters are combined and analyzed to detect the occurrence of an event, such information can be compared against a set of user-specified events, as indicated generally by reference to block 72. Matching of the detected event with a corresponding user-specified event can be accomplished automatically by comparing the semantic parameters determined by the appliance modules with a set of semantic parameters specified by the user. If, for example, an event occurring within a region of interest is defined using a set of semantic parameters including the text "vehicle", "enters from right", and "within ROI 1 ", such event can be compared against a set of user-specified semantic parameters containing the text "vehicle enters from right in ROI 1 " to find a match. In certain embodiments, such user-specified semantic parameters can be stored as event parameters within an image database (e.g. database 38), and can be provided to the event detection module 54 through the appliance manager 32 via a graphical user interface or other suitable interface. Indexing and/or subsequent retrieval of such events can also be accomplished in a similar manner using such semantic parameters, if desired.
[Para 32] When a match exists, the appliance manager 32 invokes the action dispatcher module 56 to set up the appropriate action request or requests and perform an action response, as indicated generally by reference to block 74. In certain embodiments, for example, the action dispatcher module 48 can be configured to trigger an alarm or other such response to notify a user when a configured event has been detected. The appliance manager 32 can also be configured to record a video clip containing the detected event and/or send a video feed to a terminal station, browser, network server, or other such location for further analysis by a user and/or host application. In some embodiments, the video feed may contain one or more supporting event parameters. Other action responses such as that described below with respect to the illustrative graphical user interface 1 22 of Figure 6 can also be performed, if desired.
[Para 33] Figure 4 is a pictorial view showing an illustrative video frame 76 including multiple regions of interest. As can be seen in Figure 4, video frame 76 may define a pixel area 78 having a first region of interest 80 and a second region of interest 82. The first and second regions of interest 80,82 may represent, for example, regions of interest within a security zone or other such area in which the monitoring of individuals, vehicles, or other objects of interest is desired. In certain embodiments, for example, the first region of interest 80 may represent an entranceway of a building or structure whereas the second region of interest 82 may represent a parking garage or carport. While two regions of interest 80,82 are specifically depicted in the illustrative view of Figure 4, it should be understood that a greater or lesser number of regions of interest could be defined. Moreover, while the two regions of interest 80,82 are shown having a substantially rectangular shape in Figure 4, it should be understood that the regions of interest 80,82 may assume other polygonal shapes within the video frame 76. If, for example, a particular region of interest to be monitored comprises a parking lot, then a more complex polygonal shape approximating the boundaries of that region may be utilized.
[Para 34] The first and second regions of interest 80,82 may be shaped as any closed polygon in the image defined by the vertices of the polygon. In the illustrative view of Figure 4, for example, the first region of interest 80 is shown as a polygon {e.g. a rectangle) having a left side boundary 84, a top side boundary 86, a right side boundary 88, and a lower side boundary 90, which define a first pixel area 92 within the video frame 76. In similar fashion, the second region of interest 82 is shown as a polygon having a left side boundary 94, a top side boundary 96, a right side boundary 98, and a lower side boundary 100, which define a second pixel area 102 within the video frame 76. In the illustrative video frame 76 depicted in Figure 4, the pixel areas 92,102 defined by each respective region of interest 80,82 are non- overlapping such that none of the image pixels contained in the first pixel area 92 overlap with any of the image pixels contained in the second pixel area 1 02. It should be understood, however, that other , embodiments are envisioned wherein portions of the regions of interest 80,82 overlap one another. In some embodiments, other polygons having a non-rectangular shape may be defined as the ROI. In such case, the major and minor axes of the ROI are determined and a rectangular region defined by the major and minor axes can be used for evaluating the event conditions, as described below.
[Para 35] As can be further seen in Figure 4, the first and second regions of interest 80,82 may each define an ROI boundary area 104,106, which as indicated by shading, can be located about the outer periphery of each respective pixel area 92,1 02. Each ROI boundary area 104, 106 can be defined as a percentage or fraction of the total number of pixels contained within the pixel area 92, 1 02. In the illustrative video frame 76 of Figure 4, for example, the first and second regions of interest 80,82 may have respective ROI boundary areas 104,1 06 defined as percentage values xτ% and X2%, respectively, wherein each percentage value xi%, xz% is expressed as a percentage of the pixel area of the ROI boundary area 104,106 relative to their corresponding pixel area 92,102. The size, shape and locations of the ROI boundary areas 1 04,1 06 may vary depending on the type of region selected by the user, the characteristics of the objects located within that scene, as well as other factors. Thus, while rectangular-shaped ROI boundary areas 1 04, 1 06 are specifically depicted in the illustrative embodiment Figure 5, it should be understood that other types of ROI boundary areas 1 04,1 06 can be utilized. If, for example, the ROI to be monitored is a building or structure having an irregular shape, then the ROI boundary area 104,1 06 used to define that ROI may have also have an irregular shape to better approximate the ROI. Such irregular shape may be defined, for example, by selecting a greater number of reference points on the video frame 76, using curved lines to define the boundary area, and/or by some other suitable technique. The number of ROI's located within the video frame 76 may vary depending on the number of ROI's to be monitored.
[Para 36] Figure 5 is another pictorial view of the illustrative video frame 76 of Figure 4 showing the juxtaposition of a minimum boundary rectangle 1 08 of an object within the first region of interest 80. As shown in Figure 5, the minimum boundary rectangle 108 may in 1 clude an outer boundary 1 10 defining an area 1 12 containing an object OBJ such as an individual or vehicle. In the illustrative view depicted in Figure 5, the minimum boundary rectangle 108 is shown covering a lower-left portion of the ROI boundary area 1 04, intersecting the left side boundary 84 and bottom side boundary 90 of the first region of interest 80.
[Para 37] Using the minimum boundary rectangle 108 to represent the general shape of the object OBJ, a number of events relating to the object's motion relative to the first region of interest 80 can be determined by comparing the coordinates of the ROI boundary area 104 with a corresponding set of coordinates of the minimum boundary rectangle 108. In the illustrative embodiment of Figure 5, for example, an upper-left location 1 14 and lower-right location 1 16 of the ROI boundary area 104 can be assigned a set of coordinates of (RXI .RYI) and (Rχ2,Rγ2), respectively. In similar fashion, an upper-left location 1 18 and lower-right location 120 of the minimum boundary rectangle 108 can be assigned a corresponding set of coordinates (OXI ,OYI) and (Oχ2, Oγ2), respectively. A similar coordinate scheme can be utilized for other objects and regions of interest situated within the video frame 76, as desired.
[Para 38] While the upper-left and lower-right locations 1 14,1 16,1 1 8,120 are selected in Figure 5 as reference points to determine the position of the minimum boundary rectangle 108 relative to the first region of interest 80, it should be understood that other reference locations could also be utilized, if desired. In one alternative embodiment, for example, the lower-left and upper-right locations of the first ROI boundary area 104 and minimum boundary rectangle 108 could be used as reference points. Moreover, while the use of a minimum boundary rectangle 108 is specifically shown in the illustrative view of Figure 5, those of skill will recognize that other boundary shapes could be used to approximate the size and/or contour of the object. Moreover, annotation information such as object labels, motion trails, motion direction arrows, highlighting of detected objects, etc. can be further provided to aid the user in identifying and tracking objects within the video frame 76, if desired. [Para 39] In certain embodiments, and as further shown in Table 1 . reproduced below, the sets of upper-left and lower-right coordinates (RXI ,RYI),(RX2,RY2) and (Oχi,Oγτ),(Oχ2,Oγ2) can be used to define various event conditions (i.e. semantic parameters) relating to the object's location and direction relative to the ROI boundary area 104 by comparing the corresponding coordinate values. If, for example, the left side of the minimum boundary rectangle 108 is located within the left side boundary 84 of the ROI boundary area 104, then the equation 0 < Oχ2 - Rxi < xi%(Rχ2 - Rxi) will be satisfied, thus indicating that a "Left" event has occurred. Conversely, if the right side of the minimum boundary rectangle 108 is located within the right side boundary 88 of the ROI boundary area 104, then the equation 0 < Rχ2 - Oxi < xi%(Rχ2 - Rxi) will be satisfied, indicating that a "Right" event condition has occurred. In similar fashion, if the bottom side of the minimum boundary rectangle 108 is located within the top side boundary 86 of the ROI boundary area 104, then the equation 0 < Oγ2 - RYI < XI%(RY2 - RYI) will be satisfied, indicating that a "Top" event condition has occurred. Likewise, if the top side of the minimum boundary rectangle 108 is located within the bottom side boundary 90 of the ROI boundary area 104, then the equation 0 < Rγ2 - OYI < xi%(Rγ2 - RYO will be satisfied, indicating that a "Bottom" event condition has occurred. If desired, motion in directions other than "left", "right", "top", and "bottom" may also be determined. In addition, other event conditions {e.g. "Start Track", "Enters Region", "Stop Track", "Exits Region", etc.) may also be defined by determining the state of the object track and comparing the coordinate values of the minimum boundary rectangle 108 relative to the ROI boundary area 104, and/or by comparing the location of the minimum boundary rectangle 108 relative to other locations within the video frame 76.
[Para 40] Table 1
Figure imgf000018_0001
[Para 41 ] In certain cases, multiple event conditions may be satisfied for the same object. If, as shown for example in Figure 5, the minimum boundary rectangle 108 of the object OBJ enters the ROI boundary area 104 simultaneously from both the left and bottom side boundaries 84,90, a "Left" and "Bottom" event condition will be detected. In such case, the monitoring system can be configured to combine the two semantic parameters into a string of semantic parameters using a suitable Boolean operand such as "and" or "+". Once the event conditions {i.e. semantic parameters) are combined, the monitoring system can then be configured to match these event conditions with a corresponding set of event conditions specified by the user.
[Para 42] Turning now to Figure 6, an illustrative method 122 of event-packaging a video sequence in accordance with an exemplary embodiment of the present invention will now be described. As shown in Figure 6, method 1 22 may begin at block 124, wherein a DVSS or network of DVSS's is/are tasked to acquire a video stream containing an event of interest (block 1 26). In certain embodiments, for example, the acquired video stream may comprise a digital video stream obtained from an analog camera having a digital IP-addressed streamer, or, in the alternative, a digital camera having a straight digital IP-addressed streamer.
[Para 43] Once a video stream containing an event of interest is acquired, a series of algorithms and/or management functions can be executed to extract various event information from the video stream that can later be used to package a portion of the video stream into a video clip containing the event of interest. As indicated generally by block 128, for example, method 122 may include the step of extracting various event information in a manner similar to that described above with respect to Figure 3, wherein information regarding an object located within a region of interest is compared and matched against, a set of event parameters specified by a user. Examples of such event parameters may include metadata such as the date and time of the event, a header identifying the configured event, the camera or cameras tasked to acquire the video stream containing the event, the location of the event, the type of event detected, the lighting conditions detected by the camera or cameras, the classification of the object triggering the event (e.g. individual, vehicle, animal, animate, inanimate, etc.), the direction of object motion, the color of the object (e.g. red, blue, dark, light, etc.), and/or facial information. In certain embodiments, such event information can be extracted using the video image processing appliance manager 32 and/or one or more of the appliance modules 40,42,44,46 described herein. Once extracted, the event information can then be indexed and stored as event parameters within an event database (e.g. database 38) for subsequent use.
[Para 44] Once event information has been obtained from the video stream, a video clip containing the desired event may be extracted from the video stream and then optionally stored within a video database, as indicated generally by reference to block 130. In certain embodiments, a user-selectable duration of the video stream can be recorded as a video clip within the video database, and can be associated with the one or more of the event parameters {e.g. event description, date, time, camera number, etc.) representing the event. Other characteristics describing the object or objects triggering the event as well as each , object's interaction with the region of interest may also contained within the event database, if desired. In some embodiments, a user interface {e.g. a GUI) can be employed to permit the user to adjust the pre-record and/or post-record duration of the video clip, the space or memory allocated to the video clip, the format of the video clip {e.g. MPEC, ivlJPEG, etc.), the aspect ratio of the video frames within the video clip, as well as other settings. Annotation information such as object labels, motion trails, motion direction arrows, highlighting of detected objects, etc. can also further provided along with the event- packaged video clip, if desired.
[Para 45] Once an event-packaged video clip is extracted and stored within the database, the video clip and associated metadata can then be exported to an external agent, as indicated generally by reference to block 132. The event-packaged video clip may be sent to the external agent either directly or when prompted by the user via the user interface. The event parameters that supported the detection can be associated and sent with the video clip. The databases containing the event information and associated video clips may then be searched using semantic parameters similar to that described above with respect to Figures 4-5. If, for example, the user desires to search the event database for those events containing an individual entering a particular region of interest, the user may input a command into a user interface causing any video clips associated with that particular event to be retrieved from the video database. The video clip containing the event as well as the event parameters used to define the event can then be exported to an external agent for further analysis, if desired. In some embodiments, for example, such event-packaged video clip can be exported to the external agent via a network connection, electronic mail, PDA, phone line, or other suitable transmission means. ,The event-packaged video clip can also be sent to other locations within the monitoring system such as a terminal station, network storage device or other system appliance. [Para 46] By automatically providing an event-packaged video clip • containing the event of interest, the search time usually required to locate an event within a video sequence may be reduced significantly. Moreover, such event-packaged video clips may reduce the time required for an operator to respond to an event, allowing quicker response times to be achieved in some applications.
[Para 47] Referring now to Figure 7, an illustrative graphical user interface 134 in accordance with an exemplary embodiment of the present invention will now be described in conjunction with the illustrative appliance manager 32 of Figure 2. Graphical user interface 1 34 may include a display screen 1 36 configured to display various information related to events configured within the appliance manager 32 as well as the various appliance modules {e.g. the event detection module 54). In the illustrative embodiment of Figure 7, for example, the graphical user interface 1 34 may include a REGIONS OF INTEREST section 1 38 that permits the user to identify various regions of interest within the field of view (FOV) of a DVSS in the monitoring system. The REGION OF INTEREST section 1 38 may include a DVS ID menu button 140 that can be used to specify where the region of interest is location, and a REGION NAME menu button 142 that can be used to name a particular region of interest (e.g. "Carpark A") in which to detect events. The region name uniquely associates with an area in the FOV of the DVSS whose identity is specified by the DVS ID menu button 140. This area can be displayed and annotated in the display screen 1 36 of the graphical user interface 134. When selected via a mouse, keyboard, keypad, touch screen, or other suitable selection means, the graphical user interface 134 can be configured to display a list of those regions of interest available for detection within the FOV of the DVS ID 140 by the monitoring system.
[Para 48] Other information (e.g. the coordinates of the corners of the region of interest) can also be displayed and/or be configured via the graphical user interface 1 34, if desired. In the illustrative embodiment of Figure 7, for example, the graphical user interface 1 34 may include a PARAMETER SETTING section 144 that permits the user to specify the parameter values of the various appliance modules. A SCENARIO menu button 146 within the PARAMETER SETTING section 144 can be provided to permit the user to adjust the sensitivity at which events are detected. If, for example, the selected region of interest is located indoors where lighting conditions are typically bright, the user may select "Indoors (High)" or other appropriate text using the SCENARIO menu button 146. Other suitable selections such as "Outdoors (Low)" or "Mixed Conditions (Intermediate)" may also be implemented using the SCENARIO menu button 146, if desired. In some embodiments, a SUB-SAMPLING menu button 148 and MINIMUM OBJECT SIZE text box 1 50 may also be provided to1 permit the user to adjust the sub-sampling rate of images captured and/or to set the minimum size of objects to be detected. In some embodiments, a CLASSIFIER menu button 1 52 may be provided to permit the user to specify which classifier and its associated parametric values to be used in classifying the object. Other parameters in addition to those specifically depicted in Figure 7 may also be provided to facilitate operation and/or configuration of the monitoring system, if desired.
[Para 49] A VIDEO MONITOR section 1 54 of the graphical user interface 1 34 can be configured to display video images and/or still images acquired by one or more of the DVSS's within the monitoring system. In some embodiments, the DVS ID, which captures the video image, may also be superimposed on the VIDEO MONITOR section 1 54. In the illustrative view depicted in Figure 7, for example, the graphical user interface 1 34 is shown displaying a first region of interest 1 56 in an upper-left portion of the VIDEO MONITOR section 1 54, and a second region of interest 1 58 in a lower-right portion of the VIDEO MONITOR ' section 1 54. Each region of interest 1 56, 1 58 can be demarcated on the display screen 1 36 using a dashed boundary box 1 60,1 62 or other suitable visual indicator. In some embodiments, the corresponding region name (e.g. "Carpark A", "Security Zone 1 ", etc.) 1 64,1 66 may also be superimposed on each region of interest 1 56, 1 58 displayed on the display screen 1 36, if desired.
[Para 50] An EVENTS Tu DETECT WITHIN REGION OF INTEREST section 168 of the graphical user interface 1 34 can be provided to permit the user to select those event conditions to be detected within the particular region of interest selected via the REGION NAME menu button 142. A list of event conditions currently programmed within the monitoring system can be displayed within a text box 1 70 located within section 168. An ADD icon button 1 72 located to the right of the text box 1 70 can be provided to permit the user to add additional event conditions to be detected by the monitoring system. A DELETE icon button 1 74 also located to the right of the text box 1 70, in turn, can be provided to permit the user to delete one or more event conditions currently programmed within the monitoring system. If desired, the current event condition selected within the text box 170 (i.e. "Red Vehicle Enter Carpark A From Left") can be highlighted by blinking text, italics, inverted text, or other suitable visual means. When the ADD icon button 1 72 is selected, the user can be prompted to enter the desired event conditions.
[Para 51 ] An EVENT CONDITION DETAILS section 1 76 of the graphical user interface 1 34 can be configured to permit user entry as well as display those details associated with each event condition configured within the monitoring system. In the illustrative embodiment of Figure 7, for example, an OBJECT TYPE menu button 1 78 and a CONDITION TO DETECT menu button 1 80 can be provided to permit the user to select the type of object and event condition(s) to be detected. Jf, for example, the user desires to monitor vehicles entering a carpark, the user may select "vehicle" using the OBJECT TYPE menu button 1 78 and "enters region" using the CONDITON TO DETECT menu button 1 80. Examples of other object types or classifications that can be selected using the OBJECT TYPE menu button 1 78 may include "individual", "animal", "animate object", "inanimate object", "any object", "unknown object", etc. Examples of other types of event conditions that can be selected using the CONDITION TO DETECT menu button 180 may include "start track", "stop track", "exits region", "movement detected", etc.
[Para 52] A DIRECTION OF MOTION menu button 182 can be provided to permit the user to select the direction or directions in which an object triggers an event. If, for example, the user desires to detect only those events in which an object travels through a region of interest from the left, the user may select "From Left" using the DIRECTION OF MOTION menu button 1 82, causing the monitoring system to perform an action response only when the object is traveling from this direction. Alternatively, if the user desires to detect only those events in which an object is traveling from the right, from above, or from below, the user may select "From Right", "From Above", and/or "From Below" or other appropriate text using the DIRECTION OF MOTION menu button 1 82, causing the monitoring system to perform an action response only when the object is traveling from these directions. If desired, the graphical user interface 1 34 may permit the user to select multiple directions (e.g. "From Left" and "From Below)" in which to detect and/or track object motion.
[Para 53] In certain embodiments, the graphical user interface
1 34 can be configured to permit the user to specify more complex motion to be detected and tracked. If, for example, the user desires to detect when an object traverses a serpentine-like path within a particular region of interest, the user may select "Serpentine Path" or other appropriate text using the DIRECTION OF MOTION menu button 1 82, causing the monitoring system to perform an action response when the object travels in such pattern. Other user-specified directions and/or paths can be further implemented in similar fashion via the DIRECTION OF MOTION menu button 1 82, if desired.
[Para 54] A VIDEO CAMERA menu button 1 84 can be provided to permit the user to select those video cameras to be tasked in detecting events. If, for example, the user desires to separately task one or more video cameras (e.g. "Camera 1 ", "Camera 2", "Cameras 1 & 2", etc.) to detect object motion within a particular region of interest, the user may select the appropriate camera name using the VIDEO CAMERA menu button 1 84. Alternatively, and as shown in Figure 7, if the user desires to task all of the video cameras to detect a particular event, the user may select "Any Camera" or other appropriate text using the VIDEO CAMERA menu button 1 84. In certain embodiments, other components (e.g. motion sensors, temperature sensors, etc.) used by the video monitoring system to detect events may be tasked in similar fashion using the VIDEO CAMERA menu button 1 84, if desired.
[Para 55] A DURATION OF CONDITON menu button 1 86 can be provided on the graphical user interface 1 34 to permit the user to select the period of time necessary to trigger an event. If, for example, the user desires to trigger only those events lasting for a duration of two minutes or more, the user may select "2 Min" or other appropriate text , using the DURATION OF CONDITON menu button 1 86. Other durations (e.g. "1 Min", "30 Min", "1 Hr.", "4 Hrs.", "1 Day", etc.) may also be provided via the DURATION OF CONDITION menu button 186 to permit the user to select other time periods, as desired. In some embodiments, the graphical user interface 1 34 can be configured to accept a user-specified duration if the user desires to set a time period different than that contained in memory.
[Para 56] In certain embodiments, a DOMINANT COLOR menu button 1 88 and LICENSE NUMBER menu button 1 90 can be provided to permit the user to select the color and/or license number of any vehicles to be detected by the monitoring system. If, for example, the user desires to detect only those vehicles that are of a particular color (e.g. "red", "green", "blue", "white", "light", "dark", etc.) or of a particular license plate number or type (e.g. "AIK388", <Minnesota>, <Blacklisted>, etc.), the user may enter such information into the , graphical user interface 134 via the DOMINANT COLOR and LICENSE NUMBER menu buttons 1 88,190. A set of selection boxes 192,194 can be selected on the display screen 136 to task the monitoring system to attempt to read license plate numbers and/or to count the number of event conditions satisfied. In certain embodiments, for example, box 194 can be selected if the user desires to count the number of vehicles, individuals, and/or other objects passing in and out of a region of interest such as a security checkpoint.
[Para 57] A WHEN CONDITION IS DETECTED section 196 of the graphical user interface 1 34 can be provided to permit the user to select those actions to be associated with a particular event. An ALARM selection box 198 can be selected to generate an alarm when an event is detected by the monitoring system, or when an error or other user- specified condition has occurred. If desired, an ALARM LEVEL menu button 200 can be provided to vary the level of the alarm (e.g. "High", "Medium", "Low", etc). In certain embodiments, an ALAiRM MODE menu button 202 can be provided to toggle the mode (e.g. "Audible Only", "Audible/Visual", "Visual Only") of the alarm, when activated.
[Para 58] A START RECORDING selection box 204 can be selected to activate a recording when one or more events are detected by the monitoring system. A PRE-RECORD TIME menu button 206 and RECORD TIME menu button 208 within section 1 96 can be provided to permit the user select the amount of time to be allocated to recording , before and after the detected event. In the illustrative view depicted in Figure 7, for example, "No Pre-recording" and "30 seconds" are shown selected using the PRE-RECORD TIME and RECORD TIME menu buttons 206,208, causing the monitoring system to record for a period of 30 seconds after the detection of an event. If the user desires to increase or decrease the time allocated for recording, or if the user desires to pre-record video prior to the occurrence of an event, the user may select menu buttons 206,208 and make the appropriate adjustments, as desired.
[Para 59] A RECORD FRAME RATE menu button 21 0 can be further provided to permit the user to adjust the frame rate of each video camera tasked by the monitoring system. If, for example, the user desires to record at a frame rate of 25 fps, the user may select "25 frames per second" or other appropriate text using the RECORD FRAME RATE menu button 21 0. In certain embodiments, the graphical user interface 1 34 can be configured to display an alphanumeric message 21 2 informing the user of the maximum image bandwidth to be delivered by each video camera tasked by the monitoring system. A DELETE AFTER menu button 214 can also be provided to permit the user to specify a time period {e.g. "1 day", "2 days", "1 week", "1 month", "indefinitely", etc.) in which to store the recorded video clip prior to being deleted.
[Para 60] A SEND VIDEO TO STATION(S) selection button 21 6 can be selected to output video feeds to selected stations upon the detection of an event by the monitoring system. If, for example, the user desires to output video to a particular terminal or network station, the user may select a STATION NUMBER icon button 21 8 on the graphical user interface 1 34 and then enter the particular station or stations in which to feed the video using text box 220. Alternatively, if the user desires to output video to all stations tasked to monitor a particular area, the user may select the ALL STATIONS IN AREA icon button 222 on the graphical user interface 1 34 and select the appropriate area {e.g. "Region A", "Zone B", etc.) using menu button 224.
[Para 61 ] Having thus described the several embodiments of the present invention, those of skill in the art will readily appreciate that other embodiments may be made and used which fall within the scope of the claims attached hereto. Numerous advantages of the invention covered by this document have been set forth in the foregoing description. It will be understood that this disclosure is, in many respects, only illustrative. Changes can be made with respect to various elements described herein without exceeding the scope of the invention.

Claims

What is claimed is:
[Clai m 1 ] I . A method of event-packaging a video sequence, comprising the steps of: acquiring a video stream containing an event of interest; extracting event information from the video stream and storing one or more event parameters within an event database; detecting a user-defined event from the video stream; extracting a video clip from the video stream containing the event of interest; associating metadata representing the event parameters with the extracted video clip; and exporting the video clip containing the event of interest and the associated metadata to an external agent.
[Clai m 2] 2. The method of claim 1 , further comprising the step of providing a video image-processing appliance manager in communication with one or more appliance modules.
[Clai m 3] 3. The method of claim 2, wherein said one or more appliance modules includes a video detection module, a video motion tracking module, an object classification module, and an event detection module.
[Clai m 4] 4. The method of claim 2, wherein said one or more appliance modules includes a face detection module, a face tracking module, a face recognition module, and an event detection module.
[Claim 5] 5. The method of claim 2, wherein said step of extracting event information from the video stream is accomplished automatically using at least one of said one or more appliance modules.
[Clai m 6] 6. The method of claim 2, wherein said step of extracting event information from the video stream is accomplished automatically using the appliance manager.
[Clai m 7] 7. The method of claim 1 , further including the step of storing the video clip within a video database after said step of extracting the video clip from the video stream containing the event of interest.
[Clai m 8] 8. The method of claim 7, wherein said step of storing the video clip includes the step of associating the metadata to the stored video clip.
[Clai m 9] 9. The method of claim 1 , further including the step of providing a graphical user interface.
[Cl ai m 1 0] 10. The method of claim 9, wherein the graphical user interface includes a means for adjusting the pre-record and/or post record duration of the extracted video clip.
[Cl ai m 1 1 ] l l . The method of claim 9, wherein the graphical user interface includes a means for providing annotation information along with the extracted video clip.
[Clai m 1 2] 1 2. The method of claim 9, further including the step of comparing the extracted event information against a set of user-specified event parameters.
[Clai m 1 3] 1 3. The method of claim 1 2, wherein said one or more user-specified event parameters are selected from the group of event parameters consisting of a date parameter, a time parameter, a camera identification parameter, a location of event parameter, a type of event parameter, a lighting parameter, an object type parameter, a direction of object motion parameter, a color of object parameter, and a license number parameter.
[Cl ai m 1 4] 14. The method of claim 1 2, wherein said one or more user-specified event parameters are semantic parameters.
[Cl ai m 1 5] 1 5. The method of claim 1 2, further including the step of searching the event database using the one or more user- specified event parameters.
[Cl ai m 1 6] 16. The method of claim 1 , wherein said step of exporting the video clip containing the event of interest to an external agent includes the step of sending a video clip and associated metadata to a terminal station.
[C l ai m 1 7] 1 7. The method of claim 1 , wherein said step of exporting the video clip containing the event of interest to an external agent includes the step of sending a video clip and associated metadata to a networked storage device.
[C lai m 1 8] 1 8. A method of event-packaging a video sequence, comprising the steps of: acquiring a video stream containing an event of interest; extracting event information from the video stream and storing one or more event parameters within an event database; detecting a user-defined event from the video stream; extracting a video clip from the video stream containing the event of interest and storing the extracted video clip within a video database; and exporting the video clip containing the event of interest to an external agent.
[Clai m 1 9] 1 9. A method of event-packaging a video sequence, comprising the steps of: acquiring a video stream containing an event of interest; extracting event information from the video stream and storing one or more event parameters within an event database; extracting a video clip from the video stream containing the event of interest and storing the extracted video clip within a video database; associating metadata representing the event parameters with the extracted video clip; and searching the event database using one or more user-specified event parameters and retrieving an associated video clip from the video database.
[Claim 20] 20. A method of event-packaging a video sequence, comprising the steps of: providing a graphical user interface; acquiring a video stream containing an event of interest; extracting event information from the video stream and storing one or more event parameters within an event database; comparing the extracted event information against a set of user- specified event parameters configured using the graphical user interface; extracting a video clip from the video stream containing the event of interest; and exporting the video clip containing the event of interest to an external agent.
[Clai m 2 1 ] 21 . The method of claim 20, wherein the graphical user interface includes a means for adjusting the pre-record and/or post record duration of the extracted video clip.
[Clai m 22] 22. The method of claim 20, wherein the graphical user interface includes a means for providing annotation information along with the extracted video clip.
[Clai m 23] 23. The method of claim 20, wherein said one or more user-specified event parameters are selected from the group of event parameters consisting of a date parameter, a time parameter, a i camera identification parameter, a location of event parameter, a type of event parameter, a lighting parameter, an object type parameter, a direction of object motion parameter, a color of object parameter, and a license number parameter.
[Clai m 24] 24. The method of claim 20, wherein said one or more user-specified event parameters are semantic parameters.
[Clai m 25] 25. The method of claim 20, further including the step of searching the event database using the one or more user- specified event parameters.
[Clai m 26] 26. The method of claim 20, wherein said step of exporting the video clip containing the event of interest to an external agent includes the step of sending a video clip and associated event parameters to a terminal station.
[Clai m 27] 27. The method of claim 20, wherein said step of exporting the video clip containing the event of interest to an external , agent includes the step of sending a video clip and associated event parameters to a networked storage device.
[Clai m 28] 28. A method of event-packaging a video sequence, comprising the steps of: providing a video image-processing appliance manager in communication with one or more appliance modules; providing a graphical user interface in communication with the appliance manager; acquiring a video stream containing an event of interest; automatically extracting event information from the video stream using, the appliance manager and/or one or more of the appliance modules; storing one or more event parameters within an event database; extracting a video clip from the video stream containing the event of interest; and exporting the video clip containing the event of interest to an external agent.
[Cl ai m 29] 29. The method of claim 28, wherein said one or more appliance modules includes a video detection module, a video motion tracking module, an object classification module, and an event detection module.
[Clai m 30] 30. The method of claim 28, wherein said one or more appliance modules includes a face detection module, a face tracking module, a face recognition module, and an event detection module.
[Cl ai m 3 1 ] 31 . The method of claim 28, further including the step of storing the video clip and associated event parameters within a video database after said step of extracting the video clip from the video stream containing the event of interest.
[Clai m 32] 32. The method of claim 28, wherein the graphical user interface includes a means for adjusting the pre-record and/or post record duration of the extracted video clip.
[Clai m 33] 33. The method of claim 28, wherein the graphical user interface includes a means for providing annotation information along with the extracted video clip.
[C lai m 34] 34. The method of claim 28, further including the step of comparing the extracted event information against a set of user-specified event parameters.
[C lai m 35] 35. The method of claim 34, wherein said one or more user-specified event parameters are selected from the group of event parameters consisting of a date parameter, a time parameter, a camera identification parameter, a location of event parameter, a type of event parameter, a lighting parameter, an object type parameter, a direction of object motion parameter, a color of object parameter, and a license number parameter.
[Clai m 36] 36. The method of claim 34, wherein said one or more user-specified event parameters are semantic parameters.
[C l ai m. 37] 37. The method of claim 34, further including the step of searching the event database using the one or more user- specified event parameters.
[Cl ai m 38] 38. The method of claim 28, wherein said step of exporting the video clip containing the event of interest to an external agent includes the step of sending a video clip to a terminal station.
[Clai m 39] 39. The method of claim 28, wherein said step of exporting the video clip containing the event of interest to an external agent includes the step of sending a video clip to a networked storage device.
[Clai m 40] 40. A method of event-packaging a video sequence, comprising the steps of: providing a video image-processing appliance manager in communication with one or more appliance modules; providing a graphical user interface in communication with the appliance manager; acquiring a video stream containing an event of interest; automatically extracting event information from the video stream using the appliance manager and/or one or more of the appliance modules; comparing the extracted event information against a set of user- specified event parameters configured using the graphical user interface; storing one or more event parameters within an event database; extracting a video clip from the video stream containing the event of interest; and exporting the video clip containing the event of interest to an external agent.
[Clai m 41 ] 41 . A video monitoring system, comprising: a video image-processing appliance manager in communication with one or more appliance modules, said appliance manager adapted to run an algorithm or routine for extracting an event-packaged video clip from a video stream containing an event of interest; and means for configuring one or more user-specified event parameters within the appliance manager and/or one or more of the appliance modules.
[Clai m 42] 42. The video monitoring system of claim 41 , wherein said means for configuring one or more user-specified event parameters within the appliance manager and/or one or more of the appliance modules includes a graphical user interface.
[Cl ai m 43] 43. The video monitoring system of claim 42, wherein the graphical user interface includes a means for adjusting the pre-record and/or post record duration of the event-packaged video clip.
[Clai m 44] 44. The video monitoring system of claim 42, wherein the graphical user interface includes a means for providing annotation information along with the event-packaged video clip.
[Clai m 45 ] 45. The video monitoring system of claim 41 , wherein said one or more user-specified event parameters are selected from the group of event parameters consisting of a date parameter, a time parameter, a camera identification parameter, a location of event parameter, a type of event parameter, a lighting parameter, an object type parameter, a direction of object motion parameter, a color of object parameter, and a license number parameter.
[Cl ai m 46] 46. The video monitoring system of claim 41 , wherein said one or more user-specified event parameters are semantic parameters.
PCT/US2006/005943 2005-03-31 2006-02-21 Event-packaged video sequence WO2006107430A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP06735551A EP1864496A1 (en) 2005-03-31 2006-02-21 Event-packaged video sequence

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US10/907,411 2005-03-31
US10/907,411 US7760908B2 (en) 2005-03-31 2005-03-31 Event packaged video sequence

Publications (1)

Publication Number Publication Date
WO2006107430A1 true WO2006107430A1 (en) 2006-10-12

Family

ID=36371000

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2006/005943 WO2006107430A1 (en) 2005-03-31 2006-02-21 Event-packaged video sequence

Country Status (3)

Country Link
US (1) US7760908B2 (en)
EP (1) EP1864496A1 (en)
WO (1) WO2006107430A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009078946A1 (en) * 2007-12-18 2009-06-25 Eastman Kodak Company Video customized to include person-of-interest
WO2011059659A1 (en) * 2009-11-12 2011-05-19 Siemens Industry, Inc. System and method for annotating video with geospatially referenced data
EP2442284A1 (en) * 2010-10-14 2012-04-18 Honeywell International Inc. Graphical bookmarking of video data with user inputs in video surveillance
EP2557784A3 (en) * 2011-08-12 2014-08-20 Honeywell International Inc. System and method of creating an intelligent video clip for improved investigations in video surveillance
US8837906B2 (en) 2012-12-14 2014-09-16 Motorola Solutions, Inc. Computer assisted dispatch incident report video search and tagging systems and methods
EP2580738A4 (en) * 2010-08-10 2018-01-03 LG Electronics Inc. Region of interest based video synopsis

Families Citing this family (155)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10271017B2 (en) * 2012-09-13 2019-04-23 General Electric Company System and method for generating an activity summary of a person
US8675059B2 (en) * 2010-07-29 2014-03-18 Careview Communications, Inc. System and method for using a video monitoring system to prevent and manage decubitus ulcers in patients
US9311540B2 (en) 2003-12-12 2016-04-12 Careview Communications, Inc. System and method for predicting patient falls
US7760908B2 (en) 2005-03-31 2010-07-20 Honeywell International Inc. Event packaged video sequence
US7801328B2 (en) * 2005-03-31 2010-09-21 Honeywell International Inc. Methods for defining, detecting, analyzing, indexing and retrieving events using video image processing
EP1913557B1 (en) * 2005-06-23 2016-06-01 Israel Aerospace Industries Ltd. A system and method for tracking moving objects
JP4538008B2 (en) 2005-08-12 2010-09-08 株式会社ソニー・コンピュータエンタテインメント Face image display device, face image display method, and face image display program
US20070071404A1 (en) * 2005-09-29 2007-03-29 Honeywell International Inc. Controlled video event presentation
GB2432064B (en) * 2005-10-31 2011-01-19 Hewlett Packard Development Co Method of triggering a detector to detect a moving feature within a video stream
US7956735B2 (en) 2006-05-15 2011-06-07 Cernium Corporation Automated, remotely-verified alarm system with intrusion and video surveillance and digital video recording
WO2007136745A2 (en) 2006-05-19 2007-11-29 University Of Hawaii Motion tracking system for real time adaptive imaging and spectroscopy
KR100815565B1 (en) * 2006-08-23 2008-03-20 삼성전기주식회사 Movement sensing system and method thereof
US8380558B1 (en) * 2006-12-21 2013-02-19 Videomining Corporation Method and system for analyzing shopping behavior in a store by associating RFID data with video-based behavior and segmentation data
US9075807B2 (en) * 2006-12-29 2015-07-07 Echostar Technologies L.L.C. System and method for creating, receiving and using interactive information
WO2008092255A1 (en) * 2007-01-30 2008-08-07 March Networks Corporation Method and system for task-based video analytics processing
US8665333B1 (en) * 2007-01-30 2014-03-04 Videomining Corporation Method and system for optimizing the observation and annotation of complex human behavior from video sources
US9172918B2 (en) * 2007-02-02 2015-10-27 Honeywell International Inc. Systems and methods for managing live video data
US20080232688A1 (en) * 2007-03-20 2008-09-25 Senior Andrew W Event detection in visual surveillance systems
US7679507B2 (en) 2007-05-16 2010-03-16 Honeywell International Inc. Video alarm verification
US8439733B2 (en) * 2007-06-14 2013-05-14 Harmonix Music Systems, Inc. Systems and methods for reinstating a player within a rhythm-action game
US8678896B2 (en) 2007-06-14 2014-03-25 Harmonix Music Systems, Inc. Systems and methods for asynchronous band interaction in a rhythm action game
US10116904B2 (en) 2007-07-13 2018-10-30 Honeywell International Inc. Features in video analytics
EP2174310A4 (en) 2007-07-16 2013-08-21 Cernium Corp Apparatus and methods for video alarm verification
US8675074B2 (en) 2007-07-20 2014-03-18 Honeywell International Inc. Custom video composites for surveillance applications
US20090031381A1 (en) 2007-07-24 2009-01-29 Honeywell International, Inc. Proxy video server for video surveillance
US20090177633A1 (en) * 2007-12-12 2009-07-09 Chumki Basu Query expansion of properties for video retrieval
US8314838B2 (en) * 2007-12-28 2012-11-20 Motorola Mobility Llc System and method for collecting media associated with a mobile device
US10462409B2 (en) * 2007-12-28 2019-10-29 Google Technology Holdings LLC Method for collecting media associated with a mobile device
JP5213105B2 (en) * 2008-01-17 2013-06-19 株式会社日立製作所 Video network system and video data management method
KR101607224B1 (en) 2008-03-03 2016-03-29 아비길론 페이턴트 홀딩 2 코포레이션 Dynamic object classification
US8432492B2 (en) * 2008-03-20 2013-04-30 Institut Fuer Rundfunktechnik Gmbh Method of adapting video images to small screen sizes
US10645346B2 (en) 2013-01-18 2020-05-05 Careview Communications, Inc. Patient video monitoring systems and methods having detection algorithm recovery from changes in illumination
US9866797B2 (en) 2012-09-28 2018-01-09 Careview Communications, Inc. System and method for monitoring a fall state of a patient while minimizing false alarms
US9794523B2 (en) 2011-12-19 2017-10-17 Careview Communications, Inc. Electronic patient sitter management system and method for implementing
US9959471B2 (en) 2008-05-06 2018-05-01 Careview Communications, Inc. Patient video monitoring systems and methods for thermal detection of liquids
US9579047B2 (en) 2013-03-15 2017-02-28 Careview Communications, Inc. Systems and methods for dynamically identifying a patient support surface and patient monitoring
WO2010006054A1 (en) 2008-07-08 2010-01-14 Harmonix Music Systems, Inc. Systems and methods for simulating a rock and band experience
KR101078641B1 (en) * 2008-07-14 2011-11-01 명지대학교 산학협력단 System and method for multimedia application by using metadata for sensory device
US8140215B2 (en) * 2008-07-22 2012-03-20 Lockheed Martin Corporation Method and apparatus for geospatial data sharing
US8339514B2 (en) * 2008-09-03 2012-12-25 Sony Corporation Modular flexible software architecture for TV
US20100070527A1 (en) * 2008-09-18 2010-03-18 Tianlong Chen System and method for managing video, image and activity data
US9123223B1 (en) * 2008-10-13 2015-09-01 Target Brands, Inc. Video monitoring system using an alarm sensor for an exit facilitating access to captured video
US20100097471A1 (en) * 2008-10-17 2010-04-22 Honeywell International Inc. Automated way to effectively handle an alarm event in the security applications
US8471899B2 (en) 2008-12-02 2013-06-25 Careview Communications, Inc. System and method for documenting patient procedures
US8331618B1 (en) * 2008-12-16 2012-12-11 Symantec Corporation Method and apparatus for monitoring instant messaging with visual identification
US9697535B2 (en) 2008-12-23 2017-07-04 International Business Machines Corporation System and method in a virtual universe for identifying spam avatars based upon avatar multimedia characteristics
US9704177B2 (en) 2008-12-23 2017-07-11 International Business Machines Corporation Identifying spam avatars in a virtual universe (VU) based upon turing tests
US8878931B2 (en) 2009-03-04 2014-11-04 Honeywell International Inc. Systems and methods for managing video data
US8654197B2 (en) * 2009-03-04 2014-02-18 Raytheon Company System and method for occupancy detection
US8553778B2 (en) 2009-03-19 2013-10-08 International Business Machines Corporation Coding scheme for identifying spatial locations of events within video image data
US8537219B2 (en) 2009-03-19 2013-09-17 International Business Machines Corporation Identifying spatial locations of events within video image data
US8656476B2 (en) 2009-05-28 2014-02-18 International Business Machines Corporation Providing notification of spam avatars
US8449360B2 (en) 2009-05-29 2013-05-28 Harmonix Music Systems, Inc. Displaying song lyrics and vocal cues
US8465366B2 (en) 2009-05-29 2013-06-18 Harmonix Music Systems, Inc. Biasing a musical performance input to a part
US8544341B2 (en) 2009-06-11 2013-10-01 Ernest W. Grumbles Device including an accelerometer in a watertight container for sensing a urine stream
FR2949268B1 (en) * 2009-08-20 2012-09-28 Radio Systemes Ingenierie Video Technologies DEVICE FOR DETECTION OF ENTRY AND RECOGNITION OF TRANSPONDER BADGES, MONITORING SYSTEM COMPRISING SAME, AND SURVEILLANCE METHOD USED THEREBY
CA2716637A1 (en) 2009-10-07 2011-04-07 Telewatch Inc. Video analytics with pre-processing at the source end
US9420250B2 (en) 2009-10-07 2016-08-16 Robert Laganiere Video analytics method and system
JP5570176B2 (en) * 2009-10-19 2014-08-13 キヤノン株式会社 Image processing system and information processing method
US9981193B2 (en) 2009-10-27 2018-05-29 Harmonix Music Systems, Inc. Movement based recognition and evaluation
EP2494432B1 (en) 2009-10-27 2019-05-29 Harmonix Music Systems, Inc. Gesture-based user interface
US8724970B2 (en) * 2009-10-29 2014-05-13 Verint Systems Inc. Method and apparatus to search video data for an object of interest
US8568234B2 (en) 2010-03-16 2013-10-29 Harmonix Music Systems, Inc. Simulating musical instruments
US9143739B2 (en) 2010-05-07 2015-09-22 Iwatchlife, Inc. Video analytics with burst-like transmission of video data
US9358456B1 (en) 2010-06-11 2016-06-07 Harmonix Music Systems, Inc. Dance competition game
CA2802348A1 (en) 2010-06-11 2011-12-15 Harmonix Music Systems, Inc. Dance game and tutorial
US8562403B2 (en) 2010-06-11 2013-10-22 Harmonix Music Systems, Inc. Prompting a player of a dance game
US20120120201A1 (en) * 2010-07-26 2012-05-17 Matthew Ward Method of integrating ad hoc camera networks in interactive mesh systems
US8860771B2 (en) 2010-08-04 2014-10-14 Iwatchlife, Inc. Method and system for making video calls
US8780162B2 (en) 2010-08-04 2014-07-15 Iwatchlife Inc. Method and system for locating an individual
CA2748059A1 (en) 2010-08-04 2012-02-04 Iwatchlife Inc. Method and system for initiating communication via a communication network
US9024166B2 (en) 2010-09-09 2015-05-05 Harmonix Music Systems, Inc. Preventing subtractive track separation
US9171075B2 (en) 2010-12-30 2015-10-27 Pelco, Inc. Searching recorded video
US9226037B2 (en) 2010-12-30 2015-12-29 Pelco, Inc. Inference engine for video analytics metadata-based event detection and forensic search
KR101792866B1 (en) * 2011-04-06 2017-11-20 삼성전자주식회사 Method and Apparatus for Motion Recognition using event sensor and color sensor
WO2013032933A2 (en) 2011-08-26 2013-03-07 Kinecticor, Inc. Methods, systems, and devices for intra-scan motion correction
US9656392B2 (en) * 2011-09-20 2017-05-23 Disney Enterprises, Inc. System for controlling robotic characters to enhance photographic results
US9681125B2 (en) 2011-12-29 2017-06-13 Pelco, Inc Method and system for video coding with noise filtering
WO2013131189A1 (en) * 2012-03-08 2013-09-12 Iwatchlife Inc. Cloud-based video analytics with post-processing at the video source-end
WO2014014238A1 (en) * 2012-07-17 2014-01-23 Samsung Electronics Co., Ltd. System and method for providing image
CA2822217A1 (en) 2012-08-02 2014-02-02 Iwatchlife Inc. Method and system for anonymous video analytics processing
US9124856B2 (en) * 2012-08-31 2015-09-01 Disney Enterprises, Inc. Method and system for video event detection for contextual annotation and synchronization
JP5962916B2 (en) * 2012-11-14 2016-08-03 パナソニックIpマネジメント株式会社 Video surveillance system
US10327708B2 (en) 2013-01-24 2019-06-25 Kineticor, Inc. Systems, devices, and methods for tracking and compensating for patient motion during a medical imaging scan
US9717461B2 (en) 2013-01-24 2017-08-01 Kineticor, Inc. Systems, devices, and methods for tracking and compensating for patient motion during a medical imaging scan
US9305365B2 (en) 2013-01-24 2016-04-05 Kineticor, Inc. Systems, devices, and methods for tracking moving targets
CN109008972A (en) 2013-02-01 2018-12-18 凯内蒂科尔股份有限公司 The motion tracking system of real-time adaptive motion compensation in biomedical imaging
CN103260010B (en) * 2013-04-23 2017-02-08 四川天翼网络服务有限公司 Intelligent skynet rapid video retrieval system
US9792951B2 (en) * 2013-05-08 2017-10-17 Vieu Labs, Inc. Systems and methods for identifying potentially interesting events in extended recordings
US20150010289A1 (en) * 2013-07-03 2015-01-08 Timothy P. Lindblom Multiple retail device universal data gateway
CN103475882B (en) * 2013-09-13 2017-02-15 北京大学 Surveillance video encoding and recognizing method and surveillance video encoding and recognizing system
US10341684B2 (en) * 2014-03-07 2019-07-02 Eagle Eye Networks, Inc. High definition surveillance image storage optimization apparatus and methods of retention triggering
WO2015148391A1 (en) 2014-03-24 2015-10-01 Thomas Michael Ernst Systems, methods, and devices for removing prospective motion correction from medical imaging scans
CA2947160A1 (en) * 2014-04-30 2015-11-05 Cubic Corporation Fare evasion detection using video analytics
US9600743B2 (en) * 2014-06-27 2017-03-21 International Business Machines Corporation Directing field of vision based on personal interests
US9449229B1 (en) 2014-07-07 2016-09-20 Google Inc. Systems and methods for categorizing motion event candidates
US10127783B2 (en) * 2014-07-07 2018-11-13 Google Llc Method and device for processing motion events
US9224044B1 (en) 2014-07-07 2015-12-29 Google Inc. Method and system for video zone monitoring
US10140827B2 (en) * 2014-07-07 2018-11-27 Google Llc Method and system for processing motion event notifications
US9501915B1 (en) 2014-07-07 2016-11-22 Google Inc. Systems and methods for analyzing a video stream
US9734589B2 (en) 2014-07-23 2017-08-15 Kineticor, Inc. Systems, devices, and methods for tracking and compensating for patient motion during a medical imaging scan
CN104166604A (en) * 2014-08-15 2014-11-26 小米科技有限责任公司 Video backup method and device
US9009805B1 (en) 2014-09-30 2015-04-14 Google Inc. Method and system for provisioning an electronic device
USD782495S1 (en) 2014-10-07 2017-03-28 Google Inc. Display screen or portion thereof with graphical user interface
US20160133036A1 (en) * 2014-11-12 2016-05-12 Honeywell International Inc. Systems and methods for displaying facility information
US9639768B2 (en) * 2014-12-09 2017-05-02 Xerox Corporation Methods and systems to adaptively detect object borders from electronic device images
WO2016122540A1 (en) * 2015-01-29 2016-08-04 Hewlett Packard Enterprise Development Lp Encoder
US10021119B2 (en) 2015-02-06 2018-07-10 Honeywell International Inc. Apparatus and method for automatic handling of cyber-security risk events
US20160234243A1 (en) * 2015-02-06 2016-08-11 Honeywell International Inc. Technique for using infrastructure monitoring software to collect cyber-security risk data
US10021125B2 (en) 2015-02-06 2018-07-10 Honeywell International Inc. Infrastructure monitoring tool for collecting industrial process control and automation system risk data
US10075474B2 (en) 2015-02-06 2018-09-11 Honeywell International Inc. Notification subsystem for generating consolidated, filtered, and relevant security risk-based notifications
US10075475B2 (en) 2015-02-06 2018-09-11 Honeywell International Inc. Apparatus and method for dynamic customization of cyber-security risk item rules
US10298608B2 (en) 2015-02-11 2019-05-21 Honeywell International Inc. Apparatus and method for tying cyber-security risk analysis to common risk methodologies and risk levels
US10042031B2 (en) 2015-02-11 2018-08-07 Xerox Corporation Method and system for detecting that an object of interest has re-entered a field of view of an imaging device
TWI571697B (en) * 2015-04-28 2017-02-21 Chaoyang Univ Of Tech Processing of Shrinking Photography Services
US9800604B2 (en) 2015-05-06 2017-10-24 Honeywell International Inc. Apparatus and method for assigning cyber-security risk consequences in industrial process control environments
US20160365116A1 (en) * 2015-06-11 2016-12-15 Yaron Galant Video editing apparatus with participant sharing
US9361011B1 (en) 2015-06-14 2016-06-07 Google Inc. Methods and systems for presenting multiple live video feeds in a user interface
AU2015203661A1 (en) * 2015-06-30 2017-01-19 Canon Kabushiki Kaisha Method, apparatus and system for applying an annotation to a portion of a video sequence
US9943247B2 (en) 2015-07-28 2018-04-17 The University Of Hawai'i Systems, devices, and methods for detecting false movements for motion correction during a medical imaging scan
CN106384532A (en) * 2015-07-31 2017-02-08 富士通株式会社 Video data analysis method and apparatus thereof, and parking space monitoring system
CN105357475A (en) * 2015-10-28 2016-02-24 小米科技有限责任公司 Video playing method and device
US10716515B2 (en) 2015-11-23 2020-07-21 Kineticor, Inc. Systems, devices, and methods for tracking and compensating for patient motion during a medical imaging scan
US10475315B2 (en) 2016-03-22 2019-11-12 Sensormatic Electronics, LLC System and method for configuring surveillance cameras using mobile computing devices
US10192414B2 (en) * 2016-03-22 2019-01-29 Sensormatic Electronics, LLC System and method for overlap detection in surveillance camera network
US10764539B2 (en) 2016-03-22 2020-09-01 Sensormatic Electronics, LLC System and method for using mobile device of zone and correlated motion detection
US10347102B2 (en) 2016-03-22 2019-07-09 Sensormatic Electronics, LLC Method and system for surveillance camera arbitration of uplink consumption
US10318836B2 (en) * 2016-03-22 2019-06-11 Sensormatic Electronics, LLC System and method for designating surveillance camera regions of interest
US10665071B2 (en) 2016-03-22 2020-05-26 Sensormatic Electronics, LLC System and method for deadzone detection in surveillance camera network
US11216847B2 (en) 2016-03-22 2022-01-04 Sensormatic Electronics, LLC System and method for retail customer tracking in surveillance camera network
US11601583B2 (en) 2016-03-22 2023-03-07 Johnson Controls Tyco IP Holdings LLP System and method for controlling surveillance cameras
US9965680B2 (en) 2016-03-22 2018-05-08 Sensormatic Electronics, LLC Method and system for conveying data from monitored scene via surveillance cameras
US10733231B2 (en) 2016-03-22 2020-08-04 Sensormatic Electronics, LLC Method and system for modeling image of interest to users
US10506237B1 (en) 2016-05-27 2019-12-10 Google Llc Methods and devices for dynamic adaptation of encoding bitrate for video streaming
US10380429B2 (en) 2016-07-11 2019-08-13 Google Llc Methods and systems for person detection in a video feed
US10957171B2 (en) 2016-07-11 2021-03-23 Google Llc Methods and systems for providing event alerts
US10192415B2 (en) 2016-07-11 2019-01-29 Google Llc Methods and systems for providing intelligent alerts for events
US11545013B2 (en) * 2016-10-26 2023-01-03 A9.Com, Inc. Customizable intrusion zones for audio/video recording and communication devices
KR102634188B1 (en) * 2016-11-30 2024-02-05 한화비전 주식회사 System for monitoring image
US11783010B2 (en) 2017-05-30 2023-10-10 Google Llc Systems and methods of person recognition in video streams
US10599950B2 (en) 2017-05-30 2020-03-24 Google Llc Systems and methods for person recognition data management
US11450148B2 (en) 2017-07-06 2022-09-20 Wisconsin Alumni Research Foundation Movement monitoring system
US10810414B2 (en) 2017-07-06 2020-10-20 Wisconsin Alumni Research Foundation Movement monitoring system
US10482613B2 (en) 2017-07-06 2019-11-19 Wisconsin Alumni Research Foundation Movement monitoring system
US10664688B2 (en) 2017-09-20 2020-05-26 Google Llc Systems and methods of detecting and responding to a visitor to a smart home environment
US11134227B2 (en) 2017-09-20 2021-09-28 Google Llc Systems and methods of presenting appropriate actions for responding to a visitor to a smart home environment
TWI637354B (en) * 2017-10-23 2018-10-01 緯創資通股份有限公司 Image detection method and image detection device for determining postures of user
US10599947B2 (en) 2018-03-09 2020-03-24 Ricoh Co., Ltd. On-demand visual analysis focalized on salient events
US10958955B1 (en) * 2018-09-13 2021-03-23 Amazon Technologies, Inc. System for generating modified video output
JP7368822B2 (en) * 2019-05-31 2023-10-25 i-PRO株式会社 Camera parameter setting system and camera parameter setting method
AU2019454248B2 (en) * 2019-06-25 2023-07-20 Motorola Solutions, Inc System and method for saving bandwidth in performing facial recognition
US11587361B2 (en) 2019-11-08 2023-02-21 Wisconsin Alumni Research Foundation Movement monitoring system
US11893795B2 (en) 2019-12-09 2024-02-06 Google Llc Interacting with visitors of a connected home environment
US20230140686A1 (en) * 2020-07-17 2023-05-04 Philip Markowitz Video Enhanced Time Tracking System and Method
US11809675B2 (en) * 2022-03-18 2023-11-07 Carrier Corporation User interface navigation method for event-related video

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030058341A1 (en) * 2001-09-27 2003-03-27 Koninklijke Philips Electronics N.V. Video based detection of fall-down and other events
US6570608B1 (en) * 1998-09-30 2003-05-27 Texas Instruments Incorporated System and method for detecting interactions of people and vehicles
US20030133614A1 (en) * 2002-01-11 2003-07-17 Robins Mark N. Image capturing device for event monitoring
EP1403817A1 (en) * 2000-09-06 2004-03-31 Hitachi, Ltd. Abnormal behavior detector
US20040263621A1 (en) * 2001-09-14 2004-12-30 Guo Chun Biao Customer service counter/checkpoint registration system with video/image capturing, indexing, retrieving and black list matching function
US20050008198A1 (en) * 2001-09-14 2005-01-13 Guo Chun Biao Apparatus and method for selecting key frames of clear faces through a sequence of images

Family Cites Families (69)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5377258A (en) * 1993-08-30 1994-12-27 National Medical Research Council Method and apparatus for an automated and interactive behavioral guidance system
US5521841A (en) * 1994-03-31 1996-05-28 Siemens Corporate Research, Inc. Browsing contents of a given video sequence
US5634008A (en) * 1994-07-18 1997-05-27 International Business Machines Corporation Method and system for threshold occurrence detection in a communications network
US5821945A (en) * 1995-02-03 1998-10-13 The Trustees Of Princeton University Method and apparatus for video browsing based on content and structure
US5708767A (en) * 1995-02-03 1998-01-13 The Trustees Of Princeton University Method and apparatus for video browsing based on content and structure
US7076102B2 (en) * 2001-09-27 2006-07-11 Koninklijke Philips Electronics N.V. Video monitoring system employing hierarchical hidden markov model (HMM) event learning and classification
US6181867B1 (en) * 1995-06-07 2001-01-30 Intervu, Inc. Video storage and retrieval system
US5751336A (en) * 1995-10-12 1998-05-12 International Business Machines Corporation Permutation based pyramid block transmission scheme for broadcasting in video-on-demand storage systems
US5969755A (en) * 1996-02-05 1999-10-19 Texas Instruments Incorporated Motion based event detection system and method
US5828809A (en) * 1996-10-01 1998-10-27 Matsushita Electric Industrial Co., Ltd. Method and apparatus for extracting indexing information from digital video data
US5974235A (en) * 1996-10-31 1999-10-26 Sensormatic Electronics Corporation Apparatus having flexible capabilities for analysis of video information
US6222532B1 (en) * 1997-02-03 2001-04-24 U.S. Philips Corporation Method and device for navigating through video matter by means of displaying a plurality of key-frames in parallel
US6445409B1 (en) * 1997-05-14 2002-09-03 Hitachi Denshi Kabushiki Kaisha Method of distinguishing a moving object and apparatus of tracking and monitoring a moving object
JP3780623B2 (en) * 1997-05-16 2006-05-31 株式会社日立製作所 Video description method
US6091821A (en) * 1998-02-12 2000-07-18 Vlsi Technology, Inc. Pipelined hardware implementation of a hashing algorithm
US6724915B1 (en) * 1998-03-13 2004-04-20 Siemens Corporate Research, Inc. Method for tracking a video object in a time-ordered sequence of image frames
US6697103B1 (en) * 1998-03-19 2004-02-24 Dennis Sunga Fernandez Integrated network for monitoring remote objects
US6018359A (en) * 1998-04-24 2000-01-25 Massachusetts Institute Of Technology System and method for multicast video-on-demand delivery system
US6359647B1 (en) * 1998-08-07 2002-03-19 Philips Electronics North America Corporation Automated camera handoff system for figure tracking in a multiple camera system
US20040261626A1 (en) * 2002-03-26 2004-12-30 Tmio, Llc Home appliances provided with control systems which may be actuated from a remote location
US6628835B1 (en) * 1998-08-31 2003-09-30 Texas Instruments Incorporated Method and system for defining and recognizing complex events in a video sequence
JP4103192B2 (en) * 1998-09-17 2008-06-18 ソニー株式会社 Editing system and editing method
US6721454B1 (en) * 1998-10-09 2004-04-13 Sharp Laboratories Of America, Inc. Method for automatic extraction of semantically significant events from video
US6643387B1 (en) * 1999-01-28 2003-11-04 Sarnoff Corporation Apparatus and method for context-based indexing and retrieval of image sequences
JP2000276577A (en) 1999-03-25 2000-10-06 Fujitsu Ltd Image sensitive event generator
US6779027B1 (en) * 1999-04-30 2004-08-17 Hewlett-Packard Development Company, L.P. Intelligent management module application programming interface with utility objects
US6424370B1 (en) * 1999-10-08 2002-07-23 Texas Instruments Incorporated Motion based event detection system and method
US6754389B1 (en) * 1999-12-01 2004-06-22 Koninklijke Philips Electronics N.V. Program classification using object tracking
AUPQ535200A0 (en) * 2000-01-31 2000-02-17 Canon Kabushiki Kaisha Extracting key frames from a video sequence
US6940998B2 (en) * 2000-02-04 2005-09-06 Cernium, Inc. System for automated screening of security cameras
GB0004142D0 (en) 2000-02-23 2000-04-12 Univ Manchester Monitoring system
US7106885B2 (en) * 2000-09-08 2006-09-12 Carecord Technologies, Inc. Method and apparatus for subject physical position and security determination
US6678413B1 (en) * 2000-11-24 2004-01-13 Yiqing Liang System and method for object identification and behavior characterization using video analysis
US20030051026A1 (en) * 2001-01-19 2003-03-13 Carter Ernst B. Network surveillance and security system
US7346186B2 (en) * 2001-01-30 2008-03-18 Nice Systems Ltd Video and audio content analysis system
US20020107949A1 (en) * 2001-02-08 2002-08-08 International Business Machines Corporation Polling for and transfer of protocol data units in a data processing network
US6970640B2 (en) * 2001-05-14 2005-11-29 Microsoft Corporation Systems and methods for playing digital video in reverse and fast forward modes
US20020171734A1 (en) 2001-05-16 2002-11-21 Hiroshi Arakawa Remote monitoring system
US20030123703A1 (en) * 2001-06-29 2003-07-03 Honeywell International Inc. Method for monitoring a moving object and system regarding same
US20030053659A1 (en) * 2001-06-29 2003-03-20 Honeywell International Inc. Moving object assessment system and method
US6845357B2 (en) * 2001-07-24 2005-01-18 Honeywell International Inc. Pattern recognition using an observable operator model
KR100442170B1 (en) * 2001-10-05 2004-07-30 (주)아이디스 Remote Control and Management System
US7020336B2 (en) * 2001-11-13 2006-03-28 Koninklijke Philips Electronics N.V. Identification and evaluation of audience exposure to logos in a broadcast event
US20030126293A1 (en) * 2001-12-27 2003-07-03 Robert Bushey Dynamic user interface reformat engine
EP1329869A1 (en) * 2002-01-16 2003-07-23 Deutsche Thomson-Brandt Gmbh Method and apparatus for processing video pictures
US6879709B2 (en) * 2002-01-17 2005-04-12 International Business Machines Corporation System and method for automatically detecting neutral expressionless faces in digital images
US20030156824A1 (en) * 2002-02-21 2003-08-21 Koninklijke Philips Electronics N.V. Simultaneous viewing of time divided segments of a tv program
JP3999561B2 (en) * 2002-05-07 2007-10-31 松下電器産業株式会社 Surveillance system and surveillance camera
US6948082B2 (en) * 2002-05-17 2005-09-20 International Business Machines Corporation Method and apparatus for software-assisted thermal management for electronic systems
EP1527554B1 (en) * 2002-07-29 2014-12-31 Baumüller Anlagen-Systemtechnik GmbH & Co. KG Computer network with diagnosis computer nodes
US20040080615A1 (en) * 2002-08-21 2004-04-29 Strategic Vista Intenational Inc. Digital video security system
US7200266B2 (en) * 2002-08-27 2007-04-03 Princeton University Method and apparatus for automated video activity analysis
JP2004112153A (en) * 2002-09-17 2004-04-08 Fujitsu Ltd Image processing system
US7295673B2 (en) * 2002-10-23 2007-11-13 Divx, Inc. Method and system for securing compressed digital video
WO2004045215A1 (en) * 2002-11-12 2004-05-27 Intellivid Corporation Method and system for tracking and behavioral monitoring of multiple objects moving throuch multiple fields-of-view
US7194110B2 (en) * 2002-12-18 2007-03-20 Intel Corporation Method and apparatus for tracking features in a video sequence
US7469343B2 (en) * 2003-05-02 2008-12-23 Microsoft Corporation Dynamic substitution of USB data for on-the-fly encryption/decryption
US7986339B2 (en) * 2003-06-12 2011-07-26 Redflex Traffic Systems Pty Ltd Automated traffic violation monitoring and reporting system with combined video and still-image data
US7159234B1 (en) * 2003-06-27 2007-01-02 Craig Murphy System and method for streaming media server single frame failover
US7352952B2 (en) * 2003-10-16 2008-04-01 Magix Ag System and method for improved video editing
JP2005295255A (en) 2004-03-31 2005-10-20 Toshiba Corp Imaging apparatus and method for transmitting photographed image
US8724891B2 (en) * 2004-08-31 2014-05-13 Ramot At Tel-Aviv University Ltd. Apparatus and methods for the detection of abnormal motion in a video stream
HK1066447A1 (en) * 2004-09-14 2005-02-04 Multivision Intelligent Survei Backup system for digital surveillance system.
US20060064731A1 (en) * 2004-09-20 2006-03-23 Mitch Kahle System and method for automated production of personalized videos on digital media of individual participants in large events
US8019175B2 (en) * 2005-03-09 2011-09-13 Qualcomm Incorporated Region-of-interest processing for video telephony
US8977063B2 (en) * 2005-03-09 2015-03-10 Qualcomm Incorporated Region-of-interest extraction for video telephony
US7760908B2 (en) 2005-03-31 2010-07-20 Honeywell International Inc. Event packaged video sequence
US20060238616A1 (en) * 2005-03-31 2006-10-26 Honeywell International Inc. Video image processing appliance manager
US20070006270A1 (en) 2005-06-29 2007-01-04 Nortel Networks Limited Timely recovery for media on demand streaming

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6570608B1 (en) * 1998-09-30 2003-05-27 Texas Instruments Incorporated System and method for detecting interactions of people and vehicles
EP1403817A1 (en) * 2000-09-06 2004-03-31 Hitachi, Ltd. Abnormal behavior detector
US20040263621A1 (en) * 2001-09-14 2004-12-30 Guo Chun Biao Customer service counter/checkpoint registration system with video/image capturing, indexing, retrieving and black list matching function
US20050008198A1 (en) * 2001-09-14 2005-01-13 Guo Chun Biao Apparatus and method for selecting key frames of clear faces through a sequence of images
US20030058341A1 (en) * 2001-09-27 2003-03-27 Koninklijke Philips Electronics N.V. Video based detection of fall-down and other events
US20030133614A1 (en) * 2002-01-11 2003-07-17 Robins Mark N. Image capturing device for event monitoring

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2009078946A1 (en) * 2007-12-18 2009-06-25 Eastman Kodak Company Video customized to include person-of-interest
WO2011059659A1 (en) * 2009-11-12 2011-05-19 Siemens Industry, Inc. System and method for annotating video with geospatially referenced data
EP2580738A4 (en) * 2010-08-10 2018-01-03 LG Electronics Inc. Region of interest based video synopsis
EP2442284A1 (en) * 2010-10-14 2012-04-18 Honeywell International Inc. Graphical bookmarking of video data with user inputs in video surveillance
US8346056B2 (en) 2010-10-14 2013-01-01 Honeywell International Inc. Graphical bookmarking of video data with user inputs in video surveillance
EP2557784A3 (en) * 2011-08-12 2014-08-20 Honeywell International Inc. System and method of creating an intelligent video clip for improved investigations in video surveillance
US8837906B2 (en) 2012-12-14 2014-09-16 Motorola Solutions, Inc. Computer assisted dispatch incident report video search and tagging systems and methods

Also Published As

Publication number Publication date
US20060239645A1 (en) 2006-10-26
EP1864496A1 (en) 2007-12-12
US7760908B2 (en) 2010-07-20

Similar Documents

Publication Publication Date Title
US7760908B2 (en) Event packaged video sequence
US7801328B2 (en) Methods for defining, detecting, analyzing, indexing and retrieving events using video image processing
EP1435170B2 (en) Video tripwire
US6970083B2 (en) Video tripwire
EP1346577B1 (en) Method and apparatus to select the best video frame to transmit to a remote station for cctv based residential security monitoring
US8675074B2 (en) Custom video composites for surveillance applications
CA2814366C (en) System and method of post event/alarm analysis in cctv and integrated security systems
US7606425B2 (en) Unsupervised learning of events in a video sequence
CA2601477C (en) Intelligent camera selection and object tracking
US20060238616A1 (en) Video image processing appliance manager
US20130208123A1 (en) Method and System for Collecting Evidence in a Security System
EP0967584A2 (en) Automatic video monitoring system
EP2770733A1 (en) A system and method to create evidence of an incident in video surveillance system
US20170300751A1 (en) Smart history for computer-vision based security system
WO2003005315A1 (en) Vision based method and apparatus for detecting an event requiring assistance or documentation
US11056150B2 (en) Multi-time search analytics

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 2006735551

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

NENP Non-entry into the national phase

Ref country code: RU