WO2014182898A1 - User interface for effective video surveillance - Google Patents

User interface for effective video surveillance Download PDF

Info

Publication number
WO2014182898A1
WO2014182898A1 PCT/US2014/037297 US2014037297W WO2014182898A1 WO 2014182898 A1 WO2014182898 A1 WO 2014182898A1 US 2014037297 W US2014037297 W US 2014037297W WO 2014182898 A1 WO2014182898 A1 WO 2014182898A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
information
facility
schematic
floor map
Prior art date
Application number
PCT/US2014/037297
Other languages
French (fr)
Inventor
Xianjun S. Zheng
Vinay Damodar SHET
Siyu CHEN
Original Assignee
Siemens Aktiengesellschaft
Siemens Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Siemens Aktiengesellschaft, Siemens Corporation filed Critical Siemens Aktiengesellschaft
Publication of WO2014182898A1 publication Critical patent/WO2014182898A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/18Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/20Scenes; Scene-specific elements in augmented reality scenes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/52Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/54Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19678User interface
    • G08B13/19682Graphic User Interface [GUI] presenting system data to the user, e.g. information on a screen helping a user interacting with an alarm system

Definitions

  • Embodiments of the present disclosure are directed to a method and user interface (UI) for effective video surveillance.
  • UI user interface
  • Exemplary embodiments of the disclosure as described herein generally include methods and user interfaces (UI) for effective video surveillance that include three characteristics that distinguish them from existing conventional video surveillance system.
  • UI user interfaces
  • a method for creating a video layout for video surveillance of a facility including defining pathway information for a facility, locating camera positions of a plurality of video surveillance cameras in a floor map of said facility, overlaying the pathway information on the floor map, locating target path elements in the overlaid floor map, optimizing the overlaid floor map to simplify all paths into one of a horizontal, vertical, or diagonal line, and generating a schematic video layout from the optimized overlaid floor map facility, said schematic video layout comprising a plurality of video images from the plurality of video surveillance cameras in said facility, wherein said schematic video layout represents spatial and geographic relationships among the plurality of video surveillance cameras.
  • the method includes extracting topological information from the pathway information on the floor map, including space and corridor information through which people can move, and physical constrains or blocks through which people cannot go.
  • target path elements are points of interests, including specific entrances, rooms, and spaces.
  • optimization comprises linear fitting.
  • optimization includes setting thresholds and tolerance levels for an acceptable level of error for a deviation angle between an angle of a schematic path and the angle of an actual geographical path.
  • pathway information is defined from a building information model. According to a further embodiment of the invention, pathway information is defined manually.
  • a method of performing video surveillance of a facility including providing a schematic video layout of a facility, said schematic video layout comprising a plurality of video images from a plurality of video surveillance cameras in said facility, wherein said schematic video layout represents spatial and geographic relationships among the plurality of video surveillance cameras, enlarging a size of a video image for a video camera in said schematic video layout when an event is detected for said camera, presenting event type information next to said enlarged video image, and creating a 2-dimensional space-time trail from successive enlarged images to present the integrated temporal and spatial information.
  • the event type information includes a thumbnail of a detected object, and an action button.
  • creating a 2-dimensional space- time trail from successive enlarged images includes linearizing the spatial information using the schematic video layout to transform 2-dimensional or 3-dimensional spatial information into 1 -dimensional path information, and creating a 2-dimensional space-time trail by combining the path information with a timeline of detected events, wherein one axis is a spatial axis and a second axis is a time axis, wherein a detected event is represented by a symbol located at a space-time coordinate in said 2-dimensional space-time trail that corresponds to a location and time of said detected event.
  • linearizing the spatial information includes receiving one or more user defined paths extracted from the floor map and schematic video layout, inferring a most significant path from the one or more user defined paths by conducting a graph analysis of said one or more user defined paths, and tracking path usage of said most significant path.
  • the method when an event is detected for a camera, the method includes separately displaying an image from the camera and for one or more cameras proximal to the camera in a schematic video layout on another video monitor.
  • a non- transitory program storage device readable by a computer, tangibly embodying a program of instructions executed by the computer to perform the method steps for creating a video layout for video surveillance of a facility.
  • FIG. 1 illustrates an example of a conventional grid video layout, in which the cameras are displayed in an almost random fashion in a 3 by 3 matrix, according to an embodiment of the disclosure.
  • FIG. 2 shows an example of mapping cameras directly on a map, according to an embodiment of the disclosure.
  • FIG. 3 is a flowchart of a method for generating schematic video layout, according to an embodiment of the disclosure.
  • FIG. 4 illustrates a schematic video layout example, showing the relationships among the various cameras, according to an embodiment of the disclosure.
  • FIG. 5 illustrates an adaptive resizing in which the size of the video becomes bigger as the alert is triggered, according to an embodiment of the disclosure.
  • FIGS. 6(a)-(b) illustrates how an alerts' information is attached along the side of the camera view, providing the contextual information showing where the events occurred, according to an embodiment of the disclosure.
  • FIGS. 7(a)-(b) illustrates an alert tray displayed next to the camera view, according to an embodiment of the disclosure.
  • FIG. 8 illustrates an example of an activity trail for integrating both temporal and spatial information of events, according to an embodiment of the disclosure.
  • FIG. 9 illustrates an example of how to linearize 2D or 3D spatial information into 1- dimensional path information, according to an embodiment of the disclosure.
  • FIG. 10 illustrates two persons' paths or activity trails, according to an embodiment of the disclosure.
  • FIGS. 1 l(a)-(b) illustrates monitoring and tracking for situation awareness, according to an embodiment of the invention.
  • FIG. 12 illustrates an aggregate view of the activity trails according to embodiments of the disclosure.
  • FIG. 13 is a flowchart of a method for producing an activity trail, according to an embodiment of the disclosure.
  • FIG. 14 is a block diagram of an exemplary computer system for implementing a method and user interface (UI) for effective video surveillance, according to an embodiment of the disclosure.
  • UI user interface
  • Exemplary embodiments of the disclosure as described herein generally include methods and user interfaces (UI) for effective video surveillance. Accordingly, while the disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that there is no intent to limit the disclosure to the particular forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the disclosure.
  • image refers to multi-dimensional data composed of discrete image elements (e.g., pixels for 2-D images and voxels for 3-D images).
  • the image may be, for example, a medical image of a subject collected by computer tomography, magnetic resonance imaging, ultrasound, or any other medical imaging system known to one of skill in the art.
  • the image may also be provided from non-medical contexts, such as, for example, remote sensing systems or video surveillance systems, etc.
  • an image can be thought of as a function from R 3 to R, the methods of the inventions are not limited to such images, and can be applied to images of any dimension, e.g. a 2-D picture or a 3-D volume.
  • the domain of the image is typically a 2- or 3-dimensional rectangular array, wherein each pixel or voxel can be addressed with reference to a set of 2 or 3 mutually orthogonal axes.
  • digital and digitized as used herein will refer to images or volumes, as appropriate, in a digital or digitized format acquired via a digital acquisition system or via conversion from an analog image.
  • FIG. 1 illustrates an example of a conventional grid video layout, in which the cameras (seven in FIG. 1) are displayed in an almost random fashion in a 3 by 3 matrix. This idea probably can be traced back to the previous technology of a physical video wall, where one TV monitor is connected to one video camera.
  • this grid layout presentation with cameras almost randomly placed in the matrix, does not support operators' mental model for the actual physical space. For example, information regarding the spatial relationships among the different cameras is lost. The operator needs to memorize which camera maps to which specific location in the actual physical space. Linking and connecting information across different cameras becomes a mental challenging task. Imagine a suspicious person is moving across different locations in the space; tracking that person can be a challenging task because the operator has to guess which cameras would capture that person next, and then try to locate the camera in the video matrix.
  • FIG. 2 shows an example of mapping cameras directly on a map.
  • this direct overlay approach can easily generate visual clutter because the map already contains a lot of data that are not necessarily relevant for video surveillance.
  • the video thumbnails are directly mapped to their geographical locations, the small video size does not show much information to the operator, yet a large video size will cover the map.
  • the lack of visual alignment among the video thumb nails can also contribute to the visual clutter.
  • a schematic video layout according to an embodiment of the disclosure can represent the most important information of the video cameras to the operators, namely, namely the relationships among different video cameras rather than capturing the accurate geographical information. As a result, it is easy for operators to look for the information they need: which location is being viewed, what is the next location, and where to make a transition to another location, etc.
  • Research in psychology shows that people's the mental representation of a space, i.e., the cognitive map, is schematic rather than an accurate representation of the physical world.
  • FIG. 4 illustrates a schematic video layout example for six cameras according to an embodiment of the disclosure, showing the relationships among the various cameras.
  • FIG. 4 shows that camera 1 is above camera 2, and there is a connection between them; then both camera 3 and 5 are also connected to camera 2. If a person is spotted in camera 2, then there are three possible cameras, i.e., 1, 3, 5, for the person to show up next. If a person is spotted in camera 5, then both camera 2 and 6 may be the next likely cameras to see the person.
  • FIG. 3 A flowchart of a method for generating schematic video layout of a plurality of video surveillance cameras of a building or facility is presented in FIG. 3.
  • layout information such as a floor map or a schematic video layout
  • G (N, E) comprising a set of nodes N together with a set E of edges or lines.
  • a method starts at step 31 by defining pathway information in the building or facility using a building information model (BIM), which is a digital representation of the physical and functional characteristics of a building that can be exchanged or networked, and can be used as a shared knowledge resource for making decisions during the life-cycle of a building.
  • BIM building information model
  • IFCs Industry Foundation Classes
  • aecXML is the is a specific XML mark-up language which is based on an IFC to create a vendor-neutral means to access data generated by BIM.
  • pathways related information such as corridors and doors, is defined in the Architecture Domain, and this information can be automatically extracted by a XML parser following the aecXML definitions. If no BIM is available, pathway information can be defined manually.
  • Camera positions of the video surveillance cameras are located in a floor map of the building at step 33, which may be any type of digital floor plan, such as a CAD drawing.
  • the camera locations can be marked directly on the floor map, represented by camera icons, and their directions represented by, e.g., fanout shapes.
  • the pathway information is overlaid on the floor map, and topological information is extracted.
  • the topological information relates to space and corridor information where people can move through, as well as the physical constrains or blocks where people cannot go through.
  • topological information may be automatically extracted by extracting the nodes and connection information directly from the BIM model, or manually by having a user define nodes, i.e. areas of interests, and their connections, i.e. edges, based on the provided floor map.
  • target path elements which are points of interests, such as a specific entrance, or a room, or any space, are located in the overlaid floor map.
  • Target path elements include both nodes, the area of interests, and their connections, represented as edges, based on the floor map.
  • Optimization such as linear fitting, can be applied at step 36, and the user may adjust a threshold or tolerance level. Thresholds and tolerance levels refer to the level of error acceptable during the optimization.
  • An optimization according to an embodiment of the invention abstracts and simplifies all the paths into either horizontal, vertical, or diagonal (45 degree angle) lines.
  • An exemplary, non- limiting threshold value can be from 0 to 90 degrees depending on the tolerance levels.
  • a schematic video layout is generated from the optimized overlaid floor map at step 37.
  • An optimization algorithm according to an embodiment of the invention can reduce the visual complexity of the graph by straightening the lines or connections to make them, for example, either horizontal, vertical or diagonal.
  • a schematic video layout according to an embodiment of the disclosure therefore, can effectively support video surveillance operators in linking and connecting different videos together, to gain a better situation awareness of the environment. For example, because there are often many videos that an operator needs to monitor, and the operator can also be distracted by other tasks, it is useful to catch an operator's attention when some event does happen. Events, such as a person is detected, or a person is entering a room, are detected in real-time by video content analysis modules.
  • Video content analysis makes use of technologies disclosed in copending applications numbers 13/646867, "Method and User Interface for Forensic Video Search” filed on October 8, 2012, and 13/693,231, “Configuration Tool for Video Analytics” filed on December 4, 2012, the contents of both of which are herein incorporated by reference in their entireties.
  • the time and camera location information are recorded, both of which will be mapped to the activity trail, as will be described below.
  • adaptive resizing can cause the size of the camera video to be enlarged and highlighted to catch operators' attention.
  • FIG. 5 illustrates an adaptive resizing according to an embodiment of the disclosure, in which the size of the video becomes bigger as the alert is triggered.
  • a system can provide a vocabulary and grammar to allow a facility operator to define an unlimited number of event types or categories.
  • the grammar in general, is "Who” does "What", or “What”, for example, "Enter the lobby”, “Exit the building”, “Enter the mailroom”, and "Who” can be a person, or a couple of persons, or even an object, like a car.
  • FIGS. 6(a)-(b) illustrate how an alerts' information is attached along the side of the camera view for camera 4, providing the contextual information showing where the events occurred.
  • FIG. 6(a) shows the view from camera 4 before the event was detected, as shown in FIG. 4
  • FIG. 6(b) shows the view from camera 4 after the event was detected, as shown in FIG. 5.
  • a quick action button can be displayed, supporting the quick actions for the operator.
  • FIGS. 7(a)-(b) illustrates an alert tray displayed next to the camera view according to an embodiment of the disclosure, which contains the gist information for the event, e.g., showing a thumbnail 71 for the detected person or object, in FIG. 7(a).
  • FIG. 7(a) illustrates an alert tray displayed next to the camera view according to an embodiment of the disclosure, which contains the gist information for the event, e.g., showing a thumbnail 71 for the detected person or object, in FIG. 7(a).
  • FIG. 7(a) illustrates
  • action items 72, 73 may also be displayed, allowing operators' quick actions.
  • the eyeball 72 activates a tracking function, in which the system will automatically monitor and collect information about the object or person of interests.
  • the magnifying glass 72 activates a forensic search, which invokes another interface where the operator can look for more information regarding this object or person.
  • an activity trail is a 2-dimensional space-time trail that can effectively present the integrated temporal and spatial information.
  • FIG. 8 illustrates an example of an activity trail for integrating both temporal and spatial information of events, according to an embodiment of the disclosure. Referring now to FIG. 8, different spatial locations are identified in the highlighted column 80a on the left side, and a time axis 80b in 24 hour units on the top side of the figure.
  • Needle 88 is a scene picker for controlling where to start viewing a video of the detected events.
  • FIG. 13 A flow chart of a method for producing an activity trail according to an embodiment of the disclosure is presented in FIG. 13.
  • a first step 131 in producing an activity trail according to an embodiment of the disclosure is to linearize the spatial information, which was recorded when the event occurred, transforming the 2D or 3D spatial information into ID, e.g., path information.
  • FIG. 9 illustrates an example of how to linearize 2D or 3D spatial information into 1 -dimensional path information.
  • the spatial order of the images is ordered by the time an event was detected by the camera associated with each image. As described above, the time of an event is recorded and saved.
  • FIG. 9 there is a camera associated with the building's exterior (FIG.
  • a 2D floor map or a 3D spatial relation can be reduced into a number of paths, from point A to point B, via other points. Without constraints, the number of path can easily be large or even unlimited. However, an approach according to an embodiment of the invention can allow an operator to define the most
  • Event instances may be represented by symbols, such as image thumbnails, displayed based on their associated time and space.
  • the links among events illustrate a path or activity trail of the person or object, moving across the space at different times.
  • An activity trails visualization can support users in discovering the relationships among different events and objects.
  • FIG. 10 illustrates two persons' paths or activity trails 101 and 102, along with a scene picker 108 to control where to start viewing a video. From the visualization, one can see that two persons overlapped at the same time and space twice, therefore, there could be potential interactions between them.
  • a schematic video layout according to an embodiment of the disclosure could include as many as 50 or 60 video images, streaming video information from a corresponding number of video cameras.
  • the number of cameras in the layout may still obscure the events from an operator attempting to follow them.
  • a subset of video images that include the video image where the event was detected and video images from video cameras proximal to the camera that detected to event can be separately displayed in another layout on another video monitor.
  • FIGS. l l(a)-(b) illustrates monitoring and tracking for situation awareness, according to an embodiment of the invention.
  • FIG. 11(a) depicts a schematic video layout in which 4 events 111, 112, 113, and 114 have been detected
  • FIG. 11(b) shows the video images for the most recent detected event 114 along with video images from nearby cameras, enlarged to show more detail.
  • FIG. 12 illustrates an aggregate view of the activity trails according to embodiments of the disclosure, illustrating the summary of all events that happened during a certain time period.
  • An aggregated event view is a summary or overview of all events with the path information collapsed.
  • the present invention can be implemented in various forms of hardware, software, firmware, special purpose processes, or a combination thereof.
  • the present invention can be implemented in software as an application program tangible embodied on a computer readable program storage device.
  • the application program can be uploaded to, and executed by, a machine comprising any suitable architecture.
  • FIG. 14 is a block diagram of an exemplary computer system for implementing a method and user interface (UI) for effective video surveillance, according to an embodiment of the disclosure.
  • a computer system 141 for implementing the present invention can comprise, inter alia, a central processing unit (CPU) 142, a memory 143 and an input/output (I/O) interface 144.
  • the computer system 141 is generally coupled through the I/O interface 144 to a display 145 and various input devices 146 such as a mouse and a keyboard.
  • the support circuits can include circuits such as cache, power supplies, clock circuits, and a communication bus.
  • the memory 143 can include random access memory (RAM), read only memory (ROM), disk drive, tape drive, etc., or a combinations thereof.
  • RAM random access memory
  • ROM read only memory
  • the present invention can be implemented as a routine 147 that is stored in memory 143 and executed by the CPU 142 to process the signal from the signal source 148.
  • the computer system 141 is a general purpose computer system that becomes a specific purpose computer system when executing the routine 147 of the present invention.
  • the computer system 141 also includes an operating system and micro instruction code.
  • the various processes and functions described herein can either be part of the micro instruction code or part of the application program (or combination thereof) which is executed via the operating system.
  • various other peripheral devices can be connected to the computer platform such as an additional data storage device and a printing device.

Abstract

A method for creating a video layout for video surveillance of a facility includes defining (31) pathway information for a facility, locating (33) camera positions of a plurality of video surveillance cameras in a floor map of said facility, overlaying (34) the pathway information on the floor map, locating (35) target path elements in the overlaid floor map, optimizing (36) the overlaid floor map to simplify all paths into one of a horizontal, vertical, or diagonal line, and generating (37) a schematic video layout from the optimized overlaid floor map facility, said schematic video layout comprising a plurality of video images from the plurality of video surveillance cameras in said facility, wherein said schematic video layout represents spatial and geographic relationships among the plurality of video surveillance cameras.

Description

USER INTERFACE FOR EFFECTIVE VIDEO SURVEILLANCE
Cross Reference to Related United States Applications
This application claims priority from "Method and User Interface for Effective Video Surveillance", U.S. Provisional Application No. 61/821,404 of Zheng, et al., filed on May 9, 2013, the contents of which are herein incorporated by reference in their entirety.
Technical Field
Embodiments of the present disclosure are directed to a method and user interface (UI) for effective video surveillance.
Discussion of the Related Art
The number of installed video surveillance cameras has exploded in the past ten years due to the pressing need to enhance security and safety worldwide. In the United Kingdom (UK) alone, as of 2006, it is estimated that there were more than 4.2 million surveillance cameras installed just in London.
Acquiring and storing video data is only the first step for ensuring safety and security. Most existing video surveillance systems fail to consider the need for presenting the information in a more effective way to human operators. As a result, the amount of data acquired and stored by visual surveillance devices far exceeds an operator's capacity to monitor, understand, and search. This represents a fundamental bottleneck in the security and safety infrastructure, and has prevented video surveillance technology from reaching its true potential.
Summary
Exemplary embodiments of the disclosure as described herein generally include methods and user interfaces (UI) for effective video surveillance that include three characteristics that distinguish them from existing conventional video surveillance system.
These characteristics include: (1) A schematic video layout to support operators' mental model and monitoring, in contrast to a conventional grid video layout; (2) Adaptive video resizing based on notifications to catch an operators' attention; and (3) Combining the spatial information into the timeline to create an activity trail to support operators' discovery. Embodiments of the disclosure are based on a thorough analysis of an operator's mental model and the workflow of video surveillance, and are built upon the systematic framework of What, Where, When, and Who.
According to an embodiment of the invention, there is provided a method for creating a video layout for video surveillance of a facility, including defining pathway information for a facility, locating camera positions of a plurality of video surveillance cameras in a floor map of said facility, overlaying the pathway information on the floor map, locating target path elements in the overlaid floor map, optimizing the overlaid floor map to simplify all paths into one of a horizontal, vertical, or diagonal line, and generating a schematic video layout from the optimized overlaid floor map facility, said schematic video layout comprising a plurality of video images from the plurality of video surveillance cameras in said facility, wherein said schematic video layout represents spatial and geographic relationships among the plurality of video surveillance cameras.
According to a further embodiment of the invention, the method includes extracting topological information from the pathway information on the floor map, including space and corridor information through which people can move, and physical constrains or blocks through which people cannot go.
According to a further embodiment of the invention, target path elements are points of interests, including specific entrances, rooms, and spaces.
According to a further embodiment of the invention, optimization comprises linear fitting.
According to a further embodiment of the invention, optimization includes setting thresholds and tolerance levels for an acceptable level of error for a deviation angle between an angle of a schematic path and the angle of an actual geographical path.
According to a further embodiment of the invention, pathway information is defined from a building information model. According to a further embodiment of the invention, pathway information is defined manually.
According to another embodiment of the invention, there is provided a method of performing video surveillance of a facility, including providing a schematic video layout of a facility, said schematic video layout comprising a plurality of video images from a plurality of video surveillance cameras in said facility, wherein said schematic video layout represents spatial and geographic relationships among the plurality of video surveillance cameras, enlarging a size of a video image for a video camera in said schematic video layout when an event is detected for said camera, presenting event type information next to said enlarged video image, and creating a 2-dimensional space-time trail from successive enlarged images to present the integrated temporal and spatial information.
According to a further embodiment of the invention, the event type information includes a thumbnail of a detected object, and an action button.
According to a further embodiment of the invention, creating a 2-dimensional space- time trail from successive enlarged images includes linearizing the spatial information using the schematic video layout to transform 2-dimensional or 3-dimensional spatial information into 1 -dimensional path information, and creating a 2-dimensional space-time trail by combining the path information with a timeline of detected events, wherein one axis is a spatial axis and a second axis is a time axis, wherein a detected event is represented by a symbol located at a space-time coordinate in said 2-dimensional space-time trail that corresponds to a location and time of said detected event.
According to a further embodiment of the invention, linearizing the spatial information includes receiving one or more user defined paths extracted from the floor map and schematic video layout, inferring a most significant path from the one or more user defined paths by conducting a graph analysis of said one or more user defined paths, and tracking path usage of said most significant path.
According to a further embodiment of the invention, when an event is detected for a camera, the method includes separately displaying an image from the camera and for one or more cameras proximal to the camera in a schematic video layout on another video monitor. According to a another embodiment of the invention, there is provided a non- transitory program storage device readable by a computer, tangibly embodying a program of instructions executed by the computer to perform the method steps for creating a video layout for video surveillance of a facility.
Brief Description of the Drawings
FIG. 1 illustrates an example of a conventional grid video layout, in which the cameras are displayed in an almost random fashion in a 3 by 3 matrix, according to an embodiment of the disclosure.
FIG. 2 shows an example of mapping cameras directly on a map, according to an embodiment of the disclosure.
FIG. 3 is a flowchart of a method for generating schematic video layout, according to an embodiment of the disclosure.
FIG. 4 illustrates a schematic video layout example, showing the relationships among the various cameras, according to an embodiment of the disclosure.
FIG. 5 illustrates an adaptive resizing in which the size of the video becomes bigger as the alert is triggered, according to an embodiment of the disclosure.
FIGS. 6(a)-(b) illustrates how an alerts' information is attached along the side of the camera view, providing the contextual information showing where the events occurred, according to an embodiment of the disclosure.
FIGS. 7(a)-(b) illustrates an alert tray displayed next to the camera view, according to an embodiment of the disclosure.
FIG. 8 illustrates an example of an activity trail for integrating both temporal and spatial information of events, according to an embodiment of the disclosure.
FIG. 9 illustrates an example of how to linearize 2D or 3D spatial information into 1- dimensional path information, according to an embodiment of the disclosure. FIG. 10 illustrates two persons' paths or activity trails, according to an embodiment of the disclosure.
FIGS. 1 l(a)-(b) illustrates monitoring and tracking for situation awareness, according to an embodiment of the invention.
FIG. 12 illustrates an aggregate view of the activity trails according to embodiments of the disclosure.
FIG. 13 is a flowchart of a method for producing an activity trail, according to an embodiment of the disclosure.
FIG. 14 is a block diagram of an exemplary computer system for implementing a method and user interface (UI) for effective video surveillance, according to an embodiment of the disclosure.
Detailed Description of Exemplary Embodiments
Exemplary embodiments of the disclosure as described herein generally include methods and user interfaces (UI) for effective video surveillance. Accordingly, while the disclosure is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that there is no intent to limit the disclosure to the particular forms disclosed, but on the contrary, the disclosure is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the disclosure.
As used herein, the term "image" refers to multi-dimensional data composed of discrete image elements (e.g., pixels for 2-D images and voxels for 3-D images). The image may be, for example, a medical image of a subject collected by computer tomography, magnetic resonance imaging, ultrasound, or any other medical imaging system known to one of skill in the art. The image may also be provided from non-medical contexts, such as, for example, remote sensing systems or video surveillance systems, etc. Although an image can be thought of as a function from R3 to R, the methods of the inventions are not limited to such images, and can be applied to images of any dimension, e.g. a 2-D picture or a 3-D volume. For a 2- or 3-dimensional image, the domain of the image is typically a 2- or 3-dimensional rectangular array, wherein each pixel or voxel can be addressed with reference to a set of 2 or 3 mutually orthogonal axes. The terms "digital" and "digitized" as used herein will refer to images or volumes, as appropriate, in a digital or digitized format acquired via a digital acquisition system or via conversion from an analog image.
The video matrix "Grid Layout" is the prevailing layout used in existing video surveillance or CCTV (closed-circuit television) systems. FIG. 1 illustrates an example of a conventional grid video layout, in which the cameras (seven in FIG. 1) are displayed in an almost random fashion in a 3 by 3 matrix. This idea probably can be traced back to the previous technology of a physical video wall, where one TV monitor is connected to one video camera. However, this grid layout presentation, with cameras almost randomly placed in the matrix, does not support operators' mental model for the actual physical space. For example, information regarding the spatial relationships among the different cameras is lost. The operator needs to memorize which camera maps to which specific location in the actual physical space. Linking and connecting information across different cameras becomes a mental challenging task. Imagine a suspicious person is moving across different locations in the space; tracking that person can be a challenging task because the operator has to guess which cameras would capture that person next, and then try to locate the camera in the video matrix.
One potential solution to restore the spatial information and relationships of the cameras is to map the cameras directly onto a map. FIG. 2 shows an example of mapping cameras directly on a map. As can be seen from FIG. 2, this direct overlay approach can easily generate visual clutter because the map already contains a lot of data that are not necessarily relevant for video surveillance. In addition, because the video thumbnails are directly mapped to their geographical locations, the small video size does not show much information to the operator, yet a large video size will cover the map. Furthermore, the lack of visual alignment among the video thumb nails can also contribute to the visual clutter.
A schematic video layout according to an embodiment of the disclosure can represent the most important information of the video cameras to the operators, namely, namely the relationships among different video cameras rather than capturing the accurate geographical information. As a result, it is easy for operators to look for the information they need: which location is being viewed, what is the next location, and where to make a transition to another location, etc. Research in psychology shows that people's the mental representation of a space, i.e., the cognitive map, is schematic rather than an accurate representation of the physical world. FIG. 4 illustrates a schematic video layout example for six cameras according to an embodiment of the disclosure, showing the relationships among the various cameras. Referring now to the figure, a thumbnail 40 of the floor map with overlaid camera IDs is displayed in the upper right corner, and the video images 41, 42, 43, 44, 45 and 46 from cameras 1 to 6, respectively, are displayed at positions in the layout that correspond to the locations in the thumbnail 40. FIG. 4 shows that camera 1 is above camera 2, and there is a connection between them; then both camera 3 and 5 are also connected to camera 2. If a person is spotted in camera 2, then there are three possible cameras, i.e., 1, 3, 5, for the person to show up next. If a person is spotted in camera 5, then both camera 2 and 6 may be the next likely cameras to see the person.
A flowchart of a method for generating schematic video layout of a plurality of video surveillance cameras of a building or facility is presented in FIG. 3. According to embodiments of the invention, layout information, such as a floor map or a schematic video layout, is represented by a graph, which is an ordered pair G = (N, E) comprising a set of nodes N together with a set E of edges or lines. Referring now to the figure, a method starts at step 31 by defining pathway information in the building or facility using a building information model (BIM), which is a digital representation of the physical and functional characteristics of a building that can be exchanged or networked, and can be used as a shared knowledge resource for making decisions during the life-cycle of a building. The international standard implementations of BIM are Industry Foundation Classes (IFCs) and aecXML. IFCs are data models that describe building and construction industry data, and aecXML is the is a specific XML mark-up language which is based on an IFC to create a vendor-neutral means to access data generated by BIM. Within IFCs, pathways related information, such as corridors and doors, is defined in the Architecture Domain, and this information can be automatically extracted by a XML parser following the aecXML definitions. If no BIM is available, pathway information can be defined manually. Camera positions of the video surveillance cameras are located in a floor map of the building at step 33, which may be any type of digital floor plan, such as a CAD drawing. The camera locations can be marked directly on the floor map, represented by camera icons, and their directions represented by, e.g., fanout shapes. At step 34, the pathway information is overlaid on the floor map, and topological information is extracted. The topological information relates to space and corridor information where people can move through, as well as the physical constrains or blocks where people cannot go through. According to embodiments of the invention, topological information may be automatically extracted by extracting the nodes and connection information directly from the BIM model, or manually by having a user define nodes, i.e. areas of interests, and their connections, i.e. edges, based on the provided floor map. At step 35, target path elements, which are points of interests, such as a specific entrance, or a room, or any space, are located in the overlaid floor map. Target path elements include both nodes, the area of interests, and their connections, represented as edges, based on the floor map. Optimization, such as linear fitting, can be applied at step 36, and the user may adjust a threshold or tolerance level. Thresholds and tolerance levels refer to the level of error acceptable during the optimization. An optimization according to an embodiment of the invention abstracts and simplifies all the paths into either horizontal, vertical, or diagonal (45 degree angle) lines. More formally, the deviation angle between the angle of the schematic path (Ps) and the angle of actual geographical path (Pg) can be calculated, and the optimization math function can be defined as: Ps - Pg <= threshold. An exemplary, non- limiting threshold value can be from 0 to 90 degrees depending on the tolerance levels. Finally, a schematic video layout is generated from the optimized overlaid floor map at step 37. An optimization algorithm according to an embodiment of the invention can reduce the visual complexity of the graph by straightening the lines or connections to make them, for example, either horizontal, vertical or diagonal.
A schematic video layout according to an embodiment of the disclosure, therefore, can effectively support video surveillance operators in linking and connecting different videos together, to gain a better situation awareness of the environment. For example, because there are often many videos that an operator needs to monitor, and the operator can also be distracted by other tasks, it is useful to catch an operator's attention when some event does happen. Events, such as a person is detected, or a person is entering a room, are detected in real-time by video content analysis modules. Video content analysis makes use of technologies disclosed in copending applications numbers 13/646867, "Method and User Interface for Forensic Video Search" filed on October 8, 2012, and 13/693,231, "Configuration Tool for Video Analytics" filed on December 4, 2012, the contents of both of which are herein incorporated by reference in their entireties. For every event detected, the time and camera location information are recorded, both of which will be mapped to the activity trail, as will be described below. According to an embodiment of the disclosure, when an event is detected in a camera, adaptive resizing can cause the size of the camera video to be enlarged and highlighted to catch operators' attention. FIG. 5 illustrates an adaptive resizing according to an embodiment of the disclosure, in which the size of the video becomes bigger as the alert is triggered. As shown in FIG. 5, events were detected by cameras 2 and 4, as indicated by reference numbers 52 and 54, and the corresponding video images were enlarged. In addition, the event type or category information can be presented next to the video. A system according to an embodiment of the invention can provide a vocabulary and grammar to allow a facility operator to define an unlimited number of event types or categories. The grammar, in general, is "Who" does "What", or "What", for example, "Enter the lobby", "Exit the building", "Enter the mailroom", and "Who" can be a person, or a couple of persons, or even an object, like a car.
FIGS. 6(a)-(b) illustrate how an alerts' information is attached along the side of the camera view for camera 4, providing the contextual information showing where the events occurred. For comparison, FIG. 6(a) shows the view from camera 4 before the event was detected, as shown in FIG. 4, and FIG. 6(b) shows the view from camera 4 after the event was detected, as shown in FIG. 5. In addition, a quick action button can be displayed, supporting the quick actions for the operator. FIGS. 7(a)-(b) illustrates an alert tray displayed next to the camera view according to an embodiment of the disclosure, which contains the gist information for the event, e.g., showing a thumbnail 71 for the detected person or object, in FIG. 7(a). In addition, as shown in FIG. 7(b), action items 72, 73 may also be displayed, allowing operators' quick actions. The eyeball 72 activates a tracking function, in which the system will automatically monitor and collect information about the object or person of interests. The magnifying glass 72 activates a forensic search, which invokes another interface where the operator can look for more information regarding this object or person.
Events happening in a building have both temporal (e.g., when did the event happen?) and spatial information (e.g., where did the event happen?). To integrate multiple small events into a bigger picture for a holistic understanding of what has been going on, it is useful to organize and synthesize multiple individual events in both a temporal and spatial fashion.
However, because spatial information is typically represented as a 2-dimensional or 3- dimensional map, and time is typically represented as a 1 -dimensional timeline, a method of simply combining a spatial map with a timeline will be visually complex. According to an embodiment of the disclosure, an activity trail is a 2-dimensional space-time trail that can effectively present the integrated temporal and spatial information. FIG. 8 illustrates an example of an activity trail for integrating both temporal and spatial information of events, according to an embodiment of the disclosure. Referring now to FIG. 8, different spatial locations are identified in the highlighted column 80a on the left side, and a time axis 80b in 24 hour units on the top side of the figure. Events are represented by symbols such as the thumbnails 81, 82, 83, 84, 85, and 86, positioned at coordinates corresponding to the spatial location and time of the detected event. Needle 88 is a scene picker for controlling where to start viewing a video of the detected events.
A flow chart of a method for producing an activity trail according to an embodiment of the disclosure is presented in FIG. 13. Referring now to the figure, a first step 131 in producing an activity trail according to an embodiment of the disclosure is to linearize the spatial information, which was recorded when the event occurred, transforming the 2D or 3D spatial information into ID, e.g., path information. FIG. 9 illustrates an example of how to linearize 2D or 3D spatial information into 1 -dimensional path information. Referring now to FIG. 9, the spatial order of the images is ordered by the time an event was detected by the camera associated with each image. As described above, the time of an event is recorded and saved. In FIG. 9, there is a camera associated with the building's exterior (FIG. 9(a)), two cameras associated with the building's lobby (FIGS. 9(b)-(c)), a camera associated with the corridor (FIG. These locations are 9(d)), a camera associated with office area A (FIG. 9(e)), and a camera associated with office area B (FIG. 9(f)). These locations can be arbitrarily scattered about the building.
According to embodiments of the disclosure, a 2D floor map or a 3D spatial relation can be reduced into a number of paths, from point A to point B, via other points. Without constraints, the number of path can easily be large or even unlimited. However, an approach according to an embodiment of the invention can allow an operator to define the most
"significant or interesting paths", conduct graph or network analysis to infer the most
"important" paths, and keep track of the actual usage of the paths and to update their priority accordingly. Using a 2D schematic layout map according to an embodiment of the disclosure, operators can define any number of significant or interesting paths based on their experience. For instance, an operator can say that the path from the lobby entrance to the first floor zone A is particular interesting because that is the typical path for people to come in to work. What is generated is path information, which is extracted from the floor map and schematic video layout. The path information can be used for the graph or network analysis and for tracking path usage. Then, at step 132, the linearized spatial information will be combined with a timeline, which represents the time. FIG. 8 shows one example, where the timeline is visualized horizontally as the x-axis, and the linearized spatial information is shown vertically as the y-axis. Event instances may be represented by symbols, such as image thumbnails, displayed based on their associated time and space. The links among events illustrate a path or activity trail of the person or object, moving across the space at different times.
An activity trails visualization according to an embodiment of the disclosure can support users in discovering the relationships among different events and objects. For example, FIG. 10 illustrates two persons' paths or activity trails 101 and 102, along with a scene picker 108 to control where to start viewing a video. From the visualization, one can see that two persons overlapped at the same time and space twice, therefore, there could be potential interactions between them.
In a typical real situation, a schematic video layout according to an embodiment of the disclosure could include as many as 50 or 60 video images, streaming video information from a corresponding number of video cameras. In such a situation, even after an event has been detected and the corresponding video images have been adaptively resized, the number of cameras in the layout may still obscure the events from an operator attempting to follow them. In this case, according to an embodiment of the invention, a subset of video images that include the video image where the event was detected and video images from video cameras proximal to the camera that detected to event can be separately displayed in another layout on another video monitor. FIGS. l l(a)-(b) illustrates monitoring and tracking for situation awareness, according to an embodiment of the invention. FIG. 11(a) depicts a schematic video layout in which 4 events 111, 112, 113, and 114 have been detected, and FIG. 11(b) shows the video images for the most recent detected event 114 along with video images from nearby cameras, enlarged to show more detail. FIG. 12 illustrates an aggregate view of the activity trails according to embodiments of the disclosure, illustrating the summary of all events that happened during a certain time period. An aggregated event view is a summary or overview of all events with the path information collapsed.
It is to be understood that the present invention can be implemented in various forms of hardware, software, firmware, special purpose processes, or a combination thereof. In one embodiment, the present invention can be implemented in software as an application program tangible embodied on a computer readable program storage device. The application program can be uploaded to, and executed by, a machine comprising any suitable architecture.
FIG. 14 is a block diagram of an exemplary computer system for implementing a method and user interface (UI) for effective video surveillance, according to an embodiment of the disclosure. Referring now to FIG. 14, a computer system 141 for implementing the present invention can comprise, inter alia, a central processing unit (CPU) 142, a memory 143 and an input/output (I/O) interface 144. The computer system 141 is generally coupled through the I/O interface 144 to a display 145 and various input devices 146 such as a mouse and a keyboard. The support circuits can include circuits such as cache, power supplies, clock circuits, and a communication bus. The memory 143 can include random access memory (RAM), read only memory (ROM), disk drive, tape drive, etc., or a combinations thereof. The present invention can be implemented as a routine 147 that is stored in memory 143 and executed by the CPU 142 to process the signal from the signal source 148. As such, the computer system 141 is a general purpose computer system that becomes a specific purpose computer system when executing the routine 147 of the present invention.
The computer system 141 also includes an operating system and micro instruction code. The various processes and functions described herein can either be part of the micro instruction code or part of the application program (or combination thereof) which is executed via the operating system. In addition, various other peripheral devices can be connected to the computer platform such as an additional data storage device and a printing device.
It is to be further understood that, because some of the constituent system components and method steps depicted in the accompanying figures can be implemented in software, the actual connections between the systems components (or the process steps) may differ depending upon the manner in which the present invention is programmed. Given the teachings of the present invention provided herein, one of ordinary skill in the related art will be able to contemplate these and similar implementations or configurations of the present invention.
While the present invention has been described in detail with reference to exemplary embodiments, those skilled in the art will appreciate that various modifications and substitutions can be made thereto without departing from the spirit and scope of the invention as set forth in the appended claims.

Claims

CLAIMS What is claimed is:
1. A method for creating a video layout for video surveillance of a facility, comprising the steps of:
defining pathway information for a facility;
locating camera positions of a plurality of video surveillance cameras in a floor map of said facility;
overlaying the pathway information on the floor map;
locating target path elements in the overlaid floor map;
optimizing the overlaid floor map to simplify all paths into one of a horizontal, vertical, or diagonal line; and
generating a schematic video layout from the optimized overlaid floor map facility, said schematic video layout comprising a plurality of video images from the plurality of video surveillance cameras in said facility, wherein said schematic video layout represents spatial and geographic relationships among the plurality of video surveillance cameras.
2. The method of claim 1, further comprising extracting topological information from the pathway information on the floor map, including space and corridor information through which people can move, and physical constrains or blocks through which people cannot go.
3. The method of claim 1 , wherein target path elements are points of interests, including specific entrances, rooms, and spaces.
4. The method of claim 1 , wherein optimization comprises linear fitting.
5. The method of claim 1, wherein optimization includes setting thresholds and tolerance levels for an acceptable level of error for a deviation angle between an angle of a schematic path and the angle of an actual geographical path.
6. The method of claim 1 , wherein pathway information is defined from a building information model.
7. The method of claim 1 , wherein pathway information is defined manually.
8. A method of performing video surveillance of a facility, comprising the steps of:
providing a schematic video layout of a facility, said schematic video layout comprising a plurality of video images from a plurality of video surveillance cameras in said facility, wherein said schematic video layout represents spatial and geographic relationships among the plurality of video surveillance cameras;
enlarging a size of a video image for a video camera in said schematic video layout when an event is detected for said camera;
presenting event type information next to said enlarged video image; and
creating a 2-dimensional space-time trail from successive enlarged images to present the integrated temporal and spatial information.
9. The method of claim 8, wherein said event type information includes a thumbnail of a detected object, and an action button.
10. The method of claim 8, wherein creating a 2-dimensional space-time trail from successive enlarged images comprises:
linearizing the spatial information using the schematic video layout to transform 2- dimensional or 3-dimensional spatial information into 1 -dimensional path information; and creating a 2-dimensional space-time trail by combining the path information with a timeline of detected events, wherein one axis is a spatial axis and a second axis is a time axis, wherein a detected event is represented by a symbol located at a space-time coordinate in said 2-dimensional space-time trail that corresponds to a location and time of said detected event
11. The method of claim 10, wherein linearizing the spatial information comprises:
receiving one or more user defined paths extracted from the floor map and schematic video layout;
inferring a most significant path from the one or more user defined paths by conducting a graph analysis of said one or more user defined paths; and
tracking path usage of said most significant path.
12. The method of claim 8, further comprising, when an event is detected for a camera, separately displaying an image from said camera and for one or more cameras proximal to said camera in a schematic video layout on another video monitor.
13. A non-transitory program storage device readable by a computer, tangibly embodying a program of instructions executed by the computer to perform the method steps for creating a video layout for video surveillance of a facility, the method comprising the steps of:
defining pathway information for a facility;
locating camera positions of a plurality of video surveillance cameras in a floor map of said facility;
overlaying the pathway information on the floor map;
locating target path elements in the overlaid floor map;
optimizing the overlaid floor map to simplify all paths into one of a horizontal, vertical, or diagonal line; and
generating a schematic video layout from the optimized overlaid floor map facility, said schematic video layout comprising a plurality of video images from the plurality of video surveillance cameras in said facility, wherein said schematic video layout represents spatial and geographic relationships among the plurality of video surveillance cameras.
14. The computer readable program storage device of claim 13, the method further comprising extracting topological information from the pathway information on the floor map, including space and corridor information through which people can move, and physical constrains or blocks through which people cannot go.
15. The computer readable program storage device of claim 13, wherein target path elements are points of interests, including specific entrances, rooms, and spaces.
16. The computer readable program storage device of claim 13, wherein optimization comprises linear fitting.
17. The computer readable program storage device of claim 13, wherein optimization includes setting thresholds and tolerance levels for an acceptable level of error for a deviation angle between an angle of a schematic path and the angle of an actual geographical path.
18. The computer readable program storage device of claim 13, wherein pathway information is defined from a building information model.
19. The computer readable program storage device of claim 13, wherein pathway information is defined manually.
PCT/US2014/037297 2013-05-09 2014-05-08 User interface for effective video surveillance WO2014182898A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201361821404P 2013-05-09 2013-05-09
US61/821,404 2013-05-09

Publications (1)

Publication Number Publication Date
WO2014182898A1 true WO2014182898A1 (en) 2014-11-13

Family

ID=50977078

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2014/037297 WO2014182898A1 (en) 2013-05-09 2014-05-08 User interface for effective video surveillance

Country Status (1)

Country Link
WO (1) WO2014182898A1 (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105279331A (en) * 2015-10-23 2016-01-27 国家电网公司 Substation site monitoring camera constructing, deploying and modeling method
CN109522580A (en) * 2018-03-20 2019-03-26 广东电网有限责任公司清远供电局 A kind of monitoring camera points distributing method based on substation's region significance level
CN109996032A (en) * 2017-12-29 2019-07-09 杭州海康威视系统技术有限公司 Information display method and device, computer equipment and storage medium
CN110602438A (en) * 2018-06-13 2019-12-20 浙江宇视科技有限公司 Road network-based video monitoring layout optimization method and device
EP3616095A4 (en) * 2017-04-28 2020-12-30 Cherry Labs, Inc. Computer vision based monitoring system and method
CN112804485A (en) * 2020-12-30 2021-05-14 济南和普威视光电技术有限公司 Method and system for monitoring moving target in video and security monitoring method and system
CN112860946A (en) * 2021-01-18 2021-05-28 四川弘和通讯有限公司 Method and system for converting video image information into geographic information
US11120559B2 (en) 2017-04-28 2021-09-14 Cherry Labs, Inc. Computer vision based monitoring system and method
US20230267727A1 (en) * 2020-07-20 2023-08-24 Nec Corporation Image analysis apparatus, image analysis method, and storage medium
US11809675B2 (en) 2022-03-18 2023-11-07 Carrier Corporation User interface navigation method for event-related video

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050225634A1 (en) * 2004-04-05 2005-10-13 Sam Brunetti Closed circuit TV security system
WO2005120071A2 (en) * 2004-06-01 2005-12-15 L-3 Communications Corporation Method and system for performing video flashlight
WO2006012645A2 (en) * 2004-07-28 2006-02-02 Sarnoff Corporation Method and apparatus for total situational awareness and monitoring
US20070257986A1 (en) * 2006-05-05 2007-11-08 Ivanov Yuri A Method for processing queries for surveillance tasks
US20080074494A1 (en) * 2006-09-26 2008-03-27 Harris Corporation Video Surveillance System Providing Tracking of a Moving Object in a Geospatial Model and Related Methods
US20090010493A1 (en) * 2007-07-03 2009-01-08 Pivotal Vision, Llc Motion-Validating Remote Monitoring System
US20090134968A1 (en) * 2007-11-28 2009-05-28 Fuji Xerox Co., Ltd. Segmenting time based on the geographic distribution of activity in sensor data
WO2010115186A1 (en) * 2009-04-03 2010-10-07 Siemens Corporation First responder decision support system based on building information model (bim)
EP2325820A1 (en) * 2009-11-24 2011-05-25 Nederlandse Organisatie voor toegepast -natuurwetenschappelijk onderzoek TNO System for displaying surveillance images

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050225634A1 (en) * 2004-04-05 2005-10-13 Sam Brunetti Closed circuit TV security system
WO2005120071A2 (en) * 2004-06-01 2005-12-15 L-3 Communications Corporation Method and system for performing video flashlight
WO2006012645A2 (en) * 2004-07-28 2006-02-02 Sarnoff Corporation Method and apparatus for total situational awareness and monitoring
US20070257986A1 (en) * 2006-05-05 2007-11-08 Ivanov Yuri A Method for processing queries for surveillance tasks
US20080074494A1 (en) * 2006-09-26 2008-03-27 Harris Corporation Video Surveillance System Providing Tracking of a Moving Object in a Geospatial Model and Related Methods
US20090010493A1 (en) * 2007-07-03 2009-01-08 Pivotal Vision, Llc Motion-Validating Remote Monitoring System
US20090134968A1 (en) * 2007-11-28 2009-05-28 Fuji Xerox Co., Ltd. Segmenting time based on the geographic distribution of activity in sensor data
WO2010115186A1 (en) * 2009-04-03 2010-10-07 Siemens Corporation First responder decision support system based on building information model (bim)
EP2325820A1 (en) * 2009-11-24 2011-05-25 Nederlandse Organisatie voor toegepast -natuurwetenschappelijk onderzoek TNO System for displaying surveillance images

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
THANG V PHAM ET AL: "A Multi-Camera Visual Surveillance System for Tracking of Reoccurrences of People", DISTRIBUTED SMART CAMERAS, 2007. ICDSC '07. FIRST ACM/IEEE INTERN ATIONAL CONFERENCE ON, IEEE, PI, 1 September 2007 (2007-09-01), pages 164 - 169, XP031151278, ISBN: 978-1-4244-1353-9 *

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105279331A (en) * 2015-10-23 2016-01-27 国家电网公司 Substation site monitoring camera constructing, deploying and modeling method
EP3616095A4 (en) * 2017-04-28 2020-12-30 Cherry Labs, Inc. Computer vision based monitoring system and method
US11120559B2 (en) 2017-04-28 2021-09-14 Cherry Labs, Inc. Computer vision based monitoring system and method
CN109996032A (en) * 2017-12-29 2019-07-09 杭州海康威视系统技术有限公司 Information display method and device, computer equipment and storage medium
CN109996032B (en) * 2017-12-29 2020-10-02 杭州海康威视系统技术有限公司 Information display method and device, computer equipment and storage medium
CN109522580B (en) * 2018-03-20 2023-05-30 广东电网有限责任公司清远供电局 Monitoring camera point distribution method based on importance degree of transformer substation area
CN109522580A (en) * 2018-03-20 2019-03-26 广东电网有限责任公司清远供电局 A kind of monitoring camera points distributing method based on substation's region significance level
CN110602438A (en) * 2018-06-13 2019-12-20 浙江宇视科技有限公司 Road network-based video monitoring layout optimization method and device
US20230267727A1 (en) * 2020-07-20 2023-08-24 Nec Corporation Image analysis apparatus, image analysis method, and storage medium
CN112804485A (en) * 2020-12-30 2021-05-14 济南和普威视光电技术有限公司 Method and system for monitoring moving target in video and security monitoring method and system
CN112860946B (en) * 2021-01-18 2023-04-07 四川弘和通讯集团有限公司 Method and system for converting video image information into geographic information
CN112860946A (en) * 2021-01-18 2021-05-28 四川弘和通讯有限公司 Method and system for converting video image information into geographic information
US11809675B2 (en) 2022-03-18 2023-11-07 Carrier Corporation User interface navigation method for event-related video

Similar Documents

Publication Publication Date Title
WO2014182898A1 (en) User interface for effective video surveillance
JP6055823B2 (en) Surveillance camera control device and video surveillance system
US10810438B2 (en) Setting apparatus, output method, and non-transitory computer-readable storage medium
JP5227911B2 (en) Surveillance video retrieval device and surveillance system
Fan et al. Heterogeneous information fusion and visualization for a large-scale intelligent video surveillance system
JP6062039B2 (en) Image processing system and image processing program
US9602778B2 (en) Security video system using customer regions for monitoring point of sale areas
US20180075590A1 (en) Image processing system, image processing method, and program
US20140211019A1 (en) Video camera selection and object tracking
JP6182607B2 (en) Video surveillance system, surveillance device
JP2014236896A (en) Information processor, information processing method, and program
JP6310149B2 (en) Image generation apparatus, image generation system, and image generation method
Wang et al. Contextualized videos: Combining videos with environment models to support situational understanding
JP5525495B2 (en) Image monitoring apparatus, image monitoring method and program
US20190370537A1 (en) Keypoint detection to highlight subjects of interest
CN104704817A (en) Visual field display system, visual field display method, and visual field display program
JP6852293B2 (en) Image processing system, information processing device, information terminal, program
EP3264380B1 (en) System and method for immersive and collaborative video surveillance
CN115424320A (en) Method and system for displaying movement track of person in three-dimensional scene
KR101036107B1 (en) Emergency notification system using rfid
KR20140066560A (en) Method and apparatus for setting object area for use in video monitoring device
KR20190090960A (en) Apparatus and method for generating a heat map based on a region of interest
JP2018152688A (en) Video monitoring system, video monitoring method, and video monitoring program
US20240029542A1 (en) Systems and methods for providing security system information using augmented reality effects
JP6515975B2 (en) MOBILE MONITORING DEVICE, MOBILE MONITORING SYSTEM, PROGRAM, AND CONTROL METHOD

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14731445

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14731445

Country of ref document: EP

Kind code of ref document: A1