WO2012095867A2 - Système basé sur un serveur intelligent intégré et procédé/systèmes conçus pour faciliter l'intégration à sécurité intégrée et/ou l'utilisation optimisée de diverses entrées obtenues par capteur - Google Patents
Système basé sur un serveur intelligent intégré et procédé/systèmes conçus pour faciliter l'intégration à sécurité intégrée et/ou l'utilisation optimisée de diverses entrées obtenues par capteur Download PDFInfo
- Publication number
- WO2012095867A2 WO2012095867A2 PCT/IN2012/000029 IN2012000029W WO2012095867A2 WO 2012095867 A2 WO2012095867 A2 WO 2012095867A2 IN 2012000029 W IN2012000029 W IN 2012000029W WO 2012095867 A2 WO2012095867 A2 WO 2012095867A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- server
- colour
- video
- data
- intelligent
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 300
- 230000001953 sensory effect Effects 0.000 title claims abstract description 167
- 230000010354 integration Effects 0.000 title claims abstract description 14
- 238000003860 storage Methods 0.000 claims abstract description 144
- 238000004458 analytical method Methods 0.000 claims abstract description 96
- 238000012545 processing Methods 0.000 claims abstract description 94
- 238000001514 detection method Methods 0.000 claims abstract description 84
- 230000003044 adaptive effect Effects 0.000 claims abstract description 43
- 230000007246 mechanism Effects 0.000 claims abstract description 30
- 230000001427 coherent effect Effects 0.000 claims abstract description 25
- 238000013468 resource allocation Methods 0.000 claims abstract description 11
- 230000003068 static effect Effects 0.000 claims description 85
- 230000008569 process Effects 0.000 claims description 59
- 238000007726 management method Methods 0.000 claims description 55
- 230000033001 locomotion Effects 0.000 claims description 54
- 230000004313 glare Effects 0.000 claims description 41
- 230000000694 effects Effects 0.000 claims description 28
- 238000012546 transfer Methods 0.000 claims description 25
- 238000004891 communication Methods 0.000 claims description 23
- 230000005540 biological transmission Effects 0.000 claims description 21
- 239000000470 constituent Substances 0.000 claims description 21
- 239000013598 vector Substances 0.000 claims description 21
- 230000009466 transformation Effects 0.000 claims description 20
- 238000012550 audit Methods 0.000 claims description 16
- 239000002131 composite material Substances 0.000 claims description 16
- 238000012544 monitoring process Methods 0.000 claims description 15
- 230000002123 temporal effect Effects 0.000 claims description 14
- 239000000284 extract Substances 0.000 claims description 13
- 230000009471 action Effects 0.000 claims description 12
- 230000001419 dependent effect Effects 0.000 claims description 12
- 238000009826 distribution Methods 0.000 claims description 12
- 238000007906 compression Methods 0.000 claims description 11
- 230000006835 compression Effects 0.000 claims description 11
- 230000004075 alteration Effects 0.000 claims description 10
- 230000008859 change Effects 0.000 claims description 10
- 238000005259 measurement Methods 0.000 claims description 10
- 235000003642 hunger Nutrition 0.000 claims description 9
- 238000009877 rendering Methods 0.000 claims description 9
- 230000037351 starvation Effects 0.000 claims description 9
- 239000003086 colorant Substances 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 8
- 230000006870 function Effects 0.000 claims description 8
- 230000000670 limiting effect Effects 0.000 claims description 8
- 230000003542 behavioural effect Effects 0.000 claims description 7
- 230000015556 catabolic process Effects 0.000 claims description 7
- 230000015572 biosynthetic process Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000012015 optical character recognition Methods 0.000 claims description 6
- 230000000737 periodic effect Effects 0.000 claims description 6
- 238000012795 verification Methods 0.000 claims description 6
- 230000014759 maintenance of location Effects 0.000 claims description 5
- 238000013507 mapping Methods 0.000 claims description 5
- 238000010276 construction Methods 0.000 claims description 4
- 238000013144 data compression Methods 0.000 claims description 4
- 238000012217 deletion Methods 0.000 claims description 4
- 230000037430 deletion Effects 0.000 claims description 4
- 238000005304 joining Methods 0.000 claims description 4
- 238000011897 real-time detection Methods 0.000 claims description 4
- 230000009467 reduction Effects 0.000 claims description 4
- 230000011218 segmentation Effects 0.000 claims description 4
- 230000035945 sensitivity Effects 0.000 claims description 4
- 230000000007 visual effect Effects 0.000 claims description 4
- 241001522296 Erithacus rubecula Species 0.000 claims description 3
- 238000012512 characterization method Methods 0.000 claims description 3
- 238000012558 master data management Methods 0.000 claims description 3
- 230000000877 morphologic effect Effects 0.000 claims description 3
- 230000005856 abnormality Effects 0.000 claims description 2
- 230000000052 comparative effect Effects 0.000 claims description 2
- 238000011010 flushing procedure Methods 0.000 claims description 2
- 238000001429 visible spectrum Methods 0.000 claims description 2
- 206010052128 Glare Diseases 0.000 description 37
- 230000000875 corresponding effect Effects 0.000 description 20
- 238000013459 approach Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 238000000605 extraction Methods 0.000 description 10
- 238000005286 illumination Methods 0.000 description 9
- 230000008901 benefit Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 230000003993 interaction Effects 0.000 description 6
- 230000002829 reductive effect Effects 0.000 description 6
- 230000004807 localization Effects 0.000 description 5
- 230000002776 aggregation Effects 0.000 description 4
- 238000004220 aggregation Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000001276 controlling effect Effects 0.000 description 3
- 230000007423 decrease Effects 0.000 description 3
- 230000007613 environmental effect Effects 0.000 description 3
- 238000009434 installation Methods 0.000 description 3
- 230000006855 networking Effects 0.000 description 3
- 206010009691 Clubbing Diseases 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000004438 eyesight Effects 0.000 description 2
- 230000002452 interceptive effect Effects 0.000 description 2
- 239000000047 product Substances 0.000 description 2
- 238000004353 relayed correlation spectroscopy Methods 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 230000002311 subsequent effect Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000001052 transient effect Effects 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 230000003442 weekly effect Effects 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000005267 amalgamation Methods 0.000 description 1
- 230000033228 biological regulation Effects 0.000 description 1
- 239000006227 byproduct Substances 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 229910052754 neon Inorganic materials 0.000 description 1
- GKAOGPIIYCISHV-UHFFFAOYSA-N neon atom Chemical compound [Ne] GKAOGPIIYCISHV-UHFFFAOYSA-N 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000000704 physical effect Effects 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 230000000135 prohibitive effect Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 230000001932 seasonal effect Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 238000009827 uniform distribution Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/90—Determination of colour characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0654—Management of faults, events, alarms or notifications using network fault recovery
- H04L41/0663—Performing the actions predefined by failover planning, e.g. switching to standby network elements
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S3/00—Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received
- G01S3/78—Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received using electromagnetic waves other than radio waves
- G01S3/782—Systems for determining direction or deviation from predetermined direction
- G01S3/785—Systems for determining direction or deviation from predetermined direction using adjustment of orientation of directivity characteristics of a detector or detector system to give a desired condition of signal derived from that detector or detector system
- G01S3/786—Systems for determining direction or deviation from predetermined direction using adjustment of orientation of directivity characteristics of a detector or detector system to give a desired condition of signal derived from that detector or detector system the desired condition being maintained automatically
- G01S3/7864—T.V. type tracking systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/01—Detecting movement of traffic to be counted or controlled
- G08G1/017—Detecting movement of traffic to be counted or controlled identifying vehicles
- G08G1/0175—Detecting movement of traffic to be counted or controlled identifying vehicles by photographing vehicles, e.g. when violating traffic rules
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L47/00—Traffic control in data switching networks
- H04L47/10—Flow control; Congestion control
- H04L47/18—End to end
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1004—Server selection for load balancing
- H04L67/1008—Server selection for load balancing based on parameters of servers, e.g. available memory or workload
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
- H04L67/1034—Reaction to server failures by a load balancer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/40—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass for recovering from a failure of a protocol instance or entity, e.g. service redundancy protocols, protocol state redundancy or protocol service redirection
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
- H04N21/2365—Multiplexing of several video streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
Definitions
- TITLE AN INTEGRATED INTELLIGENT SERVER BASED SYSTEM AND METHOD/SYSTEMS ADAPTED TO FACILITATE FAIL-SAFE INTEGRATION AND /OR OPTIMIZED UTILIZATION OF VARIOUS SENSORY INPUTS
- the present invention is directed to a system architecture and, in particular, an integrated Intelligent Machine Understanding and Analysis framework to automatically manage a distributed networked multi-sensory data acquisition and analysis system to integrate with the normal business flow of an organization with or without minimal human intervention.
- the invention is directed to an integrated intelligent server based system having sensory input/data acquisition cum recording server group and /or analytics server group adapted to facilitate fail-safe integration and /or optimized utilization of various sensory inputs for various utility applications.
- the system of the invention can be deployed for various purposes including Security and Surveillance, Law enforcement, Automated traffic enforcement, Forensic evidence generation, Video data acquisition and analysis, and other machine intelligence and content understanding system.
- the architecture and underlying implementation is independent of any operating system and can work in multi-OS computing environment seamlessly under various resource constraints.
- the invention is also directed to a method for cost-effective and efficient bandwidth adaptive transferring /recording sensory data from single or multiple data sources to network accessible storage devices, a fail safe and self sufficient server group based method for sensory input recording and live streaming in a multi-server environment, an intelligent and unified method of colour coherent object analysis framework and method, a modified, computationally efficient method of face detection in video images and the like, a method of resource allocation for analytical processing involving multi-channel environment, a system for multichannel join-split mechanism adapted for low and /or variable bandwidth network link, a system for enhanced multi-colour and/or mono-colour object tracking and also an intelligent automated traffic enforcement system.
- Video Management Systems are used for video data acquisition and search processes using single or multiple servers. They are often loosely coupled with one or more separate systems for performing operations on the acquired video data such as analyzing the video content, etc.
- Servers can record different types of data in storage media, and the storage media can be directly attached to the servers or accessed over IP network. This demands a significant amount of network bandwidth to receive data from the sensors (e.g., Cameras) and to concurrently transfer or upload the data in the storage media. Due to high demand in bandwidth to perform such tasks, especially for video data, often separate high speed network are dedicated to transfer data to storage media. Dedicated high speed network is costly and often require costly storage devices as well. Often this is overkill for low or moderately priced installations.
- Video content analysis is often done per frame basis which is mostly pre defined which make such systems lacking in desired efficiency of analytics but are also unnecessarily cost extensive with unwanted loss of valuable computing resources.
- IVMS Intelligent Management System
- An object of the invention is directed to advancements in methods and/or systems enabling collection of sensory data from various images, video and other sensory sources, both on-line and off-line, archiving and indexing them to seamlessly map in any relational or networked database in a fail-safe way making optimal usage of computing, communication and storage resources, facilitate efficient search, transcoding, retransmission, authentication of data, rendering and viewing of archived data at any point of time.
- Another object of the invention is directed to advancements in method and/or system for more efficient and cost-effective streaming data real time or on Demand including streaming video and other sensory content in multiple formats to multiple devices for purposes like live view in different matrix layout, relay of the content, local archiving, rendering of the sensory data in multiple forms and formats, etc. by a fail-safe mechanism without affecting speed and performance of on-going operations and services.
- a further object of the present invention is directed to advancements in method and/or system adapted for intelligently analyzing the data, on-line or off-line, to extract the meaningful content of the data, identifying the activities of foreground human and other inanimate objects in the scene from the sensor generated data, establishing correlation among various objects (living or non-living and moving or static) in the scene, establishing correlation amongst multiple types of sensory data, identifying events of interests based on the detected activities, all either automatically or in an user interactive way under various demographic and natural real life situations.
- a further object of the present invention is directed to advancements in method and/or system adapted for generating alerts, signals, video clips, other sensory data segments, and covering the events more efficiently and automatically.
- Another object of the present invention is directed to advancements in method and/or system adapted for filtering and need based transmission of data at the right recipient at the right point of time automatically or on user interaction.
- Yet further object of the present invention is directed to advancements in method and/or system adapted for directed distribution of alerts including distributing Event information in various digital forms (SMS, MMS, emails, audio alerts, animation video, Text, illustrations, etc. but not limited to) with or without received data segments (viz, video clips) to the right recipient at the right point of time automatically or on user interaction.
- Another object of the present invention is directed to advancements in method and/or system adapted for providing a unified gateway for users to access systems for configuration, management and monitoring of system components.
- Yet further object of the present invention is directed to advancements in method and/or system adapted for enabling user to view camera captured video in different matrix layouts, view other sensory data in a presentable form, recorded video and other data search and replay, event clips search and replay, providing easy navigation across camera views with help of sitemaps, PTZ control, and configuring the system as per intended use.
- a further object of the present invention is directed to advancements in method and/or system adapted for intelligently sharing the computing resource, storage, rendering devices and communication bandwidth among different processes of the system to execute the above mentioned tasks with limited resources.
- Another object of the present invention is directed to advancements in method and/or system adapted for creating a green computing environment and enabling executing the above mentioned tasks by optimal usage of the computing, storage and communication devices and thereby saving energy and extending lifetime of the said resources.
- Yet another object of the present invention is directed to advancements in method and/or system adapted for providing distributed architecture support including providing a framework so that the system can be used in a centralized environment, or in a distributed architecture involving multiple computing, storage and communication devices or infrastructural facilities.
- a further object of the present invention is directed to advancements in method and/or system adapted for providing framework for media management in real life situations wherein the overall systems architecture could be distributed in nature with integration mechanism for continuous management of network congestion and automated load balancing of the all the computing and other resources in order to ensure that the system is not vulnerable to any single point failure to avoid data loss due to failure of any resource in the distributed networked environment
- Another object of the present invention is directed to advancements in method and/or system discussed above by interconnecting a number of intelligent components consisting of hardware and software, and involving implementation techniques adapted to make the system efficient, scalable, cost effective, fail-safe, adaptive to various demographic conditions, adaptive to various computing and communication infrastructure! facilities.
- an integrated intelligent server based system having sensory input/data acquisition cum recording server group and /or analytics server group adapted to facilitate fail-safe integration and /or optimised utilization of various sensory inputs for various utility applications
- at atleast one autonomous system having :
- said sensory input acquisition cum recording server group comprising plurality of acquisition cum recording servers which are operatively linked to assess respective server capacity and operate as a group to enable fail-safe support when any of the servers in the group fail to operate the remaining operative servers in the group are adapted to distribute and take over the sensory input load of the non-operative server/s to render the system fail safe and self sufficient ; and/or B) said analytics server group comprising plurality of analytics server for intelligent analysis including resource dependent analytical accuracy control including means adapted for computing complexity of scenes and dynamically reconfigure the analytical processing steps for optimal analysis and/or availability of computational and other resources for on-line and real-time and/or on demand for efficient and user friendly streaming/analysis/detection/alert generation of events and/or follow up actions; and
- each said acquisition cum recording servers are adapted for bandwidth optimized fail-safe recording and/or join- split mechanism for multi channel sensory data/video streaming.
- each said analytics server is adapted for anyone or more of (a) intelligent colour object analysis framework and colour coherent background estimation (b) identifying moving, static, quasi-static objects, (c ) enhanced object tracking, (d) content aware resource scheduling, (e) join split mechanism for multi channel video streaming, and (f) resource dependent accuracy control.
- said intelligent interface is operatively connected to anyone or more (a) user management and client access controller (b) event controller and handler and (c) event and/or selected segments of sensory data distributor.
- integrated intelligent server based system comprising operative client modules comprises selectively standalone surveillance client, internet browser, web client, any hand held devices including mobile device client, and remote event and/or notification receiver.
- said acquisition cum recording server is adapted to (i) collect inputs from various sensory sources, archiving, tagging, and indexing to seamfessiy map in a database or data warehousing system involving any one or more of optimal usage of computing, communication and storage resources, facilitate efficient search, transcoding, retransmission, authentication of data, rendering and viewing of archived data at any point of time, and (ii) Streaming input sensory data real time or on Demand including streaming video and other sensory content in multiple formats to multiple devices for purposes including live view in different matrix layout, relay of the content, local archiving, rendering of the sensory data in multiple forms and formats, by a fail-safe mechanism without affecting speed and performance of on-going operations and services.
- said intelligent interface is adapted for anyone or more of (i) filtering and need based transmission of sensory inputs, (ii) directing distribution of alerts, (iii) providing a common gateway for heterogeneous entities.
- said client module comprises means enabling user to receive, view, analyze, search sensory inputs and include standalone surveillance clients, internet browsers, handheld devices, cell phones,, PCs, Tablet PCs and the like.
- the above integrated intelligent server based system comprising remote event receiver adapted to receive and display messages and ALERTs from various components of the system which can further be multicast or broadcasted.
- the above integrated intelligent server based system comprising central server adapted to serve as a gateway to plurality of said autonomous system and integrate the system into a single unified system.
- each said acquisition cum recording server is adapted to accept requests through the intelligent interface and/or receive inputs from various other input sources, recording sensory inputs in local storage intelligent uploading of the sensory input in a cluster of storage devices wherein said cluster comprises one or more network accessible storages in an efficient manner with fair share to individual sources utilizing optimal bandwidth in a cooperative manner, enabling searching of input and analytical sensory inputs and streaming of the sensory inputs in original or transcoded format to various other devices including surveillance clients.
- the above integrated intelligent server based system comprising means for recording sensory inputs in local storage and intelligent streaming of stored inputs continuously or on trigger from any external or internal services wherein the data stream is first segmented into small granular clips or segments of programmable and variable length sizes and said clips stored in the said local storage of the server, the clip metadata being stored in the local database.
- said sensory input analytic engine comprises of (a) scene analyzer, (b) rule engine, and (c ) event decider .
- said scene analyzer comprises means for intelligent scene adaptive colour coherent object analysis framework and method adaptive to the availability of computational bandwidth and memory enabling processing steps to be dynamically reconfigured.
- said scene analyser comprises means to generate meta-data against each frame for analysis and computing the complexity of the scene such as to dynamically reconfigure the processing steps based thereon for optimal analysis results depending upon the availability of the computational and other resources for on-line and real-time detection of events and follow up actions and further feeding the metadata along with the scene complexity measure to a controller adapted to decide the rate at which the frames of said channel should be decoded and sent to the analytic engine for processing ; said rule engine adapted to maintain history of the metadata and correlate the data across multiple frames to thereby decide the behavioural patterns of the objects in the scene for further determinations; and said event decider is adapted to receive the behavioural patterns as detected by the rule engine and also analyze the same to thereby detect various events in parallel and also to control user defined application of any external device for better decision making/study of the event identified.
- said analytical engine controller comprises:
- said intelligent interface is adapted to (i) auto register itself to the system, (i ' i) accept request from surveillance clients and relay the same to corresponding recording server and analytic server, (iii) receive configuration data from the surveillance clients and feed to the intended components of the system, (iv) receive event information from analytic server on-line and transmit to various recipients including remote event receiver, fetch outstanding event clips from analytical engine controller, if any, (v) periodically receive heartbeat signals along with status information from all active devices and relay that to other devices in same or other networks,(vi) stream live video, recorded video or event alerts at appropriate time,(vii) join multiple channel sensory inputs into a single combined stream to adapt to variable and low bandwidth network,(viii) enable search based on various criteria Including data , time, event types, channels, signal features, and other system input and (ix) enable user to perform an user-interactive smart search to filter out desired segment of the sensory input from the database.
- said acquisition cum recording server group comprise plurality of sensory data recording server adapted to : record inputs from single /multiple data sources in atleast one local storage space with the URL of the files stored in database; transfer the thus stored files from said local storage to a network based central storage provided for accessing the files for end use/applications, said transfer of sensory data from source to the central storage via said local storage being carried out taking into consideration the data download speed (inflow rate) from data source to server along With the availability of network bandwidth at any given point of time for efficient network bandwidth sharing amongst multiple data sources to said storage device in the network.
- said sensory data recording server is adapted to monitor available total network bandwidth and per channel inflow rate and based thereon decide rate of per channel video transfer from the server local storage to said central storage.
- said sensory data from the source are recorded in the form of variable length clips wherein the clip duration is set by the user or set by the server itself.
- said sensory data recording server is adapted for determining the optimal bit rate for uploading sensory inputs involving :
- a server group is adapted to allocate any one of the operative servers in said group as the group master server and continuously monitor the servers in the group and their respective capacities and decide on the allocation and release of the input sensory source from any server within the Group.
- each said analytical server is adapted for multiple component colour object analysis in a scene favouring scene analytic applications comprising : multiple component colour coherent background estimation involving colour correlation of neighbouring pixels and inter-frame multiple component colour correlation using said multiple components as a composite data and using the relative values of these components to maintain accurate colour information and appearance of the true colour in the estimated background frame.
- said analytical server is adapted for efficient face detection in video images and the like by limiting the search space involving motion detection technique and controlled computational requirements based on desired accuracy by carrying out prediction of number iterations and temporal parameter "t".
- said analytical server for said face detection is adapted for : i) involving the grey image of cropped motion rectangular area from current frame to calculate said temporal parameter "t" and updating "t” with history and calculating possible number of iterations Alterations"
- said scene complexity is determined based on (a) inter class difference of foreground and background (b) number of objects present and (c) extent of processing based on the particular processing task.
- a Controller module for spawning a number of processing threads depending on the number of CPU cores present as available from the system hardware information and a task scheduler module for generating the sequence indicating the order in which the individual channels are to be served for analytics tasks.
- multi channel join- split mechanism adapted for low and /or variable bandwidth network link
- a sender unit adapted to receive multi channel inputs from a particular site to join and compress into a single channel and a receiver unit at the client site to receive the inputs and extract the individual channels for the purposes of end use said sender unit adapted to combine while transmitting multi channel inputs into a single channel ,frame by frame, and controlling the transmission bit rate to avoid jittery out puts and/or any interference between individual channels and/or starvation for any single channel.
- a frame header is transmitted with each frame of the combined stream, said frame header containing meta data about the constituent streams, said receiver unit adapted to split the combined stream into constituent streams based on said frame header.
- a sample module is adapted to take the current frame from the channel specific memory area at a fixed rate for those channels and combines to a single frame along with generation of a look-up table to store the channel ID and its boundary within the combined frame and finally compressed and checked to identify all motion vectors which cross the allocated inter-frame boundary and forcibly set all such motion vectors to null to ensure that the video content of one constituent frame within the combined frame does not interfere with the content of another constituent frame , a frame header composed with meta data information about the position of the individual channels frames within the combined frame , the resolution of the individual frames and the time stamp; said receiver unit is adapted to open a TCP connection with the sender and request for all or selected channels including selectively specifying the format
- said event decider means comprises an enhanced object tracking system comprising : object tracking means in conjunction and one or more PTZ cameras wherein when an object is first detected in a fixed camera view of the said object tracking means the same is adapted to track the object and also generate and transmit the positional values along with a velocity prediction data to the PTZ camera controller; said PTZ camera controller adapted to receive the positional information of the object in the PTZ camera view periodically involving scene registration and coordinate transformation technique.
- said coordinate transformation following : a. identifying a set of points in the static camera as A,B, ... and also corresponding points ⁇ ', ⁇ ', ... in the PTZ camera by the user; b.
- a site map server installed within each autonomous system and also within the centralized server gateway to the entire system which is adapted to receive request from any authorised components of the system and respond with positional data corresponding to any component linked
- said site layer preferably multi-layered and components linked to any spatial position of the map in any layer.
- a method for cost- effective and efficient transferring /recording sensory data from single or multiple data sources to network accessible storage devices comprising: atleast one sensory data recording server adapted to record inputs from single /multiple data sources in atleast one local storage space with the URL of the files stored in database; transferring the thus stored files from said local storage to a network based central storage provided for accessing the files for end use/applications, said transfer of sensory data from source to the central storage via said local storage being carried out taking into consideration the data download speed(inflow rate) from data source to server along with the availability of network bandwidth at any given point of time for efficient network bandwidth sharing amongst multiple data sources to said storage device in the network.
- said sensory data recording server is adapted to monitor available total network bandwidth and per channel inflow rate and based thereon decide rate of per channel video transfer from the server local storage to said central storage.
- sensory data from the source are recorded in the form of variable length clips wherein the clip duration is set by the user or set by the server itself.
- step of determining the optimal bit rate for uploading sensory inputs comprising the following steps:
- a method for sensory input recording and live streaming in a multi-server environment comprising: a fail-safe server group
- Each said server group comprising plurality of acquisition cum recording servers said multiple recording servers adapted to exchange information amongst one another and left over capacity of each server is known along with the channel information of every other server such that in case of any server failure
- the remaining active servers in the server group automatically distribute the required operative load amongst the remaining operative servers for a fail safe recording and streaming of the sensory data, without any external control.
- each recording server auto registers in the system and a database entry is created with the server ID whereby the said recording server gets listed in the database and is then ready for recording data from one or more sources.
- the recording is done by breaking the data streams into chunks or clips of small duration and the clips are initially stored in a local server storage space and periodically uploaded to one or more network attached storage in a round robin fashion.
- the capacity of the respective servers in a server group is based on the memory, bandwidth and current processor utilization within the server.
- a server group is adapted to allocate any one of the operative servers in said group as the group master server and continuously monitor the servers in the group and their respective capacities and decide on the allocation and release of the input sensory source from any server within the Group.
- the said group master server is adapted to release or add a sensory input source to any other server within the group based on required (a) addition of an input source (b) deletion of an existing input source (c ) addition of anew recording server to the system or when a failed server again re-operates and (d) when a running server stops functioning.
- an intelligent and unified method of multiple component colour object analysis in a scene favouring scene analytic applications comprising: multiple component colour coherent background estimation involving colour correlation of neighbouring pixels and inter-frame multiple component colour correlation using said multiple components as a composite data and using the relative values of these components to maintain accurate colour information and appearance of the true colour in the estimated background frame.
- An intelligent and unified method of colour object analysis as above comprising (A) unified colour coherent background estimation involving statistical pixel processing ;(B) removal of shadow and glare from the scene along with removal of electronics induced different types of noises in sensors and vibrations of sensors;(C) characterization of pixels in the foreground regions and extract moving and/or static objects.
- An intelligent and unified method of colour object analysis as above comprising tracking variety of objects individually and generating related information for rule-engine based intelligent analytical applications.
- An intelligent and unified method of colour object analysis as above wherein said unified colour coherent background estimation involving statistical pixel processing comprises using R,G,B components as a composite single structure in a unified manner to thereby preserve the mutual relationship of theses colours components in each individual pixel in order to maintain true colour appearances in the estimative colour background frame; continuously readjusting modeled or predicted values for each colour pixel in a frame with all sequential forthcoming frames of the colour video; correlate the spatial distribution of the colour values in a local region to model the pixel background colour value.
- background frame construction comprises constructing colour background reference frame from representative colour values of the generated clusters,if matched colour cluster has significantly high occuerance relative to the overall population occuerance then the representative colour of the colour cluster is used as the value of the colour pixel in the colour background refence frame.
- the removal of the shadow, glare and sensor generated noises comprises removal of shadow and glare in background and /or foreground segmentation process for dynamic scenes involving image characteristics parameters.
- said image characteristic parameters comprise
- every column of the input frame is filtered with the same high pass filter.
- the average of the filtered values of the overall filtered image is considered as vertical sharpness parameter S y .
- a method of face detection in video images and the like comprising the step of limiting the search space involving motion detection technique and controlled computational requirements based on desired accuracy by carrying out prediction of number iterations and temporal parameter "t".
- a method of face detection in video images as above comprising the steps of:
- a method of face detection in video images as above comprising using the convolution on probable face regions with Haar feature set to confirm faces and publishing the confirmed faces based thereon.
- N * N / (ScaleFactor ' )
- iv. T f(M, N, t, pixelShift, alteration), for a fixed size window.
- v. Calculating average t in host machine and tune the parameters pixelShift, alteration accordingly using generated lookup table to suite the bandwidth; and vi.
- a second pass upon the probable face regions detected by first pass.
- a method of resource allocation for analytical processing involving multi channel environment comprising : estimating scene complexity relevant for frequency of frame processing ;
- a system for multi channel join-split mechanism adapted for low and /or variable bandwidth network link comprising: a sender unit adapted to receive multi channel inputs from a particular site to join and compress into a single channel and a receiver unit at the client site to receive the inputs and extract the individual channels for the purposes of end use said sender unit adapted to combine while transmitting multi channel inputs into a single channel , frame by frame, and controlling the transmission bit rate to avoid jittery outputs and/or any interference between individual channels and/or starvation for any Single channel.
- a system as above adapted for intelligent data compression without affecting the decoding process.
- a system as above comprising means for encoding the stream with variable bit rate depending upon the available bandwidth from server to the client, a frame header is transmitted with each frame of the combined stream, said frame header containing meta data about the constituent streams, said receiver unit adapted to split the combined stream into constituent streams based on said frame header.
- a system as above wherein the sender unit is adapted to receive raw inputs or decode the inputs to raw input and store in memory allocated for inputs from a defined channel and generate an initial fps on request from a client, on request of a subset of channel from the client , a sample module is adapted to take the current frame from the channel specific memory area at a fixed rate for those channels and combines to a single frame along with generation of a look-up table to store the channel ID and its boundary within the combined frame and finally compressed and checked to identify all motion vectors which cross the allocated inter-frame boundary and forcibly set all such motion vectors to null to ensure that the video content of one constituent frame within the combined frame does not interfere with the content of another constituent frame , a frame header composed with meta data information about the position of the individual channels frames within the combined frame , the resolution of the individual frames and the time stamp; said receiver unit is adapted to open a TCP connection with the sender and request for all or selected channels including selectively specifying the format for compression, additional commands to
- a system for enhanced object tracking comprising: object tracking means in conjunction with one or more PTZ cameras wherein when an object is first detected in a fixed camera view of the said object tracking means the same is adapted to track the object and afso generate and transmit the positional values along with a velocity prediction data to the PTZ camera controller; said PTZ camera controller adapted to receive the positional information of the object in the PTZ camera view periodically involving scene registration and coordinate transformation technique.
- a system for enhanced object tracking as above wherein said means of coordinate transformation from fixed camera view to PTZ camera view involves coordinate transformation technique comprising weighted interpolation method.
- a system for enhanced object tracking as above which is adapted to carry out said coordinate transformation following: a. identifying a set of points in the static camera as A, B, etc and also corresponding points ⁇ ', ⁇ ', etc respectively in the PTZ camera by the user; b. mapping any arbitrary point C in the static camera to the corresponding point
- a system for enhanced object tracking as above wherein for a bounding rectangle to be mapped from the static view to the PTZ view, the system is adapted to apply said coordinate transformation technique for all the four corner points of the rectangle.
- an intelligent automated traffic enforcement system comprising :
- a video surveillance system adapted to localize one or more number plates / License Plates of vehicles stationary or in motion in the field of view of atleast one camera without requiring to fix the number plate in a fixed location of the car
- the license plate can be reflective or non-reflective, independent of font and language, and using normal security camera, and filtering out other texts from the field of view not related to the number-plate, enabling to process the localized number plate region with any Optical Character Recognition, and generate localized information of the number plate with or without in other relevant composite information of car (type, possible driver snapshot, shape and contour of the vehicle) in parallel to monitor traffic and an intelligent video analytical application for event detection based on the video feeds
- An intelligent traffic enforcement system as above wherein the process depends localizes possible license plate in the field of view of the camera by (a) analysing statistically correlation and relative contrast between the number plate content region and the background region surrounding this content, (b) unique signature of number plate content based on pixel intensity and vertical and horizontal distribution, (c) color features of the content and surrounded background.
- An intelligent automated traffic enforcement system as above wherein said video analytic process is carried out in the sequence involving (a) configuration means ( b) incident detection means (c ) incident audit means (d ) reporting generation means (e) synchronization means and (f) user management means.
- An intelligent automated traffic enforcement system as above wherein said configuration means adapted to configure parameters for incident detection and management comprises (i) camera configuration means (ii) means for providing for virtual loops in regions where monitoring is required(iii) means for setting time limits for the monitoring activity (iv) means providing feed indicative of regular traffic moving directions for each camera (v) means providing for setting speed limits to detect over speeding vehicles (vi) means for setting the sensitivity and duration determining traffic abnormality and congestion.
- An intelligent automated traffic enforcement system as above wherein said incident detection means is adapted to detect deviations from set parameters, analyze appropriate video feed and check for offence involving (a) recording by way of saving video feeds from various traffic locations of interest (b) generating alarm including alerts and/or notifications visual and/or sound based on any incident detection involving traffic violation and (c ) registering the incident against the extracted corresponding license plate number of the violating vehicle.
- incident audit means comprises : filter means adapted to reach to the incident if incident is an archived incident and in case of live incident means for viewing the details; means for generating details of the incident, a link to incident video and a link to license plate image of the vehicle; means for verification of the incident by playing the video and vehicle's registration number by viewing the license plate image and If the license plate number is incorrect means to enter the correct vehicle number of the incident image; means for updating incident status changed from "Pending"/"Acknowledged" to "Audit” and saving into the database. means to enter remark about the action taken while auditing the incident and finally the remark is saved in the database ith possible re- verification for future reference.
- incident reporting means comprises means for automatized generation of incident detail reports and incident summary report and generation of offence report.
- An intelligent automated traffic enforcement system as above wherein said synchronization means includes means adapted for synchronization with handheld device applications.
- An intelligent automated traffic enforcement system as above wherein said user management means includes interface for administrative functions including (a) user creation and management (b) privilege assignment and (c) master data Management.
- a computer readable medium adapted for enabling and operating an integrated intelligent sensory input/data acquisition cum recording server group and /or analytics server group adapted to facilitate fail-safe integration and /or optimised utilization of various sensory inputs for various utility applications comprising at atleast one autonomous system having :
- said sensory input acquisition cum recording server group comprising plurality of acquisition cum recording servers which are operatively linked to assess respective server capacity and operate as a group to enable fail-safe support when any of the servers in the group fail to operate the remaining operative servers in the group are adapted to distribute and take over the sensory input load of the non-operative server/s to render the system fail safe and self sufficient ; and/or B) said analytics server group comprising plurality of analytics server for intelligent analysis including resource dependent analytical accuracy control including means adapted for computing complexity of scenes and dynamically reconfigure the analytical processing steps for optimal analysis and/or availability of computational and other resources for on-line and real-time and/or on demand for efficient and user friendly streaming/analysis/detection/alert generation of events and/or follow up actions; and
- a computer readable medium adapted for enabling and operating a method for cost-effective and efficient transferring /recording sensory data from single or multiple data sources to network accessible storage devices comprising:
- Atleast one sensory data recording server adapted to record inputs from single /multiple data sources in atleast one local storage space with the URL of the files stored in database; transferring the thus stored files from said local storage to a network based central storage provided for accessing the files for end use/applications,
- a computer readable medium adapted for enabling and operating a method for sensory input recording and live streaming in a multi-server environment comprising: a fail-safe server group
- Each said server group comprising plurality of acquisition cum recording servers said multiple recording servers adapted to exchange information amongst one another and left over capacity of each server is known along with the channel information of every other server such that in case of any server failure in said server group the remaining active servers in the server group automatically distribute the required operative load amongst the remaining operative servers for a fail safe recording and streaming of the sensory data.
- Yet further aspect of the invention is directed to a computer readable medium adapted for enabling and operating an intelligent and unified method of multiple component colour object analysis in a scene favouring scene analytic applications comprising:
- Another aspect of the invention is directed to a computer readable medium adapted for enabling and operating a method of face detection in video images and the like comprising the step of limiting the search space involving motion detection technique and controlled computational requirements based on desired accuracy by carrying out prediction of number iterations and temporal parameter "t".
- Another aspect of the invention is directed to a computer readable medium adapted for enabling and operating a method of resource allocation for analytical processing involving multi channel environment comprising: estimating scene complexity relevant for frequency of frame processing ; spawning of processor threads based on physical CPU cores involving a controller; allocation of threads to video channels for analytical processing based on requirements; and feeding the frames for processing to a video analytics engine at an fps F, where F is calculated dynamically by the analytics engine itself depending upon its processing requirements based on scene complexity to thereby favour optimal sharing of resources eliminating unnecessary computing.
- Yet another aspect of the invention is directed to a computer readable medium adapted for enabling and operating a system for multi channel join-split mechanism adapted for low and /or variable bandwidth network Jink comprising: a sender unit adapted to receive multi channel inputs from site to join and compress into a single channel and a receiver unit at the client site to receive the inputs and extract the individual channels for the purposes of end use said sender unit adapted to combine while transmitting multi channel inputs into a single channel , rame by frame, and controlling the transmission bit rate to avoid jittery outputs and/or any interference between individual channels and/or starvation for any single channel.
- a further aspect of the invention is directed to a computer readable medium adapted for enabling and operating a system for enhanced object tracking comprising: object tracking means in conjunction with one or more PTZ cameras wherein when an object is first detected in a fixed camera view of the said object tracking means the same is adapted to track the object and also generate and transmit the positional values along with a velocity prediction data to the PTZ camera controller; said PTZ camera controller adapted to receive the positional information of the object in the PTZ camera view involving scene registration and coordinate transformation technique.
- Another aspect of the invention is directed to a computer readable medium adapted for enabling and operating an intelligent automated traffic enforcement system
- a video surveillance system adapted to localize one or more number plates / License Plates of vehicles stationary or in motion in the field of view of atleast one camera without requiring to fix the number plate in a fixed location of the car/the license plate can be reflective or non-reflective, independent of font and language, and using normal security camera, and filtering out other texts from the field of view not related to the number-plate, enabling to process the localized number plate region with any Optical Character Recognition, and generate localized information of the number plate with ot without in other relevant composite information of car (type, possible driver snapshot, shape and contour of the vehicle) in parallel to monitor traffic and an intelligent video analytical application for event detection based on the video feeds
- the above disclosed invention thus includes advancement based on bandwidth adaptive data transfer with predicted optimal bandwidth sharing among multiple data transfer processes for low or moderately priced systems.
- each server not only monitors the available bandwidth but also in-flow rate for each channel into the server separately. It is done without compromising subjective fidelity of the data, and accordingly adjusts the upload rate for any particular channel without affecting the speed and performance of other channels being processed by multiple networked servers let alone the single server.
- the data stream is segmented into variable sized smaller chunks or clips and the rate of uploading the clips to the central storage is adjusted depending on the available network bandwidth and data inflow rate for that particular channel which is dependent on the scene activity or content characteristics.
- the team members In case of breakdown of one or more servers, the team members automatically detect it and share the load of the failed server(s), without any central control or without support from any fail-over or mirror server. This eliminates the need for costly failover or mirror server and the load is always evenly distributed as per the capacity of the individual server hardware. This advancement is unique serve as an example of cooperative social networking implemented in machine level.
- an enhanced multi channel data aggregation technique for data transmission over low and variable bandwidth communication network has been proposed which also avoids inter-channel interferences. While transmitting multi-channel video over low and variable bandwidth network link, they are combined into a single channel video, frame by frame, and then transmission bit rate is controlled to avoid jittery video at the other end or interference between individual channels. It also avoids starvation for any single channel.
- the underlying data compression algorithm is intelligently handled without affecting the decoding process with a standard equivalent decoder. For example in case of video, the motion vector generation step in the underlying MPEG type compression is intelligently controlled, so that no motion vector crosses-over the intra-frame boundary in the combined frame. This eliminates interference between any two channel data frames in the combined frame.
- the invention also propose a monolithic architecture by integrating video analytics functionalities as integral part of the proposed Video Management System architecture with same architectural and design philosophy. That's why the overall architecture is called a truly Intelligent Video Management System architecture.
- Controller module controls the rate at which video frames are supplied to different analytics engines.
- the Controller uses a novel technique to control the rate of decoding the video frames and sending them to the Analytics engine for content analysis based on computational bandwidth available and also on the current scene complexity measure as received from the Analytics engines themselves.
- the present invention further discloses advancement in process for analyzing moving image sequences, which comprises applying automatic adaptive unified framework for accurate predictive colour background estimation using neighbouring coherent colour and inter-frame colour appearance correlation under severe natural condition such as shadow, glare, colour changes due to varying illumination, and effect of lighting condition on colour appearance, electronics generated induced noises (e.g. shot noise, but not limited to) obtain more accurate object shape, contour & spatial position.
- the object detection and analysis process can be accelerated and the foreground selection accuracy can be improved.
- detected objects can be characterized, classified, tracked and correlated to identify different events in any natural video sequence under various demographic and environmental conditions.
- the invention further enables advancements in Static Foreground Pixel estimation technique using multi-layer hierarchical estimation to identify static objects in a video by aggregation of static pixels in parallel to other moving colour objects in the scene.
- the process involves background scene estimation, foreground background segmentation, short time still background estimation, static foreground pixel estimation and then static object generation.
- the proposed technique is thus an advancement in the related art and it gives much more control over the process of distinguishing foreground pixels (of the static object) from the background pixels.
- the present invention is also on method to enhance the efficiency of extracting face regions from a sequence of video frames. Also, depending on the availability of computational bandwidth, the number of iterations and pixel shifts as required in the proposed technique is controlled with the help of a look up table. This helps in striking a balance between the computational requirement and the accuracy of face detection. In a multi-channel, multiple analysis process system, this advanced technique can be used as a cooperative process coexisting with other compute intensive processes. In the proposed technique, the search space is reduced by considering the motion vector and sliding the window only in the blob regions where motion is detected. First, the average time t to analyze an image in host machine is calculated, and for subsequent frames pixel-shifts and number of iterations are calculated based on two lookup tables, to suite the computational bandwidth. To increase the accuracy, a second pass upon the probable face regions detected by first pass is performed. This concept of increasing the accuracy of data analysis automatically depending on available computational bandwidth is novel and unique.
- the framework disclosed herein can be used for such situations, and also for integrating multiple heterogeneous systems in a distributed environment.
- the proposed architecture is versatile enough to interface and scale it to many other management systems.
- the disclosure made herein illustrates how the systems architectural advancement can be advantageously involved for Intelligent Automated Traffic Enforcement System.
- the details of the invention and its objects and advantages are explained hereunder in greater detail in relation to the following non-limiting exemplary illustrations as per the following accompanying figures: Brief Description of the Drawings
- Fig 1 is a schematic layout of an illustrative embodiment showing an integrated intelligent server based system of the invention having sensory input/data acquisition cum recording server group and /or analytics server group adapted to facilitate fail-safe integration and /or optimized utilization of various sensory inputs for various utility applications;
- Fig 2 is an illustrative top level view of intelligent video management system with framework for multiple autonomous system integration
- Fig 3 is an illustration of fail-safe bandwidth optimized recording without any supporting failover support server in accordance with the present invention
- Fig 4. is an illustration of the dataflow diagram from a single video source through the recording server ;
- Fig. 4A to 4J illustrate an exemplary Intelligent Home Security" box involving the system of the invention;
- Fig.5 is an illustration of the single channel data flow in video analytical engine in accordance with the present invention.
- Fig. 6 is an illustration of intelligent video analytics server in accordance with the present invention.
- Fig.7 is an illustration of video management interface functionalities in accordance with the present invention
- Fig.8 is an illustration of intelligent data upload process in accordance with the present invention
- FIG. 9 Is an illustration exemplifying the manner of adding a camera (ALLOCATE) to a GROUP of recording servers in accordance with the present invention
- Fig.10 is an illustration of load balancing when an existing camera is deleted from a GROUP in accordance with the present invention
- Fig. l l is an illustration of the load balancing when a new recording server is added in accordance with the present invention
- Fig. 12 is an illustration of the method of ALLOCATION when a running server stops operation
- Fig. 13 is an illustration of a top level flow diagram of the intelligent colour object (moving, static, transient) analysis in accordance with the present invention
- Fig. 14 is an exemplary illustration of the object analysis stages with pictorial description in accordance with the present invention
- Fig. 15 is an illustration of a process flow diagram for unified computationally adaptive colour appearance correlation based predictive background estimation in accordance with the present invention
- Fig.16 is an illustration of the manner of identification and removal of shadow and glare regions in accordance with the present invention.
- Fig. 17 is an illustration of a conventional process of identification of faces with spatial information
- Fig.18 is an illustration of the process for enhanced and confirmatory identification of faces in accordance with the present invention.
- Fig.19 is an illustration of the manner of providing scene complexity feedback in accordance with the present invention.
- Fig 20 is an illustration of multi threaded video analytics in accordance with the present invention.
- Fig.21 is an i!iustration of the sender and receiver modules used in the system in accordance with the present invention
- Fig.22 is an illustration of the enhanced object tracking system in accordance with the present invention.
- Fig.23 is an illustration of the coordinate transformation used in the present invention.
- Fig. 24 is an illustration of the number plate recognition engine components in accordance with the present invention.
- Fig.25 is an illustration of the localized multiple number plate regions in video images in accordance with the present invention.
- Fig. 26 is an illustration of top level system diagram in accordance with the present invention
- Fig.27 is an illustration of the flow diagram in accordance with the surveillance system in accordance with the present invention
- Fig.28 is an illustration of the video analytics application breakdown structure in accordance with the present invention
- Fig. 29 is an illustration of the junction camera set up in accordance with the present invention.
- Fig. 30 is an illustration of the junction layout in accordance with the present invention.
- Fig. 31 is an illustration of the video recording during working hours in accordance with the present invention.
- Fig. 32 is an illustration of the transition traffic light status in accordance with the present invention
- Fig. 33 is an illustration of the captured number plate in accordance with the invention
- Fig. 34 is an illustration of the incident audit view in accordance with the present invention. Detailed Description of the Invention:
- FIG. 1 shows the broad overview of an illustrative embodiment showing an integrated intelligent server based system of the invention having sensory input/data acquisition cum recording server group and /or analytics server group adapted to facilitate fail-safe integration and /or optimized utilization of various sensory inputs for various utility applications.
- the system basically involves the self-reliant group of recording servers (101), the group of analytical servers (102) and an intelligent interface (103).
- said recording servers apart from being mutually cooperative and self-reliant to continuously monitor and distribute the operative load based on the number of active servers in the group are also adapted for bandwidth optimized fail-safe recording ((104 ) and join-split mechanism for multi channel video .streaming ( 105).
- the analytical servers (102) are also adapted to cater to atleast one of more of background estimation (106), identifying moving, static, quasi static objects ( 107), enhanced object tracking (108), content aware resource scheduling ( 109) , join-split mechanism for sensory date streaming (110) and resource dependent accuracy control (111).
- an Autonomous system (210-01)) is considered as a system capable to implement the functionalities and services involving sensory data and /or its analysis. Also, the system is capable of handling any sensory data/input and it is only by way of an illustration but not by way of any limitations of the present system that the various exemplary illustrations hereunder are discussed with reference to video sensory data.
- the underlying system architecture/methodology is applicable in other sensory data types for a true Intelligent Sensor Management System .
- a number of machine vision products spanning the domain of Security and surveillance, Law enforcement, Data acquisition and Analysis, Transmission of multimedia contents, etc can be adapted to one or more or the whole of the system components of the present invention.
- FIG. 3 shows by way of an embodiment a fail-safe bandwidth optimized recording without any failover support server.
- the input from the pool of sensors (305) are fed not to any single server but to a group of servers (301).
- communication channel (303) is provided to carry inter-VRS communication forming a team towards failover support without any central management and failover server while the communications channel (302) is provided to carry data to central storage involving intelligent bandwidth sharing technique of the invention.
- the Recording system essentially implements the functionalities and services as hereunder:
- Collecting Data real time Collect data from various images, video and other sensory sources, both on-line and off-line, archiving and indexing them to seamlessly map in any relational or networked database in a failsafe way making optimal usage of computing, communication and storage resources, facilitate efficient search, transcoding, retransmission, authentication of data, rendering and viewing of archived data at any point of time.
- Streaming data real time or on Demand Streaming video and other sensory content in multiple formats to multiple devices for purposes like live view in different matrix layout, relay of the content, local archiving, rendering of the sensory data in multiple forms and formats, etc. by a fail-safe mechanism without affecting speed and performance of on-going operations and services.
- the Video Recording system is implemented using hardware and software, where the hardware can be any standard computing platform operated under control of various operating systems like Windows, Linux, Mac OS, Unix, etc. Dependence on hardware computing platform and operating system has been avoided and no dedicated hardware and communication protocol has been used to implement the system.
- Recording server implements an open interface both for input and output, (including standard initiatives by various industry consortium such as ONVIF, PSIA, etc.) / and can input video feed from multiple and different types of video sources in parallel, with varying formats including MPEG4, H.264, MJPEG, etc. OEM specific SDKs to receive video can also be used.
- Internal operating principle of the Recording server is outline below:
- Recording Server operating principle is adapted for the following :
- All the servers in the system including the Recording servers, auto register themselves by requesting and then getting a unique Identification number (ID) from the VMI.
- ID Identification number
- All the configuration data related to the server including the identification of data sources including the video sources it caters to, the storage devices it uses, etc are stored in the database against this ID.
- This scheme has the advantage that with only one Static IP address (that of the VMI), one can access any component of the Autonomous System (AS), and the IP addresses of the individual hardware components may be kept varying.
- the cameras, other video sources or sources generating streaming data can be auto detected or manually added to the VRS.
- the details of the channels are stored in the Central Database. Once done, one or more channels can be added to the Recording System.
- the Recording system thus comprises of one or more Recording servers (VRS) and the Central Database Management System. VRS-es consults the database, know about details of the system, and records the channel streaming data either continuously, or on trigger from any external or internal services, as configured by the user.
- the data stream is first segmented into small granular clips or segments of programmable and variable length sizes (usually of 2 to 10 minutes duration) and the clips are stored in the Local storage of the server, the clip metadata being stored in local database.
- FIG 4 shows the dataflow mechanism in accordance with the invention from a single video service through the recording server.
- the sensory data stream viz. video (405) is feed to a data segment generator (401) which is next stored in segments in local storage (403/402) and thereafter uploaded through data upload module (404) to a central storage (406)/407).
- Any external component of the system can enquire the VRS to know about the details of the channels it is using and get the data streams for purposes like live view, Relaying to other devices etc using a networked mutual client-server communication protocol
- Bandwidth adaptive data uploading to central storage system In the system of the invention, an efficient technique has been designed to transfer video or other sensory data received from the channels to the central storage system via the local storage. Instead of allocating a particular data source (e.g., a camera) to a particular server (dedicated point to point) for recording of data (e.g, video), it is allocated to a 'Server group' with multiple servers in the group [Fig 3]. The members of the group exchange their capacity information amongst themselves and share the load according to their capacity. In case of breakdown of one or more servers, the team members share the load of the failed server(s), without any central control or without support from any dedicated fail-over server.
- a particular data source e.g., a camera
- server dedicated point to point
- the members of the group exchange their capacity information amongst themselves and share the load according to their capacity. In case of breakdown of one or more servers, the team members share the load of the failed server(s), without any central control or without support from any dedicated fail-
- each server not only monitors the available bandwidth but also the data inflow rate for each channel into the server, and accordingly adjusts the upload rate for an individual channel.
- the data stream is segmented into variable sized clips and the rate of uploading the clips to the central storage is adjusted depending on the available network bandwidth and data inflow rate for that particular channel [Fig 4].
- the sensor data stream ( 405) is segmented in data segment generator (401) which is next stored in local storage ( (402 ,403) and thereafter involving a data upload module (404) the same is sent to the central storages ( 406/407).
- the system of the invention is further adapted for back up support in case of server failure without the involvement of any special independent stand by support server.
- dedicated fail-over servers are used which senses the heartbeat signals broadcasted by the regular servers. Once the heart beat is found missing, the failover server takes up the task of the failed server. This technique is inefficient as it not only blocks the resources as dedicated failover servers, but cannot utilize the remaining capacity of the existing servers for back up support. Also, failure of the failover server itself jeopardizes the overall failover support system.
- the recording servers exchange information amongst themselves so that each server knows the leftover capacity and the channel information of every other server. In case of server failure, the remaining active servers distribute the load amongst themselves.
- Video Analytics System essentially implements the functionalities as hereunder:
- Data Content Analysis Intelligently analysing the data, on-line or off-line, to extract the meaningful content of the data, identifying the activities of foreground human and other inanimate objects in the scene from the sensor generated data, establishing correlation among various objects (living or nonliving) in the scene, establishing correlation amongst multiple types of sensory data, identifying events of interests based on the detected activities— all either automatically or in an user interactive way under various demographic and natural real life situations.
- Automatic Alert Generation Generating Alerts, signals, video clips, other sensory data segments, covering the events automatically as and when detected.
- the Video Analytics system comprises hardware and software, where the hardware can be any standard computing platform operated under control of various operating systems like Microsoft Windows, Linux, Mac OS, Unix, RTOS for embedded hardware, etc.
- FIG. 4A a schematic diagram of a Networked Intelligent Villa/Home/Property Monitoring System is shown. All of the intelligent video management server and intelligent monitoring applications that are described in previous sections have been embedded into the V/deonetics Box.
- the Box has an easy to use GUI using touch-screen so that any home/villa/property owner can easily operate it with minimum button pressing using visual display based instructions only.
- the top level systems architecture for the embedded hardware and details of the components in the hardware system is shown in FIG. 4B.
- the following is a micro-architectural components summary for an example of a multichannel IP-camera solution. Video from IP-Cameras is directly fed to the computer without the requirement of any encoder. There are three options: One, no network switch is required.
- the Motherboard should have multiple Ethernet ports; two, the Motherboard has only one Ethernet port assuming all the cameras are wireless IP- Cameras.
- the Motherboard should have 1 x Ethernet port and 1 x Wifi interface; and three, the Motherboard has only one Ethernet port, the cameras are wired, but a Network switch is required as an external hardware.
- the event clip is also streamed to any designated device over the Internet.
- Interfaces are required to handle the above tasks: at least one RELAY O/P for siren drive or DIO for Transmitter interface; and a 3G interface for SMS/MMS or sending event clip to Cell Phone.
- Other usual hardware includes
- Video from analog camera is received by ah encoder hardware.
- the encoded RAW image is fed to the computer for processing.
- System Hardware should be capable to handle the following activities:
- the encoder could be a separate module connected to motherboard
- the encoder circuitry may be embedded in the mother board
- Event clip is archived
- Event clip is also streamed to any designated device over Internet
- the following hardware Interfaces are required to handle the above tasks: a. At least one RELAY O/P for siren drive or External Transmitter interface (DIO)
- USB a. Touch Screen Interface
- FIG. 4C a top level heterogeneous system architecture (both IP and analog cameras) is illustrated.
- FIGS. 4D-4J an operational flow by a user and representative GUI using a touch panel display of the intelligent monitoring system is detailed in a step-by-step flow.
- the improved intelligent video surveillance system is highly adaptable and can be used in a large variety of applications can be conveniently adapted to a variety of customer-specific requirements. Also, the intelligent video surveillance system is automated, intelligent, and requires a minimum or no human intervention.
- An Analytics engine detects various activities in the video or other sensory data stream and on detection of said activities conforming to one or more Events, sends notification messages with relevant details to the recipients.
- the recipients can be the VMI, the central VMS or Surveillance Clients or any other registered devices.
- the scene is analyzed and the type of analysis depends on the type of events to be detected.
- the data flow within the Analytics Engine for a single channel, taking video stream as the channel data, is as schematized below [Fig. 5] .
- the functionalities of various internal modules of the Analytics Engine and other components are described below, taking Video channel as an example for Sensory data source.
- the Scene analyzer is the primary module of the Analytics engine and that of the IVAS as well. Depending on the Events to be detected, various techniques have been developed to analyze the video and sensory data content and extract the objects of interests in the scene or the multi-sensory acquired data. Importantly, the scene analyzer is adapted to analyze the media content (e.g., video) based on intelligent scene adaptive colour coherent object analysis framework and method . Implementation of the same has been done so that it is adaptive to the availability of computational bandwidth and memory and the processing steps are dynamically reconfigured. As for example, as described further in detail hereunder a trade-off is done automatically by the Analytics engine to strike a balance between the accuracy of face capture and the CPU clock cycles available for processing.
- media content e.g., video
- the Scene Analyzer generates meta-data against each frame supplied to it for analyzing. It also computes the complexity of the scene using a novel technique and dynamically reconfigure the processing steps in order to achieve optimal analysis result depending upon the availability of the computational and other resources for on-line and real-time detection of events and follow up actions. It feeds the metadata along with the scene complexity measure to the Controller, so that the Controller can decide the optimal rate at which the frames of that particular video channel should be sent to the Analytics engine for processing. This technique is unique and saves computational and memory bandwidth for decoding and analysis of the video frames.
- Rule Engine (502) : The Rule Engine keeps history of the metadata and correlates the data across multiple frames to decide behavioural patterns of the objects in the scene. Based on the rules, various applications can be defined. As for example it is possible to detect whether a person in jumping a fence or whether there is a formation of crowd or whether a vehicle is exceeding the speed limit, etc.
- Event Decider (503) The behavioural patterns, as detected by the Rule Engine is analyzed by this module to detect various events in parallel.
- the Events can be inherently defined or it may be configured by the user. As for example, if there is a crowd formation only in a specific zone where other areas are not crowded, that may be defined to be an event.
- a message is generated describing the type of event, time of occurrence of the Event, the location of occurrence of the Event, the Video clip URL, etc.
- the Event decider can also control any external device including a PTZ camera controller which can focus a region where the event has taken place for better viewing of the activities around that region or recording the scene in a close up the view.
- a PTZ camera controller which can focus a region where the event has taken place for better viewing of the activities around that region or recording the scene in a close up the view.
- One such advanced framework is detailed hereunder as enhanced object tracking where the utility of an Object tracking system is enhanced using a novel technique using a PTZ camera along with the Object tracking system.
- a Controller module (602) as shown in Figure 6 has been designed which can receive multiple video channels, possibly in some compressed form (e.g., MJPEG, Motion JPEG2000, MPEG, H.264, etc. for video and relevant format for other sensory data such as MP4 for audio, for example but not limited to), and feeds the decoded video frames to the Analytic engine.
- the Controller uses an advanced technique to decide the rate of decoding of the frames and to feed the decoded video frames of multiple channels to the Analytics engine in an optimal way, so that the number of frames sent per second for each video channel is individually and automatically controlled depending on the requirement of the Analytics engine and also on the computational bandwidth available in the system at any point of time.
- the technique has been described in detail in relation to video content driven resource allocation for analytical processing.
- the Controller also streams the video along with all the Video Analytics data (existing configuration for Events, Event Information, video clip URL etc) ⁇ either as individual streams for each channel, or as a joined single stream of video data for all or user requested channels.
- Video Analytics data existing configuration for Events, Event Information, video clip URL etc
- a novel technique for joining the video channels and transmitting the resulting combined single channel over IP network has been deployed to adapt to varying and low bandwidth network connectivity. The technique is described in detail in relation to video channel join-split mechanism for low bandwidth communications.
- the Controller can generate Events on its own for the cases where Events can be generated without the help of Video Analytics engine (eg, Loss of Video, Camera Tampering as triggered by Camera itself, Motion detection as intimated by the Camera itself, as so on).
- Video Analytics engine eg, Loss of Video, Camera Tampering as triggered by Camera itself, Motion detection as intimated by the Camera itself, as so on.
- VMI Video Management Interface
- the Video Management Interface (702) is shown in figure 7 Which interfaces between an individual Autonomous System and rest of the world. It a/so acts as the coordinator among various other components within a single Autonomous system, viz, Video Recording System (703), Intelligent Video Analytical Server (704), Surveillance Clients (701), Remote Event Receiver (705), etc. [It essentially implements the functionalities including :
- Filtering and need based transmission of data Distribution of whole or part of the collected sensory data, including the video and other sensory data segments generated as a result of detection of an Event by the Analytical engine above, at the right recipient at the right point of time automatically or on user interaction.
- Directed distribution of Alerts Distributing Event information in various digital forms (SMS, MMS, emails, Audio alerts, animation video, Text, illustrations, etc. but not limited to) with or without received data segments (viz, video clips) to the right recipient at the right point of time automatically or on user interaction.
- Providing a common gateway for heterogeneous entities Providing a unified gateway for users to access the rest of the system for configuration, management and monitoring of system components.
- Video Management Server Interface acts only as a unified gateway to the services being executed in other hardware devices, only for configuration and status updating tasks. This opens up the possibility of keeping the User interface software unchanged while integrating new type of devices. The devices themselves can supply their configuration pages when the VMI connects to them for configuration. Similarly, the messages generated by the servers can also be shown in the VMI panel seamlessly.
- Providing Live view or recorded view of the data stream Enabling user to view camera captured video in different matrix layouts, view other sensory data in a presentable form, recorded video and other data search and replay, Event clips Search and replay, providing easy navigation across camera views with help of sitemaps, PTZ control, and configuring the system as per intended use.
- the VMS system can be accessed through the standalone surveillance client or any standard Internet browser can be used to access the system.
- Handheld devices like Android enabled cell phone or tablet PCs can also be used as a Client to the system for the purposes (wholly or partially) as mentioned above.
- the Remote Event receiver (705) The Remote Event receiver (705)
- RER (705) shown in Figure 7 is the software module which can be integrated to any other modules of the IVMS.
- the Remote Event Receiver is meant to receive and display messages and ALERTs from other components, which are multicast or broadcasted. Those messages include Event ALERTS, ERROR status from VRS or IV AS, operator generated messages, etc.
- the Messages can be in the Video as wed as Audio form, or any other form as transmitted by the Video management system components and the resulting response from by the RER depends on the capability and configuration of the hardware where the RER is installed.
- the IVMC can switch to RER mode and thus will respond to ALERTs and messages only.
- Central VMS System (204 in Figure 2) is adapted to serve as a gateway to any Autonomous System (210-01...210-0n) components. It also stores the configuration data for all ASes in its Centralized database. It is possible to integrate otherwise running independent VMS systems into a single unified system by including Central VMS in a Server and configure that accordingly.
- a Sitemap server is included within each Autonomous System (210-01...210-0n) and also within the Centralized VMS(204 in Figure 2).
- the Sitemap server listens to requests from any authorized components of the System and responds with positional data corresponding to any component (Camera, server, user etc.) which is linked to the Site map.
- the Site map is multilayered and components can be linked to any spatial position of the map in any layer.
- an Access Control System or a Fire Detection System can be integrated similar to VRS or IVAS, configured using IVMC and VMI, and their responses or messages can be received, shown or displayed and responded to by IVMC or RER, stored as done for Event clips or Video segments and searched on various criteria.
- the system of the invention detailed above is further versatile enough to interface and scale to many other management systems such as the involvement in intelligent automated traffic enforcement system also discussed in later sections.
- FIG. 8 illustrates the manner of segmented data system based stage wise data uploading from local storage to a central storage.
- the various stages/components are illustrated therein under references 801 to 807.
- What is disclosed is a fault tolerant and efficient method for recording sensory data (e.g video)as received from a single or multiple number of data sources like Cameras to network accessible storage devices, estimation of optimal required bandwidth for individual data channels taking into consideration the data download speed (inflow rate) from data source to server along with the availability of network bandwidth at any given point of time, efficient network bandwidth sharing amongst the data channels for uploading data to storage devices over network.
- the framework and technique is disclosed and described below taking example for Video receiving and storing, though the same framework can be used for other type of data also.
- this method is more effective to provide a demand based network bandwidth to all the services and also to maintain the QOS for client machines, especially when the client machine is used for live viewing of the camera FOVs.
- Video Management System using IP enabled video capturing devices has become an integral part of Surveillance industry today.
- a basic requirement of this type of systems is to input compressed video streams from multiple cameras and record the video in storage devices.
- the complexity and hence the chafienges for efficient deployment of the system were less. This is because each DVR or NVR was a standalone system taking feed from a handful of cameras (typically 16 or 32), and used their dedicated local storage devices to record the video.
- Video Management System emerged as a solution.
- each server catering a set of Video Capture devices (e.g., Cameras), one or more network accessible RAID configured storage devices, and multiple workstations.
- Video Capture devices e.g., Cameras
- network accessible RAID configured storage devices e.g., Ethernets
- workstations e.g., a set of network accessible RAID configured storage devices
- Each server now needs to handle 64 or more cameras, stream the video from the cameras to the client machines.
- a Video Management Server system there is a requirement for an efficient Network bandwidth management, so that all the network bandwidth hungry tasks assigned to the servers, viz, grabbing video from IP- cameras, uploading video to Network accessible storage devices and streaming the video channels to the Clients on demand, are executed in an optimal way. Also, the system must be fault tolerant so that intermittent failure of the Network connectivity from the Server to the Network accessible storage devices does not result loss of video in the storage. All these activities should happen automatically without any user interaction. Due to high demand in bandwidth to perform such tasks, especially for video data, often separate high speed network are dedicated to transfer data to storage media. Dedicated high speed network is costly and often require costly storage devices as well. Often this is a overkill for low or moderately priced installations.
- the proposed system is unique as it handles all the above tasks in an efficient way, with optimal use of the resources (Network, Storage space), even using a decent server having onl one Network interface card.
- the video from the cameras are not directly recorded to the Central Storage (N AS/SAN). Instead, the Video Recording Server first stores the video in a local storage space and then transfers the video to NAS/SAN periodically with the URL of the video files stored in the database. Intermittent loss of connectivity from the server to the network accessible NAS/SAN and/or that to the Database Management System does not result in loss of recorded video, as during this period the data is recorded in the Local storage space within the server hardware.
- the video from the local storage is transferred to the Central storage automatically without any user interaction.
- a good amount of network bandwidth is consumed if the number of video channels (camera etc) is high. Therefore, the video transfer to NAS/SAN introduces a peak bandwidth requirement which may not be available in the network interface of the server, and therefore, may affect the QOS desired by the Surveillance clients for live view, as the Video Recording Server also serves as the Video Streamer to the Video Surveillance Clients. Further, this activity of uploading video to NAS/SAN may also disrupt the activity of grabbing the video from the cameras due to bandwidth throttling, which is not permissible at all.
- the server monitors the available total network bandwidth and per channel video inflow rate, and decides the rate of per channel video transfer from the server (local storage) to the NAS/SAN.
- the Video from the cameras are recorded in the form of variable length (typically 2 to 5 minutes) video clips.
- the clip duration may be set by the user or it can be decided by the server itself.
- the video clips are then uploaded to the Central storage (NAS/SAN).
- the advancement is directed- to use optimal bit rate for uploading video.
- the average bit rate for each channel is calculated separately in periodic intervals. For that, the video streaming rate (Dj ) of a particular camera (Q) camera to the server is estimated. Also the available network bandwidth (B) at that instant is known from the System.
- the frequency of Clip upload for channel, Q is then calculated as:
- the rate of uploading the clips to the NAS/SAN is varied dynamically so that the effective average bit rate of video upload to the Central Storage for a particular channel is controlled based on the availability of Network Bandwidth and the actual optimal rate so that the requirement of local storage space stays within acceptable limits and the system comes to equilibrium.
- FIG. 9 illustrates the fail safe mechanism for sensory data such as video recording and live view streaming in a multi - server, multi-camera system in accordance with the present invention.
- Figure 9 the manner of adding a camera (ALLOCATE) to a GROUP of recording servers is shown by way of components/features 901 to 908.
- FIG 10 the manner of load balancing when an existing camera is deleted from a GROUP is shown by way of components/features 1001 to 1002.
- FIG 11 the manner of load balancing when a new recording server is added is illustrated by way of components/features 1101 to 1109.
- FIG 12 the manner of ALLOCATE method when a running server stops operation is shown by way of components/features 1201 to 1202.
- What is disclosed is a fail-safe architecture for recording video in a multi-camera Video Management system, a novel technique for estimating server capability for load balancing, automatic uniform distribution of video recording load across all the active servers, auto-registration of recording servers when they are active in the network, use of multiple distributed NAS/SAN storage devices , automatic back up of recorded video in the server local storage space in case of failure of the central storage, automatic upload of the video files to the central storage once the storage system is recovered from failure, video streaming to the clients without passing the video through any central hardware and thus avoiding single point of failure, automatic camera add and release operation on new server addition in the system and in case of server failure, without any manual intervention.
- the recording system thus constituted using multiple servers is highly scalable with respect to increase or decrease in the number of cameras, tolerant to intermittent or permanent failure of one or more servers or one or more storage devices.
- Video Management System using IP enabled video capturing devices has become an integral part of Surveillance industry today.
- a basic requirement of this type of systems is to input compressed video streams from multiple cameras and record the video in storage devices.
- DVR and then NVR were predominant components, the complexity and hence the challenges for efficient deployment of the system were less. This is because each DVR or NVR was a standalone system taking feed from a handful of cameras (typically 16 or 32), and used their dedicated local storage devices to record the video.
- Video Management System emerged as a solution.
- each server catering a set of Video Capture devices (e.g., Cameras), one or more network accessible RAID configured storage devices, and multiple workstations.
- Each server now needs to handle 64 or more cameras, stream the video from the cameras to the client machines.
- the servers are grouped into one or more clusters and one or more redundant servers are kept as standby per cluster so that they can back up the functionalities of the failed server(s). This has the disadvantage of non-optimal use of the server resources, both under normal scenario as well as when one or more servers fail.
- one or more dedicated fail-over (sometimes called mirror) servers are often deployed in prior art.
- Dedicated fail-over servers remain unused during normal operations and hence resulting in wastage of such costly resources.
- a central server process either installed in the failpver server or in a central server is required to initiate the back-up service, in case a server stops operating. This strategy does not avoid a single point of failure.
- a present invention thus proposes a fail-safe mechanism without a centra/ server and support from any dedicated failover or mirror server.
- a particular data source e.g., a camera and other sensors
- data e.g., video or other data types
- it is allocated to a 'Server group' with multiple servers in the group.
- the members of the group continuously and mutually exchange their capacity information amongst themselves and automatically share the load according to their capacity.
- the team members automatically detect it and share the load of the failed server(s), without any central control or without support from any fail-over or mirror server.
- This eliminates the need for costly failover or mirror server and the load is always evenly distributed as per the capacity of the individual server hardware. This is a clear advancement in the related art. This can be implemented as an example of cooperative social networking implemented in machine level.
- a recording server when introduced in the system, announces its presence and auto- registers itself to the Video Management Server.
- a database entry is created with the Server ID.
- the server gets the list of network accessible storage devices (typically NAS or SAN) from the database and is thus prepared to record data once one or more data sources (viz, cameras) are added to the server.
- the recording is done by breaking up the video stream into chunks or clips of small duration (typically 2 to 5 minutes), and the clips are initially stored in the local server storage space. Periodically, the clips are uploaded to the NAS/SAN using all the IMAS/SAN in a round robin fashion.
- the administrator of the system can form several "Server groups" by first forming a GROUP and then assigning any server to that GROUP.
- All servers are assigned to the DEFAULT group. As soon a server registers itself, it starts multicasting a message describing its IP-address, group-ID and remaining capacity to handle more cameras.
- the capacity is represented with a number. The number is calculated based on the memory, bandwidth and current processor utilization within the server, or it can be set by the administrator to be equal to the number of cameras the server should handle, and the number is decremented or incremented when a camera is added or removed from the server, respectively.
- Video Management Server and all other recording servers within the GROUP listens to all such messages and maintains a list (LIST), as described below [taking example for 4 Video Recording Servers (VRSes)]
- FIG. 13 a top level flow diagram of the intelligent colour object (moving, static, transient) analysis is shown by way of components/features 1301 to 1309.
- figure 15 there is illustrated a process flow diagram for unified computationally adaptive colour appearance correlation based predictive background estimation by way of components/features/stages 1501 to 1505.
- figure 16 there is illustrated the manner of removal of shadow and glare regions by way of components/features/stages 1601 to 1607.
- the goal of foreground object extraction is to divide an image into its constituent regions which are sets of connected pixels or objects, so that each region itself will be homogeneous with respect to the different physical objects whereas different regions will be heterogeneous with each other.
- the foreground object extraction accuracy may determine the eventual success or failure of many sub-sequent techniques for video analytics and object recognition, and object based different event-detection.
- nature of the shadow and glare can be static, moving or both. Static or very slowly moving shadow and glare can be mode/fed by some background estimation techniques. But moving shadows and glares that are associated with moving objects are hard to model and eliminate from being detected. Hence effective identification of shadow and giare regions and elimination of those regions from actual foreground objects remain to be challenging and important for any video analytic applications.
- shadow and glare are detected using fixed thresholding methods where a set of fixed and trained thresholds are used to detect the shadow and glare regions.
- these fixed thresholds are derived by observing the variation of pixel intensity over video frames due to presence of shadow and glare in a specific type of scene, so their applicability is limited to that type of scene only.
- Some techniques improve the . fixed thresholding approach by introducing a estimation of shadow and glare thresholds to make them adaptive, but till either they are very specific to type of the scene or they require a lot of computations.
- Another type of shadow detection approaches applies scene knowledge based object-wise shadow regions identification. These approaches use a scene knowledge (e.g. difference of shape, size, colour etc.
- ii More specifically, it is a sequence of processes of the presented method which provides more accurate information of colour objects in an image taken from any video sequence by low cost cameras. Any sequential video images can be processed with this method to locate all possible detectable colour objects and their related information which can be further be processed to analyze the scene dynamics with respect to the object itself and ih association with other foreground objects. The extracted information can be used to measure any statistical information regarding the object or association of the colour object with any other animate or inanimate colour objects in the scene. iii.
- the proposed method provides improved colour background information by eliminating the defects encountered in the prior state-of-art in presence of video noises like spatial movement of non-meaningful objects, change of appearance of colour due to presence of shadow, change of appearance of the colour in the object when it moves to a low intensity (darker) region from a higher intensity (brighter) region and vice versa.
- the technique is also adaptive when the colour appearance of the foreground objects and background of the scene changes frame to frame due to change in global intensity or other phenomena such flickering, sensitivity of the sensor in the camera, etc.
- the proposed object analysis technique is also capable of detecting and characterizing static objects along side with colour moving objects in the same scene by a novel unified framework based on multi-layer estimation technique. Instead of tracking the position of the objects to locate the static objects, it estimates the possible foreground object pixels that may belong to any static object in the scene and then generates static objects from detected static pixels.
- the proposed multi-layer static foreground pixel estimation technique overcomes the inability of any traditional background estimation technique to distinguish the background pixels from the foreground pixels that remain static for a long duration.
- the multi-layer approach also gives much more control over the process of distinguishing the static foreground pixels from the background.
- the present invention thus also discloses advancement in the process and an intelligent unified framework for colour object analysis in a scene in order to develop efficient video analytics applications and other intelligent machine vision technologies.
- the overall framework comprises of several novel approaches to develop underlying tasks to accomplish this.
- One such task is an adaptive process for accurate and predictive technique for colour coherent background estimation.
- the technique relies on colour correlation of neighboring pixels and inter-frame colour correlation under severe natural conditions such as shadow, glare, colour changes due to varying illumination, and effect of lighting condition on colour appearance, electronics generated induced noises (e.g. shot noise, but not limited to).
- the developed technique is adaptive to the content in the scene and their features such as colour variation, complexity of the scene, motion activity, as well as naturally induced noise in the scene. Because of the adaptive nature of the proposed technique, it can handle minor vibration in the scene because of vibration of the camera.
- the underlying philosophy of the proposed method is to use the red, green, and blue components as a composite data and use the relative values of these components to maintain accurate colour information and appearance of the true colour in the estimated background frame. It should be noted that we have exemplified the present invention in terms of Red-Green-Blue colour space. But the underlying philosophy is not restricted to this particular colour space only. Variation of the concept can be adopted in other colour spaces as well.
- the present invention also disclose a method of distinguishing and eliminating shadow and glare regions from video frames to minimize erroneous foreground estimation in order to reduce unnecessary false alerts due to wrongly interpreted events using wrongly detected objects in a video analytic application. It is achieved using image characteristics driven adaptive and dynamic threshold generation technique.
- the technique requires very low computation due to use of a look up table that characterizes shadow and glare in various environments.
- the outcome of this technique is a set of accurate foreground pixels that are grouped together to construct foreground objects in the scene. These objects are further characterized, classified, and tracked to detect meaningful events in the scene.
- the present invention enable characterize, classify and generate some basic information of these detected static and moving objects such as their position, size, type, temporal information such as when it first appeared in the scene, duration of appearance in the scene, whether it is occluded, and if so the duration of occlusion, etc. Using this information, we can infer certain activities or events in the scene using a rule-engine applying different logic depending upon desired video analytics applications.
- Stage A A novel technique for removal of shadow and glare from the scene (Stage B) is proposed.
- the proposed method also removes electronics induced different type of noises prevalent in any electronic sensor based camera, as well as handles small vibration of cameras.
- Stages C and D characterize the pixels in the foreground regions and extract both moving and static objects.
- Static objects can be of two types - (1) a new static object appeared in the scene and remained static for long duration of time so that it does not become part of the background due to non- movement for a while, (2) objects nearly static with very small movement but not part of the background either.
- the invention involves a unique method for stage A by adapting the computational steps based on the variation of light intensity and its effect in colour appearance in each image region or image pixel rather than using same computation blindly in all the pixels across the scene as in prior art.
- each colour plane is processed independently without keeping into consideration of the relation between three primary colour components red (R), green (G) and blue (B).
- R, G, B components as a composite single structure in a unified manner to preserve the mutual relationship of these colour components in each individual pixel in order to maintain true colour appearance in the estimated colour background frame.
- the framework continuously readjusts its modeled or predicted values for each colour pixel in a frame with all sequential forthcoming frames of the colour video.
- the present process skip estimation of the colour background in that pixel location since this pixel colour does not contribue to the background. Otherwise, we compute an adaptive size (k * h, k* w) local window centering around this pixel for computation of the background estimation using the colour pixel values within this window, where Avs(h w)
- the processing window size reduces with the reduction of intensity in the region surrounding the pixel.
- each colour cluster k consist of a mean representative colour pixel value ⁇ R ,M G , B ) k witr > s P an of colour deviation ⁇ a R ,a G ,a B ) k and a number of appearance (o k ) of a colour pixel in this cluster.
- a colour pixel (R,G,B) is matched with the colour cluster k, if the difference between each colour component in pixel (R,G,B) with the corresponding representative colour component ( ⁇ ⁇ , ⁇ ⁇ , ⁇ ⁇ ) / ⁇ ⁇ cluster k, i.e. ⁇ ⁇ - ⁇ ⁇ , ⁇ ⁇ - ⁇ ⁇ , and
- the colour background reference frarne is constructed from representative colour values of the generated clusters. If matched colour cluster has significantly high occuerance relative to the overall population occuerance then the representative colour of the colour cluster is used as the value of the colour pixel in the colour background refence frame.
- Stage B Removal of shadow, glare, and sensor generated noises
- thresholds need to be adaptive and dynamically also need to be generated depending on scene environment.
- a . way to mode/ the scene environment is to express the scene environment in terms of some image characteristics parameters and then model those parameters. These image characteristics parameters are like illumination, sharpness etc. as shown in Figure 16.
- An advanced approach has been presented here to remove shadow and glare in background and foreground segmentation process for dynamic scenes using image characteristics based adaptive thresholds. It has been observed that it removes various sensor generated noises as a by-product of the approach that we adopted.
- the sharpness parameter of the image is computed as follows:
- Every row of the input frame is filtered with a high pass filter.
- the average of the filtered values of the overall image is considered as horizontal sharpness parameter
- Every column of the input frame is filtered with the same high pass filter.
- the average of the filtered values of the overall filtered image is considered as vertical sharpness parameter S .
- Maximum of S H and ⁇ 5y is the sharpness parameter (S) of the image
- V is used to characterize the scene.
- R(x,y) — I(x ⁇ , y)3 ⁇ 4 + R(Lx,_y-) , where I(x,y) and (x,y) are the input pixel value and reference background pixel value in a colour plane.
- I(x,y) and (x,y) are the input pixel value and reference background pixel value in a colour plane.
- stage B As image characteristic measurements (from B.2) and their thresholds (from B.l) for shadow and glare are available, the shadow and glare pixels are identified by comparing these measurements with the corresponding thresholds for each pixel of the image. Once shadow and glare pixels are identified, any contribution of those pixels iri the final gray difference image is nullified by setting zero to those pixels in gray difference image. For rest of the image pixels in the image (i.e. other than shadow and glare pixels), maximum intensity difference value is put in gray difference image for respective position.
- stage B The fringe benefit of application of stage B is it also handles and filters out sensor generated noises inherent in any electronic circuit system, shot noise due to rise of temperature of the sensor, as an example. Another fringe benefit of application Stage B is that it also handles small natural vibration of the scene due to vibration of the camera.
- V indicates thresholds (77?) are independent of the calculated values ⁇ V) and possess a fixed quantity which may be a single value or a range.
- the proposed estimation process is computed to estimate static foreground pixels.
- static foreground pixel we mean the pixels which has been found not belonging to the background of the scene, but the characteristics show they possibly belong to a foreground object which has no meaningful motion during last few frames, e.g. an inanimate static object which has been introduced to the scene in last few frames.
- foreground modelling a new concept of "foreground modelling” technique has been applied and its readjust procedure is done by a selective method.
- Working principle of this "foreground modelling” technique is similar to the previously described "colour background estimation technique” that has been computed and described in stage A.
- ⁇ the occurrence parameter ( ⁇ ) of the modeled colour clusters is continuously reduced forcefully in the estimation process for all the pixels belonging to regions where no foreground has been formed for a certain interval of time (i.e., in last few frames).
- ⁇ the occurrence parameter of the modeled colour clusters is continuously reduced forcefully in the estimation process for all the pixels belonging to regions where no foreground has been formed for a certain interval of time (i.e., in last few frames).
- This two-level (multi-level) hierarchical estimation technique is novel and gives the benefit of detection and analysis of not only moving objects in the scene, it also detects the static objects for small duration as well as static objects for long duration. As a result, we achieve more accurate object extraction result without consuming a static object in the scene to become part of the background for a long duration of time.
- Detected foreground regions are now segmented using suitable image processing based object clustering methods and morphological techniques. Each captured foreground component then individually analyzed for their classification purpose. Using typical object shape, silhouette, colour feature, they are categorized into different predefined modeled object(s) for any typical scene. In particular scene like indoor house or building, detected objects are categorized into human and non-human sets; scenes like road segment in any road junction or free highway detected objects are categorized into vehicle, pedestrian; this detected objects were finally associated with previously detected object set of the scene using inter frame overlapping and colour feature based analysis for more generalized information of those objects in the video. The generated object information then transferred to different rule engines for their comparison with different application based pre-determined rules to identify occurrences of any predefined event (s).
- Figure 17 shows a tradition method of face detection using the flowchart in Figure 17 and by way of components/features/stages 1701 to 1706 while Figure illustrates the face detection in accordance with the present invention by way of components/features/stages under 1801 to 1809.
- What is disclosed is an efficient technique to find regions in a video to capture faces of people in motion, limiting the search space using motion detection technique, control the computational requirement based on desired accuracy of capturing faces.
- This technique can be used to capture faces from real time video where the accuracy of the operation can be controlled depending on the computational bandwidth available in the system.
- Extraction of particular types of objects in images based on fiduciary points is a known technique.
- computational requirement is often too high for traditional classifier used for this purpose in the prior art, e.g., Haar classifier.
- a novel method is proposed to enhance the efficiency of extracting face regions from a sequence of video frames.
- the number of iterations and pixel shifts as required in the proposed technique is controlled with the help of a look up table. This helps in striking a balance between the computational requirement and the accuracy of face detection.
- this novel technique can be used as a cooperative process coexisting with other compute intensive processes.
- the search space is reduced by considering the motion vector and sliding the window only in the blob regions where motion is detected .
- the average time t to analyze an image in host machine is calculated, and for subsequent frames pixel-shifts and number of iterations are calculated based on two lookup tables, to suite the computational bandwidth.
- An advanced technique is proposed in this disclosure so that the search space is significantly reduced by considering the motion vector of the moving objects only and applying the proposed novel algorithm in the regions represented by these motion vectors only.
- This reduced computation enables to process larger resolution video imagery to advance the face detection systems in today's era of increasingly growing demand of higher resolution surveillance cameras.
- several parameters can also be dynamically adjusted so that detection and capture of face of people in motion can be done with varying accuracy depending upon the computational bandwidth available at any point.
- the present invention involves advanced and enhanced the technology by incorporating advanced features as follows in order to accomplish effective face capture and detection system with higher resolution imagery with reduced computation requirement.
- the proposed technique of the invention is explained in Flowchart F-2 shown in accompanying Figure 18.
- Haar feature has been used to explain the advancement .
- the estimation of several parameter such as temporal estimation "t”, prediction of possible number of iterations Alteration' in above flowchart is novel and described below.
- ScaleFactor f(M, N, m, n, alteration) niteration
- T f(M, N, t, pixelShift, niteration), for a fixed size window.
- What is disclosed is a method for allocating computing resource and allied resources (e.g, Physical memory) in a computer for Analytics processing on video channels in a multi-channel environment, estimating scene complexity as relevant to the frequency of frame processing, spawning of processor threads based on physical CPU cores, allocation of threads to video channels for Analytics processing based on requirement.
- the video frames are fed to the Video Analytics engine at an fps f ⁇ F, where F is calculated dynamically by the Analytics engine itself depending on its processing requirement. This enables an optimum sharing of resources among multiple channels with constrained resources and also eliminates unnecessary computing.
- the resource requirement for Analytics processing varies to a large extend from one point of time to another during run time.
- a fixed number of Analytics Task processing threads are spawned as a function of number of processor cores present.
- the Threads are kept suspended in a thread pool.
- the channels are allocated/de-allocated to the threads.
- the Analytics engine calculates the optimum FPS requirement as a function of scene complexity.
- the Scene complexity is calculated based on : a. Inter class difference of foreground and background, (i.e. For noisy image scene complexity is high)
- a Controller module coordinates the tasks for multi-channel camera analytics.
- the Controller spawn a number of Analytics processing threads depending on the number of CPU cores present, as available from the system hardware information.
- a Task Scheduler module generates a sequence indicating the order in which the individual channels are to be served for Analytics tasks. If there are 3 channels and there ratio of processing requirement is 1:2:3, then the sequence generated is: 1 3 2 3 2 3 1 3 2 3 2 3 1 3 2 3 2
- the Controller dispatches the frames of different channels, in the order as in the sequence, to the Video Analytics Processing threads as when they are free. After a fixed amount of time, say 1 second, the Controller regenerates the sequence based on feedback from Video Analytics Engine.
- FIG 21 illustrates the video channel join-split mechanism for low bandwidth communications in accordance with the present invention by way of representative components/features/stages under 2101 to 2103.
- the system consists of two components— a Sender and a Receiver.
- the Sender and Receiver are to be used in pair, the former installed at the multi-camera site to join and compress the video streams in a single channel video, and the later at the Client side to receive the video and extract the individual channels for the purpose of viewing live, recording or retransmitting.
- the bit rate of the compression at the Sender's end is adaptable to the available network bandwidth of the network path connecting the server and the client.
- Video surveillance or video chatting domain is characterized with transmission and receiving of videos from one site to another.
- An IP-network is often used as the transmitting channel— wired or wireless.
- a WAN network is often used in between the communication path between the sender and the receiver of the video channels.
- the varying and sometimes low bandwidth of the WAN network may not be sufficient for transmitting the multiple channels on-line individually in the form as they are received from the cameras.
- the problem is enhanced when MPEG4 or H264 video compression is used inside cameras, as the video bandwidth consumption is very much video content sensitive in those cases.
- the underlying data compression algorithm is intelligently handled without affecting the decoding process with a standard equivalent decoder.
- the motion vector generation step in the underlying MPEG type compression is intelligently controlled, so that no motion vector crosses-over the inter-frame boundary in the combined frame. This eliminates interference between any two channel data frames in the combined frame.
- This technique of bandwidth adaptive multi-channel data transfer without inter-channel interference is novel and unique.
- a module in accordance with the present invention has been developed which combines multiple video channels into a single combined stream and encodes the stream with variable bit rate depending on the available bandwidth from the Server to the Client.
- the individual video stream may have varying formats (one with MPEG4, another with MJPEG, etc).
- a frame header is transmitted with each frame of the combined video stream.
- the frame header contains metadata about the constituent streams.
- a receiver at the receiving end splits the combined video stream into constituent video streams based on the frame header Information.
- Sender module The video from multiple cameras are received and decoded individually to get the RAW frames. If the video is available in RAW form itself then this step is skipped for that channel.
- the RAW frames as and when available from the individual decoder, are kept in memory, overwriting the existing frame; each channel has a dedicated space in memory for that.
- an initial fps (0 is determined. As for example, if ft is for (ive viewing the client may request for an fps of 10.
- a Sampler module takes the current frame from the channel specific memory area at a fixed rate, f, for those channels and combines them into a single frame.
- a lookup table is created to store the channel ID and its boundary within the combined frame.
- the frame is then compressed in MPEG4 or to any other similar format as desired using a default bit rate.
- the set of motion vectors generated as part of the compressing technique is then checked to identify all such motion vectors which cross the inter-frame boundary. All such motion ⁇ vectors are forcibly set to null to ensure that the video content of one constituent frame (within the combined frame) does not contribute in deciding the content of another constituent frame, and thus avoiding inter-channel interference.
- a frame header is composed with metadata information about the position of the individual channel frames within the combined frame, the resolution of the individual frames, and a timestamp.
- the receiver module open a TCP connection with the sender and requests for all or selective channel video. It can also specify the format for compression. Additional commands to get the existing channel information, the resolution of the channels, the fps of the individual channels at the senders end, etc are available to facilitate the client in selecting the channels of interest and specifying other parameters as the transmitting fps (f), initial bit rate etc.
- bit rate controller At the server end prepares the encoder for new bit rate, flushes the transmission queue and responds to the client with the new bit rate as set.
- the Client reacts with clearing its own session and prepares itself to receive video with new bit- rate.
- the accompanying figure 21 clearly illustrates the above discussed Sender module & Receiver module.
- Object tracking systems are used to detect the presence of any moving object in a sdene and track the object to distinguish it from other similar objects in the scene and also to record the trajectory of the object.
- Video data of the scene as captured by a fixed camera is analyzed to detect and track moving objects.
- this requires the background to be stable and the camera should cover the whole region where the trajectory is to be formed. This has the side effect that the size of the object in the camera view becomes small, particularly when the object is far.
- PTZ Camera based Tracking Systems are used where A PTZ camera is used to automatically track the object and zoom on the object so .that the detail features of the object is visible in the video frames.
- traditional PTZ based tracking system suffers from some major drawbacks and is not deployable in a real life video, particularly when the video is infected with noises like shadow, glare, electronic noises etc.
- One of the reasons is the inability of such systems to form a good reference background frame.
- the system is non adaptive to demographic and environmental variations.
- PTZ camera when PTZ camera starts tracking an object, it loses the visibility of other parts of the scene. Therefore, some important scene event may be missed while the PTZ camera tracks one of the objects. This may encourage miscreants to fool the system.
- the accuracy of detection and tracking of objects is also very low, as there is no fixed background while the tracking is in progress and the foreground objects are to be extracted based on motion detection or some modified version of the method or using some modified version of object extraction technique from still images. In case of some tracking error, which is likely to occur when the speed of the object in the scene is high or random, the system cannot recover from this error state in a short time, as it loses visibility of the object.
- an Object tracking system is used in conjunction with one or more PTZ cameras.
- the object tracking system tracks the object and pass on the positional information of the object along with a velocity prediction data to the PTZ camera controller in a periodic manner. If more than a single object is detected, one object is taken at a time for handling based on some criteria (viz, the priority of the zone where the object appeared, the duration of the object in the scene etc.).
- a PTZ camera controller receives the positional information of the object periodically and estimates corresponding position of the object in the PTZ camera view using a novel Scene Registration and coordinate transformation technique.
- the P, T and Z values are set by the Controller such that the object remains nearly at the center of the PTZ camera view and is sufficiently large.
- the proposed system enhances the functionalities and utility of a traditional Object tracking system and at the same time eliminates the drawbacks of a standalone PTZ camera based tracking mechanism.
- This concept and implementation technique is novel and unique. The concept can be extended to develop a system to handle multiple objects in parallel with the more than one PTZ cameras. Also, trigger from multiple fixed cameras can be received to develop a system with multiple fixed cameras and multiple PTZ cameras together to cover a wider range in the scene, or to enhance multiple Object tracking systems over a single framework.
- Fig. 22 thus shows an embodiment of the enhanced object tracking system.
- a weighted interpolation technique is used to map the bounding rectangle of an object visible in the Static camera view to the corresponding Rectangle in the PTZ camera view.
- the technique requires as input a set of points (A, B %) spread uniformly over the static camera view and their corresponding positions in the PTZ camera view. This can be done by the user while configuring the system.
- Fig. 23 Illustrates the Coordinate Transformation involved in the present invention enhanced object tracking.
- a and B be any two such points in the static camera view as marked by the user, and let A and B be the corresponding mapped points in the PTZ camera view as also marked by the user.
- any arbitrary point (C) in the static camera view is mapped to the corresponding point (C ) in the PTZ camera view dynamically, using the following method:
- an estimate of x-coordinate of the same point C is calculated for all pair of points (A, B) in the Static camera view.
- the y-coordinate C y is calculated for the point C.
- FIG. 24 illustrates in detail an intelligent and automatic traffic enforcement system built in accordance with the advancement of the present invention including components/ features/ stages 2401 to 2409 in Figure 24 ,2501 to 2512 in Figure 25,2601 to 2605 in.figure 26, 2701 to 2704 in figure 27 ,2801 to 2818 in figure 28.
- Traffic signal violation is a burning traffic enforcement issue throughout the world. Beyond optimistic illusions, ground realities are too fierce to be accepted, as the fearsome road accident, traffic jam are the main effect of the same. Seeds of improvement are however being planted at all possible arenas but they are very costly and high human resource consuming too.
- the proposed system describes an Intelligent Automated Traffic Enforcement System.
- Road transportation department requires intelligent automatic enforcement system for the surveillance in each traffic junction and for the on-field enforcement team, allowing them to book offences and access other Transport department application's events in real time.
- the present advancement is targeted a the following:
- Smart phone solution for the on-field enforcement team allowing them to book offences and access other Transport Department Application's events via GPS / GPRS enabled Mobile / Handheld devices.
- the additional data center hardware set-up for Road Transportation Department to store evidence / archive data for all the relevant events. Connectivity management in real time by data transfer between the above components to ensure synchronized communication.
- the proposed intelligent automated traffic enforcement system of the present invention can help the traffic management department to identify the violation by traffic department personnel by remotely observing the video feeds coming to the control room from the junction through computer monitor. Alternately, it can be automatically detected by our proposed system and automatically alert a traffic personnel without physically being present at the traffic junction or sitting in the control room.
- Videorietics proposed system does not require any specialized or proprietary camera to detect these violations. It analyzes video feed from traditional security cameras in a computer to detect the events. Security cameras are installed at strategic locations around the traffic junction in such a way so that video analytic engine can capture and process the video to detect the violating vehicles, automatically find the identity of the vehicle such as Number Plate, shape, size, color, logo, type of the vehicle, and possibly the snapshot of the driver if visible.
- the engine then automatically stores these information and images of those vehicles in event log database.
- the traffic inspector can identify possible violations like red light violation, over speed vehicle, wrong way vehicle, vehicle rider without helmet, without wearing seat belt, using mobile phone while driving, motorcycle with more than two passengers, etc. either by automated video analytic application or manually through computer monitor. Images can be manually tagged with comments by the traffic personnel or automatically tagged with possible violation type, and can be manually or automatically sent to handheld devices of on-field enforcement team through communication network for subsequent physical action and are also kept in database for future use.
- Exemplary illustrative components of The proposed solution :
- the proposed solution consists of SEVE major COMPONENTS.
- OPD - Engine Object presence detection engine
- NPR Number Plate Recognition
- G(x, y) in pixel coordinate (x,y).
- A(x,y) is the average of all the pixels in a 2-dimensional window of size (h, w) centring (x,v) ,
- grouped characters can be split into multiple sub-groups. Merge possible sub-groups. Two subgroups are merged if the sub-groups fall in a horizontal line (case of split group) or vertical line (case of multi-line number plate).
- Novel Number Plate localization Algorithm to localize appearance of a number plate in any part of the video.
- Novel Number Plate localization Algorithm to localize appearance of multiple number plates in different parts of the image for multiple vehicles at a time. 5. Effective with English Alapha-numerical characters independent of the font, size, style, and color of the characters.
- Lighting condition independent - Works in Day and light condition with sufficient illumination of any type of light (neon, fluorescent, IR, etc.)
- OCR algorithm independent The localized number plate region can be processed by any OCR device or algorithm.
- Processing of the type of vehicle, color of vehicle, logo, make of vehicle, silhouette of the vehicle, possible driver snapshot, all can be processed in real time.
- FIG. 26 An illustrative top level system overview for such traffic surveillance system is shown in accompanying figure 26.
- the proposed system thus comprises of two main modules viz. Video Surveillance
- the Video Surveillance System facilitates monitoring using security cameras in traffic junctions.
- the videos feeds can be displayed in the control room for monitoring.
- the video feeds are continuously and automatically recorded, indexed, and properly archived in databases.
- the time of recording is configurable at administrator level. It is typically configured inline with the operation shift / day shift.
- the Video Analytics Application supports various functions as shown in the figure below. Each function consists of various use cases of incident detection and managemen
- the video Analytical Process flows in a sequence starting from Configuration - Incident Detection - Incident Audit - Reporting - Synchronization - User Management.
- Figure 27 illustrates a schematic diagram of the various features in such traffic surveillance system of the invention.
- Figure 28 is a detailed breakdown illustration of the video analytics application for the purposes of traffic surveillance and violation detection and registration and follow-up actions.
- system and method of traffic surveillance and violation detection and action is adapted to facilitate configuring the parameters for incident detection and management in following manner.
- Camera configuration Add cameras to the configuration server with a high resolution image for detailed information. Start applicable application with event configuration.
- Virtual Loop For each camera in the junction/ free way, a zone which is to be monitored is defined using this parameter. This is configured before starting the system operations and only once. However the rights of modification are available for administrator user level. The camera is always focused on the zone and it keeps on capturing the videos of the "marked" zone. The zone is marked so as to capture the maximum of the traffic in one direction. For each camera a zone is defined separately. A typical configuration is shown in figure 29
- Time Limit The application facilitates defining the working hours and / or nonworking hours for the purpose of recording the videos.
- the rights of modification in these time limits are available at administrator level.
- the system captures and records all the videos from the junction / free way cameras during working hours. It captures all the videos and archives the offences detected during non - working hours.
- Traffic Direction To detect the vehicle(s) moving in the wrong direction, the application facilitates defining the regular traffic moving direction for each camera with minimum 10
- Speed Limit To detect the over speeding vehicles crossing the zone, the application facilitates defining maximum allowable speed limit for the vehicles. An incident is generated on detecting the vehicle crossing the speed limits (not clubbed with Red light camera).
- Sensitivity & Duration To detect the traffic congestion or vehicle presence crossing the zone (virtual loop), the application facilitates defining maximum allowable vehicle in percentage and the duration (time) for which it should not considered as traffic congestion or vehicle present in a zone (not clubbed with Red light violation detection or speed violation detection camera).
- Each junction has junction cameras for capturing the junction videos lane wise and an I/O module monitoring the status of traffic signal.
- the videos from junction cameras and status of traffic signal are sent to the control room via a dedicated link.
- the analytical application in the control room monitors the change in status of the traffic signal. On detecting the change, it starts analyzing appropriate video and check for an offence happening in the junction.
- the scenario is explained below.
- the figure below shows a typical layout of a 4 way junction.
- the system can operate multiple lane / road which had red signal.
- a junction layout is shown in figure 30.
- the serial number is generated using junction ID, camera ID, Date & Time and sequence number.
- the next consecutive video starting from 10:06 am on the same day will have the video ID as J01CS20110810606000025 as an example. However the format is customizable as required * An illustrative manner of video recording is shown in figure 31.
- the recording module is adapted to also display message in case any error is found while playing the video or receiving the video from the camera.
- the connectivity error is also detected and displayed on the screen and stored in the database.
- Trigger The application monitors the status of traffic lights continuously. As the traffic light status is changed, the same is reported to the control room.
- Figure 32 illustrates a transition traffic light status.
- Incident Detection On receiving a trigger from I/O Module, the application starts analyzing the videos. For e.g. When TN is Green, the traffic moves from S - N, S - E or S— W. The traffic in other direction is standstill as the traffic signal is Red. The application checks for following events to detect incidents
- Traffic presence (Vehicle density).
- Incident Display Once the incident (alerts and notifications) is detected, an alarm with visual along with sound effects is generated at operator's workstation or hand held device.
- the alerts and notifications are recorded and stored in the operator's inbox.
- the alert is generated when an incident is detected and a notification is generated after detecting the alert.
- the notification gives details of the incident. It consists of incident type, date and time of incident, junction name i.e. location of incident, camera IP, and a link to the incident image / video for verification.
- the notification is shown on the screen and it is flashed continuously till it is acknowledged by the operator. The operator can accept or deny the notification by verifying the video. On denying the alert / notification it is archived and can be reviewed later.
- License Plate Recognition To register an incident the application request the NPR - Engine to extracts the license plate number (Text) of the violating vehicles.
- Figure 33 illustrates an exemplary illustration of capture number plate.
- Incident audit is ensures correct enforcement by verifying the incidents and vehicle numbers.
- the application keeps on raising the alarms for incidents.
- the operator is sitting in the control room or via handheld device audits these incidents by verifying with the video / images.
- the audit is carried out in following sequence:
- the operator selects an incident by applying suitable filters if this is an archived incident. For a live incident he double clicks on the record to view the details.
- the system shows details of the incident, a link to incident video and a link to license plate image of the vehicle.
- the operator verifies the incident by playing the video and vehicle's registration number by viewing the license plate image.
- Incident status is changed from "Pending” / "Acknowledged” to "Audit” and it is saved into the database.
- the operator enters the remark about the action taken while auditing the incident.
- the remark is saved in the database for future reference.
- the operator Before saving the changes the operator is warned for re-verification of his inputs. He previews the video and the license plate number and saves the audited transaction in the database.
- Figure 34 is an illustration of an incident audit view generated by the system of the invention. Reports
- the above The traffic surveillance system application in accordance with the invention further facilitates generating various reports including as below: Incident Details Report:
- the report shows details of all incidents occurred during selected time slot, for selected junction.
- the report portrait various details about the incidents including junction name, type of incident, offence vehicle, date & time of occurrence etc.
- the report can also be generated on hourly, daily, weekly and monthly basis.
- Incident Summary Report The report shows incident count for selected time and junction. The count is provided for each ' type of incident. The report can also be generated on hourly, daily, weekly and monthly basis.
- Offence Report The report shows the details of a particular incident, with license plate image. The report is generated by providing vehicle number, date and time details and junction name.
- the analytical software stores the data into the database and provides access to the external application (such as Mobile application) to pull the required data.
- the external application such as Mobile application
- the Mobile application checks the duplication of records and avoids the same. Administrative Functions
- User Creation and Management The access to the application is restricted using user name and password for each system user.
- the user names and information is registered into the system and each registered user is provided with a unique user name and password.
- the users are created under defined categories such as operators, supervisors, administrator etc. Access levels for each user category are pre-defined.
- Privilege Assignment Customization of access level is done using this functionality. An administrator can modify the privileges assigned for a particular user category.
- Master Data management This includes entering the data into the system that defines the system Boundaries.
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Computer Hardware Design (AREA)
- Electromagnetism (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- Medical Informatics (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
L'invention se rapporte à un système intelligent intégré conçu pour n'importe quel système d'exploitation et/ou pour un environnement informatique comprenant plusieurs systèmes d'exploitation qui acquièrent en continu des entrées/données obtenues par capteur avec un groupe de serveurs enregistreurs et/ou un groupe de serveurs analytiques permettant l'intégration à sécurité intégrée et/ou l'utilisation optimisée de diverses entrées obtenues par capteur pour différentes applications utilitaires. Les développements apportés par l'invention comprennent : un procédé/système intelligent pour le transfert/l'enregistrement rentable et efficace de données obtenues par capteur avec adaptation de bande à partir d'une ou plusieurs sources de données et à destination de dispositifs de stockage accessibles par réseau ; un procédé basé sur un groupe de serveurs qui possèdent une sécurité intégrée et qui sont autosuffisants, destiné à l'enregistrement et à la diffusion en direct d'entrées obtenues par capteur dans un environnement à plusieurs serveurs ; un procédé intelligent et unifié d'analyse d'objets en fonction des couleurs, de détection de visage dans des images vidéo et autres ; l'affectation de ressources pour un traitement analytique impliquant un environnement multi-canal ; un mécanisme de réunion et de séparation multi-canal conçu pour une liaison réseau à bande passante étroite et/ou variable ; la poursuite améliorée des objets polychromes et/ou monochromes ; et un système d'application des règles de trafic intelligent et automatisé.
Priority Applications (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CA2824330A CA2824330C (fr) | 2011-01-12 | 2012-01-10 | Systeme base sur un serveur intelligent integre et procede/systemes concus pour faciliter l'integration a securite integree et/ou l'utilisation optimisee de diverses entrees obten ues par capteur |
| GB1314003.3A GB2501648C2 (en) | 2011-01-12 | 2012-01-10 | An integrated intelligent server based system and systems adapted to facilitate fail-safe integration and/or optimised utilisation of various sensory inputs |
| SG2013053624A SG191954A1 (en) | 2011-01-12 | 2012-01-10 | An integrated intelligent server based system and method/systems adapted to facilitate fail-safe integration and /or optimized utilization of various sensory inputs |
| IL227378A IL227378B (en) | 2011-01-12 | 2013-07-08 | A server-based intelligent integrated system and method/systems adapted to facilitate fail-safe integration and/or optimal utilization of a variety of sensory input |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| IN33KO2011 | 2011-01-12 | ||
| IN33/KOL/2011 | 2011-01-12 |
Publications (3)
| Publication Number | Publication Date |
|---|---|
| WO2012095867A2 true WO2012095867A2 (fr) | 2012-07-19 |
| WO2012095867A3 WO2012095867A3 (fr) | 2012-10-04 |
| WO2012095867A4 WO2012095867A4 (fr) | 2012-11-29 |
Family
ID=46507521
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/IN2012/000029 WO2012095867A2 (fr) | 2011-01-12 | 2012-01-10 | Système basé sur un serveur intelligent intégré et procédé/systèmes conçus pour faciliter l'intégration à sécurité intégrée et/ou l'utilisation optimisée de diverses entrées obtenues par capteur |
Country Status (5)
| Country | Link |
|---|---|
| CA (1) | CA2824330C (fr) |
| GB (3) | GB2501648C2 (fr) |
| IL (1) | IL227378B (fr) |
| SG (1) | SG191954A1 (fr) |
| WO (1) | WO2012095867A2 (fr) |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106254360A (zh) * | 2016-08-12 | 2016-12-21 | 北京东方车云信息技术有限公司 | 出车中接单失败原因分析方法、服务端及接单失败司机端 |
| CN106814670A (zh) * | 2017-03-22 | 2017-06-09 | 重庆高略联信智能技术有限公司 | 一种河道采砂智能监管方法及系统 |
| CN107734363A (zh) * | 2017-10-24 | 2018-02-23 | 威创集团股份有限公司 | 一种视频信号的显示方法及系统 |
| CN108235134A (zh) * | 2016-12-21 | 2018-06-29 | 杭州海康威视数字技术股份有限公司 | 图片分析方法及装置 |
| TWI659651B (zh) * | 2017-09-29 | 2019-05-11 | 圓展科技股份有限公司 | 影像物件的追蹤方法 |
| CN110780590A (zh) * | 2018-07-27 | 2020-02-11 | 菲尼克斯电气公司 | 用于为机器的多通道控制提供安全控制参数的技术 |
| CN110989541A (zh) * | 2019-12-26 | 2020-04-10 | 石家庄华诚供热有限公司 | 智能热网综合管控平台 |
| CN111881735A (zh) * | 2020-06-17 | 2020-11-03 | 武汉光庭信息技术股份有限公司 | 一种自动驾驶视频数据的事件分类提取方法和装置 |
| CN112071084A (zh) * | 2020-09-18 | 2020-12-11 | 城云科技（中国）有限公司 | 一种利用深度学习判断违章停车的方法和系统 |
| CN112422618A (zh) * | 2020-09-23 | 2021-02-26 | 广州启盟信息科技有限公司 | 位置感应数据上报方法、装置及系统 |
| US20210092424A1 (en) * | 2019-09-23 | 2021-03-25 | Ati Technologies Ulc | Adaptive framerate for an encoder |
| CN113099183A (zh) * | 2021-04-08 | 2021-07-09 | 天地伟业技术有限公司 | 一种视频的智能分析方法、装置及电子设备 |
| CN113741596A (zh) * | 2021-08-25 | 2021-12-03 | 中国铁路设计集团有限公司 | 一种铁路供配电的运维方法及系统 |
| CN114051120A (zh) * | 2021-10-26 | 2022-02-15 | 远光软件股份有限公司 | 视频告警方法、装置、存储介质及电子设备 |
| CN114518963A (zh) * | 2022-04-21 | 2022-05-20 | 中国航空工业集团公司沈阳飞机设计研究所 | 一种面向机载端系统的边缘信息协同处理方法及系统 |
| US11368649B2 (en) | 2020-02-24 | 2022-06-21 | Axis Ab | Streaming of a live video stream |
| CN116204371A (zh) * | 2022-12-13 | 2023-06-02 | 远峰科技股份有限公司 | 摄像头图像数据流的监控方法及装置 |
| CN117240887A (zh) * | 2023-10-13 | 2023-12-15 | 山东平安电气集团有限公司 | 一种智慧物联网能源管理平台系统 |
| CN117376534A (zh) * | 2023-12-07 | 2024-01-09 | 江西云眼视界科技股份有限公司 | 一种智能设备数据采集系统 |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| BR112020014245A2 (pt) * | 2018-10-31 | 2021-05-25 | Toshiba Mitsubishi-Electric Industrial Systems Corporation | sistema ihm web scada |
| EP3948817A4 (fr) * | 2019-05-29 | 2022-03-30 | Xirgo Technologies, LLC | Génération et rapport d'événement de haute priorité pour système de sécurité basé sur une caméra |
| CN111478820B (zh) * | 2020-06-24 | 2020-10-09 | 南京赛宁信息技术有限公司 | 网络靶场大规模网络环境的网络设备配置系统与方法 |
| CN112102367B (zh) * | 2020-09-21 | 2022-09-06 | 中国电子科技集团公司第二十八研究所 | 一种基于运动模型的视频分析算力实时分配调度方法 |
| CN112114995B (zh) * | 2020-09-29 | 2023-12-12 | 中科安信(山西)科技有限公司 | 基于进程的终端异常分析方法、装置、设备及存储介质 |
| CN112562155B (zh) * | 2020-11-04 | 2023-04-28 | 重庆恢恢信息技术有限公司 | 一种智慧建筑工地区域建筑人员安全工作方法 |
| CN113038153B (zh) * | 2021-02-26 | 2023-06-02 | 深圳道乐科技有限公司 | 金融直播违规检测方法、装置、设备及可读存储介质 |
| CN114827664B (zh) * | 2022-04-27 | 2023-10-20 | 咪咕文化科技有限公司 | 多路直播混流方法、服务器、终端设备、系统及存储介质 |
| CN117201873B (zh) * | 2023-11-07 | 2024-01-02 | 湖南博远翔电子科技有限公司 | 一种视频图像智能分析方法及装置 |
| CN117520096B (zh) * | 2023-11-23 | 2024-05-03 | 广东堡塔安全技术有限公司 | 一种智能服务器安全监控系统 |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030027577A1 (en) * | 2001-08-06 | 2003-02-06 | Metric Systems, Inc. | Wireless communication system control apparatus and method |
| US20030084283A1 (en) * | 2001-09-04 | 2003-05-01 | Pixton Jeffrey Seth | Digital broadcast system |
| US20030126202A1 (en) * | 2001-11-08 | 2003-07-03 | Watt Charles T. | System and method for dynamic server allocation and provisioning |
| US20040252193A1 (en) * | 2003-06-12 | 2004-12-16 | Higgins Bruce E. | Automated traffic violation monitoring and reporting system with combined video and still-image data |
| US20060233426A1 (en) * | 2002-04-12 | 2006-10-19 | Agency For Science, Technology | Robust face registration via multiple face prototypes synthesis |
| US20090015671A1 (en) * | 2007-07-13 | 2009-01-15 | Honeywell International, Inc. | Features in video analytics |
| US20090116692A1 (en) * | 1998-08-10 | 2009-05-07 | Paul George V | Realtime object tracking system |
| US20090202105A1 (en) * | 2008-02-07 | 2009-08-13 | Brisa-Auto Estradas De Portugal, S.A. | Automatic license plate recognition system integrated in an electronic toll collection system |
| US20090219993A1 (en) * | 2008-02-29 | 2009-09-03 | Novafora, Inc. | Resource Allocation for Frame-Based Controller |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2135240A1 (fr) * | 1993-12-01 | 1995-06-02 | James F. Frazier | Appareil de localisation et de lecture automatique de plaques d'immatriculation |
| GB2344205B (en) * | 1998-11-26 | 2003-04-30 | Roke Manor Research | Method of and apparatus for vehicle indentification |
| US6433706B1 (en) * | 2000-12-26 | 2002-08-13 | Anderson, Iii Philip M. | License plate surveillance system |
| IL151025A0 (en) * | 2002-07-31 | 2003-02-12 | Traffic violation processing method | |
| JP4442500B2 (ja) * | 2005-04-15 | 2010-03-31 | ソニー株式会社 | 素材記録装置および素材記録方法 |
| ITTO20060214A1 (it) * | 2006-03-22 | 2007-09-23 | Kria S R L | Sistema di rilevamento di veicoli |
| GB0611561D0 (en) * | 2006-06-08 | 2006-07-19 | Ibm | A validation engine |
| CN101937508B (zh) * | 2010-09-30 | 2012-07-04 | 湖南大学 | 一种基于高清图像的车牌定位与识别方法 |
-
2012
- 2012-01-10 CA CA2824330A patent/CA2824330C/fr active Active
- 2012-01-10 SG SG2013053624A patent/SG191954A1/en unknown
- 2012-01-10 GB GB1314003.3A patent/GB2501648C2/en active Active
- 2012-01-10 GB GB1807429.4A patent/GB2561100A/en not_active Withdrawn
- 2012-01-10 GB GB1807430.2A patent/GB2560832A/en not_active Withdrawn
- 2012-01-10 WO PCT/IN2012/000029 patent/WO2012095867A2/fr active Application Filing
-
2013
- 2013-07-08 IL IL227378A patent/IL227378B/en active IP Right Grant
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090116692A1 (en) * | 1998-08-10 | 2009-05-07 | Paul George V | Realtime object tracking system |
| US20030027577A1 (en) * | 2001-08-06 | 2003-02-06 | Metric Systems, Inc. | Wireless communication system control apparatus and method |
| US20030084283A1 (en) * | 2001-09-04 | 2003-05-01 | Pixton Jeffrey Seth | Digital broadcast system |
| US20030126202A1 (en) * | 2001-11-08 | 2003-07-03 | Watt Charles T. | System and method for dynamic server allocation and provisioning |
| US20060233426A1 (en) * | 2002-04-12 | 2006-10-19 | Agency For Science, Technology | Robust face registration via multiple face prototypes synthesis |
| US20040252193A1 (en) * | 2003-06-12 | 2004-12-16 | Higgins Bruce E. | Automated traffic violation monitoring and reporting system with combined video and still-image data |
| US20090015671A1 (en) * | 2007-07-13 | 2009-01-15 | Honeywell International, Inc. | Features in video analytics |
| US20090202105A1 (en) * | 2008-02-07 | 2009-08-13 | Brisa-Auto Estradas De Portugal, S.A. | Automatic license plate recognition system integrated in an electronic toll collection system |
| US20090219993A1 (en) * | 2008-02-29 | 2009-09-03 | Novafora, Inc. | Resource Allocation for Frame-Based Controller |
Cited By (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106254360A (zh) * | 2016-08-12 | 2016-12-21 | 北京东方车云信息技术有限公司 | 出车中接单失败原因分析方法、服务端及接单失败司机端 |
| US11037297B2 (en) | 2016-12-21 | 2021-06-15 | Hangzhou Hikvision Digital Technology Co., Ltd. | Image analysis method and device |
| CN108235134A (zh) * | 2016-12-21 | 2018-06-29 | 杭州海康威视数字技术股份有限公司 | 图片分析方法及装置 |
| CN106814670A (zh) * | 2017-03-22 | 2017-06-09 | 重庆高略联信智能技术有限公司 | 一种河道采砂智能监管方法及系统 |
| TWI659651B (zh) * | 2017-09-29 | 2019-05-11 | 圓展科技股份有限公司 | 影像物件的追蹤方法 |
| CN107734363A (zh) * | 2017-10-24 | 2018-02-23 | 威创集团股份有限公司 | 一种视频信号的显示方法及系统 |
| CN110780590A (zh) * | 2018-07-27 | 2020-02-11 | 菲尼克斯电气公司 | 用于为机器的多通道控制提供安全控制参数的技术 |
| CN110780590B (zh) * | 2018-07-27 | 2023-06-30 | 菲尼克斯电气公司 | 用于为机器的多通道控制提供安全控制参数的技术 |
| US20210092424A1 (en) * | 2019-09-23 | 2021-03-25 | Ati Technologies Ulc | Adaptive framerate for an encoder |
| CN110989541A (zh) * | 2019-12-26 | 2020-04-10 | 石家庄华诚供热有限公司 | 智能热网综合管控平台 |
| US11368649B2 (en) | 2020-02-24 | 2022-06-21 | Axis Ab | Streaming of a live video stream |
| CN111881735A (zh) * | 2020-06-17 | 2020-11-03 | 武汉光庭信息技术股份有限公司 | 一种自动驾驶视频数据的事件分类提取方法和装置 |
| CN111881735B (zh) * | 2020-06-17 | 2022-07-29 | 武汉光庭信息技术股份有限公司 | 一种自动驾驶视频数据的事件分类提取方法和装置 |
| CN112071084A (zh) * | 2020-09-18 | 2020-12-11 | 城云科技（中国）有限公司 | 一种利用深度学习判断违章停车的方法和系统 |
| CN112422618A (zh) * | 2020-09-23 | 2021-02-26 | 广州启盟信息科技有限公司 | 位置感应数据上报方法、装置及系统 |
| CN112422618B (zh) * | 2020-09-23 | 2023-07-04 | 广州启盟信息科技有限公司 | 位置感应数据上报方法、装置及系统 |
| CN113099183A (zh) * | 2021-04-08 | 2021-07-09 | 天地伟业技术有限公司 | 一种视频的智能分析方法、装置及电子设备 |
| CN113741596A (zh) * | 2021-08-25 | 2021-12-03 | 中国铁路设计集团有限公司 | 一种铁路供配电的运维方法及系统 |
| CN114051120A (zh) * | 2021-10-26 | 2022-02-15 | 远光软件股份有限公司 | 视频告警方法、装置、存储介质及电子设备 |
| CN114518963A (zh) * | 2022-04-21 | 2022-05-20 | 中国航空工业集团公司沈阳飞机设计研究所 | 一种面向机载端系统的边缘信息协同处理方法及系统 |
| CN116204371A (zh) * | 2022-12-13 | 2023-06-02 | 远峰科技股份有限公司 | 摄像头图像数据流的监控方法及装置 |
| CN116204371B (zh) * | 2022-12-13 | 2023-11-24 | 远峰科技股份有限公司 | 摄像头图像数据流的监控方法及装置 |
| CN117240887A (zh) * | 2023-10-13 | 2023-12-15 | 山东平安电气集团有限公司 | 一种智慧物联网能源管理平台系统 |
| CN117240887B (zh) * | 2023-10-13 | 2024-03-26 | 山东平安电气集团有限公司 | 一种智慧物联网能源管理平台系统 |
| CN117376534A (zh) * | 2023-12-07 | 2024-01-09 | 江西云眼视界科技股份有限公司 | 一种智能设备数据采集系统 |
Also Published As
| Publication number | Publication date |
|---|---|
| GB2501648C (en) | 2018-09-19 |
| IL227378A0 (en) | 2013-09-30 |
| GB201807430D0 (en) | 2018-06-20 |
| GB2561100A (en) | 2018-10-03 |
| GB2501648B (en) | 2018-06-27 |
| SG191954A1 (en) | 2013-08-30 |
| WO2012095867A4 (fr) | 2012-11-29 |
| GB2560832A (en) | 2018-09-26 |
| GB201314003D0 (en) | 2013-09-18 |
| GB201807429D0 (en) | 2018-06-20 |
| CA2824330A1 (fr) | 2012-07-19 |
| IL227378B (en) | 2018-02-28 |
| GB2501648A (en) | 2013-10-30 |
| GB2501648C2 (en) | 2018-09-26 |
| WO2012095867A3 (fr) | 2012-10-04 |
| CA2824330C (fr) | 2018-05-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9704393B2 (en) | Integrated intelligent server based system and method/systems adapted to facilitate fail-safe integration and/or optimized utilization of various sensory inputs | |
| CA2824330C (fr) | Systeme base sur un serveur intelligent integre et procede/systemes concus pour faciliter l'integration a securite integree et/ou l'utilisation optimisee de diverses entrees obten ues par capteur | |
| US10123051B2 (en) | Video analytics with pre-processing at the source end | |
| US10565680B2 (en) | Systems and methods for dewarping images | |
| US10249022B2 (en) | Systems and methods for dewarping images | |
| EP2795600B1 (fr) | Système de gestion de surveillance vidéo en nuage | |
| US10212462B2 (en) | Integrated intelligent server based system for unified multiple sensory data mapped imagery analysis | |
| US11082668B2 (en) | System and method for electronic surveillance | |
| US20120207356A1 (en) | Targeted content acquisition using image analysis | |
| US20110109742A1 (en) | Broker mediated video analytics method and system | |
| KR101964683B1 (ko) | 스마트 영상처리장치 및 그 장치의 구동방법 | |
| Zabłocki et al. | Intelligent video surveillance systems for public spaces–a survey | |
| US20150085114A1 (en) | Method for Displaying Video Data on a Personal Device | |
| US20190370559A1 (en) | Auto-segmentation with rule assignment | |
| WO2013131189A1 (fr) | Analytique vidéo à base de nuage avec post-traitement au niveau de l'extrémité source. | |
| US20230412769A1 (en) | Scalable Visual Computing System | |
| CN110505438B (zh) | 一种排队数据的获取方法和摄像机 | |
| US20210397849A1 (en) | Systems and methods for detecting patterns within video content | |
| KR102421043B1 (ko) | 영상처리장치 및 그 장치의 구동방법 | |
| US20240144796A1 (en) | Methods and systems for routing media | |
| KR20150121773A (ko) | 지능형 분산 영상 분석 시스템 및 방법 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 12734272 Country of ref document: EP Kind code of ref document: A2 |
|
| ENP | Entry into the national phase |
Ref document number: 2824330 Country of ref document: CA |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| ENP | Entry into the national phase |
Ref document number: 1314003 Country of ref document: GB Kind code of ref document: A Free format text: PCT FILING DATE = 20120110 |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 1314003.3 Country of ref document: GB |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 12734272 Country of ref document: EP Kind code of ref document: A2 |