FR2932054A1 - Intelligent videosurveillance system reconfigurable dynamically - Google Patents

Intelligent videosurveillance system reconfigurable dynamically Download PDF

Info

Publication number
FR2932054A1
FR2932054A1 FR0803052A FR0803052A FR2932054A1 FR 2932054 A1 FR2932054 A1 FR 2932054A1 FR 0803052 A FR0803052 A FR 0803052A FR 0803052 A FR0803052 A FR 0803052A FR 2932054 A1 FR2932054 A1 FR 2932054A1
Authority
FR
France
Prior art keywords
video
camera
characterized
system according
remote server
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
FR0803052A
Other languages
French (fr)
Other versions
FR2932054B1 (en
Inventor
Barz Cedric Le
Marc Leny
Didier Nicholson
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Thales SA
Original Assignee
Thales SA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Thales SA filed Critical Thales SA
Priority to FR0803052A priority Critical patent/FR2932054B1/en
Publication of FR2932054A1 publication Critical patent/FR2932054A1/en
Application granted granted Critical
Publication of FR2932054B1 publication Critical patent/FR2932054B1/en
Priority claimed from US13/218,807 external-priority patent/US8922659B2/en
Application status is Active legal-status Critical
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/18Closed circuit television systems, i.e. systems in which the signal is not broadcast
    • H04N7/181Closed circuit television systems, i.e. systems in which the signal is not broadcast for receiving images from a plurality of remote sources
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19654Details concerning communication with a camera
    • G08B13/19656Network used to communicate with a camera, e.g. WAN, LAN, Internet
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19663Surveillance related processing done local to the camera
    • GPHYSICS
    • G08SIGNALLING
    • G08BSIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00Burglar, theft or intruder alarms
    • G08B13/18Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength
    • G08B13/189Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196Actuation by interference with heat, light or radiation of shorter wavelength; Actuation by intruding sources of heat, light or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G08B13/19665Details related to the storage of video surveillance data
    • G08B13/19667Details realated to data compression, encryption or encoding, e.g. resolution modes for reducing data volume to lower transmission bandwidth or memory requirements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/115Selection of the code volume for a coding unit prior to coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/164Feedback from the receiver or from the transmission channel
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • H04N21/218Source of audio or video content, e.g. local disk arrays
    • H04N21/21805Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H04N21/2365Multiplexing of several video streams
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/238Interfacing the downstream path of the transmission network, e.g. adapting the transmission rate of a video stream to network bandwidth; Processing of multiplex streams
    • H04N21/2385Channel allocation; Bandwidth allocation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/266Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
    • H04N21/2662Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network, synchronizing decoder's clock; Client middleware
    • H04N21/434Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream
    • H04N21/4347Demultiplexing of several video streams
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network, synchronizing decoder's clock; Client middleware
    • H04N21/435Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4621Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/84Generation or processing of descriptive data, e.g. content descriptors
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/18Closed circuit television systems, i.e. systems in which the signal is not broadcast
    • H04N7/188Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position

Abstract

The present invention relates to a dynamically reconfigurable video surveillance system composed of at least one video camera (1), a remote server (3), at least one display terminal (6), means for recording the captured video sequences (5) by the camera (s), said system being characterized in that: the video camera (s) (1) and the remote server (3) comprise means for communicating with one another through a telecommunications network (2); the video camera (s) (1) comprise means for compressing the captured video sequences, the compression ratio being adapted dynamically taking into account the control messages from the remote server (3); the video camera (s) (1) comprise means for analyzing said video sequences in the compressed domain, characterizing the activity detected in their field of view and reporting the result of this characterization to the remote server (3); the remote server (3) comprises means for calculating for each of the compressed video streams from the video camera (s) a command for setting each of them taking into account the received signaling data.

Description

The invention relates to a video surveillance system based on a network provided with at least one intelligent camera and capable of dynamically reconfiguring itself. In the description of the invention, intelligent camera designates a camera comprising means for compressing the captured video sequence. It can also be an analog or digital camera coupled to a compression box. In both cases, this smart camera is also capable of performing an analysis of said sequence in the compressed domain. The intelligent camera is further capable of transmitting the compressed video sequence and the result of the analysis in the compressed domain to a server through a telecommunications network. In the description, the foreground expression is used to designate the mobile object (s) of a video sequence. The background expression will refer to the environment as well as the fixed or quasi-fixed objects of the sequence. This includes, for example, soil, trees or other immobile or not perfectly immobile objects. In addition, the expressions video sequence or video stream designate the same object.

CCTV systems have evolved considerably in recent years. The digitization of the contents and the increase of computer computing capabilities make it possible to process video sequences in real time in order to interpret them. New systems have emerged in recent years and are usually referred to as intelligent video surveillance systems. These systems rely on image processing techniques and / or video for example to compare images, to detect a movement, to detect a face or to recognize an object. A conventional video surveillance system generally comprises the following elements: at least one analog or digital camera for capturing a video sequence; at least one remote server capable of processing the video sequences transmitted by the system camera (s); at least one terminal for viewing and / or storing the video sequences; at least one memory zone for storing the video sequences.

Until recently, the video camera (s) had the main task of capturing and compressing the digital video stream before transmission to the remote server via a telecommunications network. The desire to use more and more efficiently the bandwidth of the transmission media on which these sequences transit and the objectives of reducing the cost of their storage have raised the question of video compression very early on. Conventional compression algorithms reduce the spatial redundancy and time-specific redundancy of a video sequence. These compression techniques make it possible to reduce the bit rate necessary for the transmission of a video stream through, for example, a telecommunications network. In existing video encoding systems, it is necessary to select a compression ratio adapted to the application and therefore to the service in question. Indeed, the more the video stream is compressed and therefore the reduced bit rate, the more the quality of the video as perceived by the user of the service can be degraded. It is therefore important to correctly choose the transmission rate of these flows. There are many methods to achieve this flow allocation. The existing techniques make it possible to adapt the bit rate of the video streams to the bandwidth constraints of the telecommunications networks. As noted above, one of the key components of a video surveillance system is the remote server. Its role is usually to analyze the video stream after decompression. These analyzes, for example the identification of moving objects of a video stream, are traditionally performed at the remote server and not cameras because they require algorithmic tools capable of analyzing an uncompressed video stream. Indeed, the video stream is analyzed at the pixel level of the images, which requires resources in terms of calculations and important memory. It is for this reason that the analysis of the video streams was until recently not conducted by the cameras but remotely on a server with sufficient resources to decompress the streams and analyze them. It is now possible to conduct video sequence analysis in the compressed domain and thus reduce the computation and memory loads necessary for the analysis of a video stream. This method has the advantage of using part of the work done by the video encoder and thus exploiting information available in the compressed domain such as, for example, the coefficients calculated by applying the discrete cosine transform (known under Discrete Cosine Transform - DCT) and motion estimation vectors. This information must then be analyzed. Indeed, the motion estimation vectors do not necessarily correspond to a real movement of an object in the video sequence but can be similar to noise. Using this method it is then possible, for example, to identify the areas of the image with moving objects. The computing load becomes reasonable, the video cameras can take care of the analysis of the video streams. Different steps are needed to use this information to identify moving objects. The resumption of work described in the patent proposal Optical flow estimation method (US2006 / 0188013 A1) made it possible to delineate five functions identified in the article Statistical motion vector analysis for tracking video stream by Marc Leny, Françoise Prêteux and Didier Nicholson . These modules are illustrated in FIG. 1 and described below: Low-Res Decoding (LRD) enables the reconstruction of an entire sequence at the block resolution, eliminating the motion prediction on this scale; the motion estimation vector generator (MEG Motion Estimation Generator) determines meanwhile vectors for all the blocks that the encoder encoded in "Intra" mode (within Intra or predicted images); the low-resolution object segmentation (LROS-Low-Res Object Segmentation) is based on an estimation of the background in the compressed domain thanks to the sequences reconstructed by the LRD and thus gives a first estimate of the moving objects; Motion Based Object Based Filtering (OMF) uses MEG output vectors to determine moving areas from motion estimation; a cooperative decision (Cooperative Decision) is established from these two segmentations, taking into account the specificities of each module according to the type of image analyzed (infra or predicted). The results of the analysis in the compressed domain allow the identification of zones containing moving objects (Figure 2), the generation of motion maps established from the motion estimation vectors (Figure 3) and confidence maps corresponding to the contours of the low resolution image (Figure 4). The main interest of the analysis in the compressed domain relates to the computation times which are considerably reduced compared to the conventional analysis tools. Based on the work done at the time of video compression, analysis times are now 10 to 20 times the real time (250 to 500 images processed per second) for images 720x576 4: 2: 0 .

In a video surveillance system comprising a significant number of video cameras communicating with a remote server through a telecommunications network, the available bandwidth specific to the sizing of the telecommunications network must be shared. A conventional CCTV network architecture relies on an initial network dimensioning that either passes the streams coming from all the sensors or video cameras simultaneously, or takes into account a periodic switchover from one stream to the other. It is then at the monitoring room that the operator or powerful computing and analysis servers can request the visualization of a precise flow according to the importance granted. In these conventional systems, it is common for video streams containing no relevant information to be transmitted from the sensors to the processing servers. In this case, the use of the resources of the telecommunications network used is not optimized.

The object of the present invention relates to an intelligent video surveillance system including the ability to automatically reconfigure itself in order to, for example, optimize the use of resources of said system. This system relies on the use of intelligent cameras having the ability to compress and analyze in the compressed domain the video sequences they capture and to detect specific events. Signaling data is then transmitted over a telecommunications network by the smart cameras to a remote server. This server is able to analyze this signaling data to determine commands for reconfiguring system elements dynamically. More specifically, the subject of the invention is a dynamically reconfigurable video surveillance system composed of at least one video camera, a remote server, at least one viewing terminal, means for recording the video sequences captured by the or the camera (s), said system being characterized in that: - the video camera (s) and the remote server comprise means for communicating with each other through a telecommunications network; the video camera (s) comprise means for compressing the captured video sequences, the compression ratio being adapted dynamically taking into account the control messages from the remote server; the video camera (s) comprise means for analyzing said video sequences in the compressed domain, to characterize the activity detected in their field of vision and to transmit the result of this characterization to the remote server; the remote server comprises means for calculating a command for each of the compressed video streams from the video camera (s), making it possible to parameterize each of them, taking into account the received signaling data. A variant of this video surveillance system is characterized in that a priority level is associated by the server with each of the video streams received from the camera (s) of the system, said priority level being determined according to the signaling information representative of the activity detected and reported by the video camera (s). A variant of this video surveillance system is characterized in that the remote server controls the transmission of the signaling data sent by the video camera (s) of the system by sending requests to adapt the frequency, the type and the content of the signaling messages. A variant of this video surveillance system is characterized in that the compression of the video sequences captured by the cameras is parameterized by a flow rate instruction specific to each camera, said flow rate setpoint being calculated by the remote server as a function of the associated priority level to each of the video streams and is then transmitted by the server to each of the cameras in the system.

A variant of this video surveillance system is characterized in that a priority level associated with the video streams is calculated by each camera, said priority order being determined according to the activity detected by each of the cameras. A variant of this video surveillance system is characterized in that the priority level associated with the video streams is transmitted to the server by each of the cameras of the system and that said server uses this signaling information in order to calculate a rate command transmitted to the cameras in order to that these adapt their compression parameters in order to satisfy this target flow constraint.

A variant of this video surveillance system is characterized in that the priority level of the video stream is used by each camera to calculate a flow setpoint, said flow setpoint being used to adapt the compression parameters of the video stream. A variant of this video surveillance system is characterized in that the automatic switching of the display on the display terminal or terminals is of a duration determined by the order of priority, said duration being all the longer as the corresponding order of priority is important. A variant of this video surveillance system is characterized in that the display on the viewing terminal (s) of a priority video stream is accompanied by an augmented reality mechanism making it possible to highlight the portion (s) of the image containing the unusual activity identified by the smart camera. A variant of this video surveillance system is characterized in that the augmented reality mechanism is controlled by the remote server and that said server accordingly transmits a request to the corresponding intelligent camera in order to acquire the necessary signaling data. A variant of this video surveillance system is characterized in that the portion (s) of the image containing the unusual activity are highlighted by one or more rectangle (s) flashing (s) or fixed (s). A variant of this video surveillance system is characterized in that the portion (s) of the image containing the unusual activity are highlighted by a color highlighting.

A variant of this video surveillance system is characterized in that the highlighting of the portion of the image containing the unusual activity identified by the smart camera is accompanied by a message appearing on the screen of at least a display terminal. A variant of this video surveillance system is characterized in that highlighting the portion of the image containing the unusual activity identified by the smart camera is accompanied by an alarm indicator. A variant of this video surveillance system is characterized in that the compressed video streams are recorded in a memory area of the system and that the control data, such as the priority level of the streams and the metadata from the targeted cameras, are also recorded. A variant of this video surveillance system is characterized in that a supervised mode of the system allows an operator to select a particular video stream that is not necessarily a priority and that the server consequently responds by decreasing the overall bit rate and ordering the selected camera a maximum flow.

The invention will be better understood and other advantages will become apparent on reading the description which follows given by way of non-limiting example and by virtue of the appended figures among which:

Figure 1 shows the modules used for the analysis in the compressed domain as previously described; FIG. 2 illustrates a possible result of the analysis in the compressed domain, namely the identification of zones containing moving objects; FIG. 3 shows another example of the result of the analysis in the compressed domain, namely a motion map established from the motion estimation vectors; FIG. 4 shows another example of the result of the analysis in the compressed domain, namely a low-resolution confidence card corresponding to the contours of the image; FIG. 5 illustrates a variant of the architecture of the video surveillance system according to the invention; FIG. 6 illustrates an example of sequencing of the operations carried out by an intelligent camera of the video surveillance system according to the invention; FIG. 7 illustrates an example of sequencing of the operations carried out by the remote server of the video surveillance system according to the invention;

The video surveillance system according to the invention is based on an architecture, an example of which is given in FIG. 5. The areas to be monitored are covered by means of at least one video camera 1 called the intelligent camera (s) ( s). These smart cameras are capable of at least: - capturing a video sequence of the area they cover; compressing the video sequence (using, for example, the compression techniques described in the MPEG-2 and MPEG-4 standards); to analyze the video sequences in the compressed domain; - to characterize the activity detected in the area covered by the camera. Analysis in the compressed domain allows these smart cameras to characterize activities relevant to the service rendered by the system. The advantage of performing these operations in the compressed domain is to reduce the computation time and the memory resources necessary for the analysis and thus make possible the implementation of these on-board analysis tools within the cameras. It can be realized at low cost, for example, a segmentation of moving objects and a tracking of these objects on video streams. It is then possible to integrate modeling and activity characterization tools into each camera by using, for example, Gaussian Mixture Modeling (GMM) techniques that take into account data from objects. followed (entry and exit points of the field of view, trajectory, speed, acceleration, size, etc.). In this case, an unsupervised learning phase is necessary for each camera, to which can be added one or more supervised step (s) to specify the type of abnormal behavior to be traced back to the remote server in priority. (identification of a portion of sequence by an operator, reconstruction by recorded scenes, etc.).

Relevant activities that can be detected by smart cameras are, for example: - a car traveling counter-clockwise (as part of a traffic monitoring service); a crowd movement, which may be the consequence of a critical event such as a fire; - an individual with a suspicious behavior in a parking lot; a starred departure from a point that can be triggered by any event said to be unusual in relation to a given operation; These cameras communicate to a remote server 3 the video streams and signaling data representative of the eventual activity detected and this through a first telecommunications network 2. These signaling data can be, for example: the number of detected moving objects; the surface covered by moving objects; the speed of movement of moving objects; - the location of moving objects; etc.

This telecommunications network may be, for example, an optical fiber network or a wireless network using the Internet Protocol (known as the Anglo-Saxon Internet protocol - IP). The role of the remote server 3 is to route the video streams it receives from the smart cameras to one or more storage servers 5, one or more control terminals 6 enabling one or more operators to monitor the areas concerned, and this through a second telecommunications network 4. The remote server 3 is further capable of triggering alarm signals 7 remotely. The server can also process the signaling data reported by smart cameras. It can, for example, categorize the video streams received in order of priority and use information to dynamically reconfigure the video surveillance system. The remote server 3 can also manage several tasks related to the (x) station (s) monitoring 6: - The automatic switching from one video stream to another on the monitor with a display of a determined duration according to the priority order. If multiple streams are considered high priority (for example, by thresholding on priority), the server will be able to display a mosaic of these streams by continuing to fail over to the other streams in one of the windows. - The display of a priority stream may be accompanied by an augmented reality aspect by highlighting the portion of the image containing the unusual activity identified by the camera. The type of technique used may be for example: the use of flashing rectangle, color highlighting, etc. This can be accompanied by a message on the screen and / or an alarm light. Everything is done via a request from the server 3 to obtain the corresponding signaling data of the camera concerned. The recording of the streams in a memory zone 5 of the network with their priority levels, possibly accompanied by signaling data from the targeted cameras, for example in the case of high priorities. - The taking into account of a supervised mode: the operator can select a particular stream not necessarily priority according to the camera. The server then decreases the overall rate (higher total priority P) and orders the chosen camera a maximum rate. Etc.

A variant that makes it possible to reduce the transmission rate of the signaling information flowing from the cameras to the server is to go back up to a single priority indicator per camera. Indeed, all the signaling data available at the output of one of the cameras 1 can (if the number of signaling data is high) represent a significant volume of information and it is not always necessary to transmit them. permanently on the network. In another variant of the invention, the video cameras 1 transmit to the remote server 3 a set of signaling information (also called metadata). In this case, it is the responsibility of the server to associate a priority to each stream based on this information and then determine the commands 30 for dynamically reconfiguring the system. The remote server may send an external request to control the transmission of all or part of the signaling data. If the detected activity conforms to the model established by learning and therefore the normalized maximum likelihood is close to 1, the priority will be zero or very low. The further it moves away from the model, the higher the priority order. An encrypted priority between 0 and 1 is thus obtained by 1-MLE (X, O) where X corresponds to the learned Gaussian model, O to the current parameter vector and MLE to the normed maximum likelihood function (MLE referring to Anglo-Saxon term Maximum Likelihood Estimation).

The two alternatives presented above are feasible in the context of the invention (flow priorities determined by each camera of the system or by the remote server), but to illustrate the operations performed by each of the elements with the help of the figures 6 and 7, the example of the priority calculation at the remote server is used in the following description. Figure 6 gives an example of the operations that can be performed by one of the intelligent cameras of the system. The video stream 10 captured by the camera 1 (FIG. 5) is compressed and analyzed in the compressed domain 11. The compression is performed following a target bit rate instruction 15.

The step of encoding, analyzing and characterizing the activity results in: a compressed video stream 12; a set of signaling data (or metadata) 13 resulting for example from the application of the tools for characterizing the compressed stream.

The compression step is further parameterized by a rate setpoint from the remote server 3 and optimized by the analysis in the compressed domain. Indeed, the analysis in the compressed domain can make it possible to segment the images into different areas of relevance. For each of these different zones, different rates can then be allocated to adapt the flow rate of the video stream to the target value. Considering, for example, that the most relevant part of the images of the stream is the foreground, the process may lead to the allocation of a significant bitrate for this part at the expense of the irrelevant part of the stream, in this case the background, which will be allocated a lower flow. It is also possible to intelligently enhance the protection against transmission errors by using the results of the analysis in the compressed domain. The number of redundancy bits can thus be increased for the significant parts of the images carried by the video stream in question.

Finally, the relevant part of the same stream (the foreground for example), can be encrypted for example in privacy concerns, also thanks to the results of the analysis in the compressed domain.

Figure 7 gives an example of the operations that can be performed by the remote server. The example given considers a system where three smart cameras each transmit to the remote server a set of data 21, 24 and 27. These data sets consist, for example, of video streams 22, 25, 28 and associated signaling data. 23, 26, 29 for each of the cameras. This data is then processed by the server to perform priority arbitration between the streams. In other words, the server associates a priority level 31, 32, 33 with each video stream according to the information carried by the signaling data.

The server then uses these different priority levels as well as topology and system-specific information to determine the reconfiguration commands 36, 37, 38 for each of the cameras in the system. These commands are, for example, flow instructions that will be used to adapt the compression parameters of the video stream by each camera. The orders thus calculated must then be transmitted 39, 40, 41 through the telecommunications network to the cameras concerned. As previously explained, the commands calculated by the server may consist of a target rate value. If each camera i is capable of sending a compressed video stream at a specified rate and the server is responsible for managing N cameras (in the example of Figure 7, N = 3) on a network having a maximum throughput constraint D, taking into account the order of priority P; on the flow D; by camera can be done for example according to the formula: P xD It is important to note that if in the example of Figures 6 and 7, the calculation of the priority level of the streams is performed by the remote server, it is also possible, as described previously, to perform this calculation at the level of smart cameras.

It is also possible that each of the cameras calculates its bit rate D. In this case and considering that the total bit rate D does not evolve and is transmitted once and for all to all the elements of the network, the server will return, when the network has to be configured, a single value to all cameras: NP = P. j = 1 1 The cameras can then calculate their target rate according to the previous formula and adapt the compression of the video stream to this value.

Claims (16)

  1. CLAIMS1- dynamically reconfigurable video surveillance system consisting of at least one video camera (1), a remote server (3), at least one viewing terminal (6), means for recording the captured video sequences (5) by the camera (s), said system being characterized in that: - the video camera (s) (1) and the remote server (3) comprise means for communicating with each other through a telecommunications network (2); the video camera (s) (1) comprise means for compressing the captured video sequences, the compression ratio being adapted dynamically taking into account the control messages from the remote server (3); the video camera (s) (1) comprise means for analyzing said video sequences in the compressed domain, characterizing the detected activity in their field of vision and transmitting the result of this characterization to the remote server (3); the remote server (3) comprises means for calculating for each of the compressed video streams from the video camera (s) a command for setting each of them taking into account the received signaling data.
  2. 2- video surveillance system according to claim 1 characterized in that a priority level is associated by the server to each of the video streams received from the camera (s) of the system, said priority level being determined according to signaling information representative of the activity detected and reported by the video camera (s).
  3. 3- video surveillance system according to one of claims 1 or 2 characterized in that the remote server controls the transmission of signaling data sent by the video camera (s) of the system by sending requests allowing adapt the frequency, type and content of signaling messages.
  4. 4- CCTV system according to one of claims 1 to 3 characterized in that the compression of the video sequences captured by the cameras is set by a flow rate decision specific to each camera, said flow setpoint being calculated by the remote server in a function of the priority level associated with each of the video streams and then transmitted by the server to each of the cameras of the system.
  5. 5- video surveillance system according to claim 1 characterized in that a priority level associated with the video stream is calculated by each camera, said priority order being determined according to the activity detected by each of the cameras.
  6. 6- video surveillance system according to one of claims 1 or 5 characterized in that the priority level associated with the video stream is transmitted to the server by each of the cameras of the system and said server uses this signaling information to calculate a command flow rate to the cameras so that they adapt their compression parameters to meet this target rate constraint.
  7. 7- CCTV system according to one of claims 1 or 5 characterized in that the priority level of the video stream is used by each camera to calculate a flow setpoint, said flow setpoint being used to adapt the compression parameters of the flow video.
  8. 8- video surveillance system according to one of the preceding claims characterized in that the automatic switching of the display on the display or terminals is of a duration determined by the order of priority.
  9. 9- video surveillance system according to one of the preceding claims characterized in that the display on the viewing terminal (s) of a priority video stream is accompanied by an augmented reality mechanism for highlighting the one or more portion (s) of the image containing the unusual activity identified by the smart camera.
  10. 10-CCTV system according to claim 9 characterized in that the augmented reality mechanism is controlled by the remote server and said server accordingly transmits a request to the corresponding smart camera to acquire the necessary signaling data.
  11. 11-video surveillance system according to one of claims 9 to 10 characterized in that the portion (s) of the image containing the unusual activity are highlighted by one or more rectangle (s) flashing (s) or fixed (s).
  12. 12-CCTV system according to one of claims 9 to 11 characterized in that the portion (s) of the image containing the unusual activity are highlighted by a color highlighting.
  13. 13-CCTV system according to one of claims 9 to 12 characterized in that the highlighting of the portion of the image 30 containing the unusual activity identified by the smart camera is accompanied by a message displayed on the screen of at least one display terminal.
  14. 14-CCTV system according to one of claims 9 to 13 characterized in that the highlighting of the portion of the image containing the unusual activity identified by the smart camera is accompanied by an alarm indicator.
  15. 15-CCTV system according to one of the preceding claims, characterized in that the compressed video streams are recorded in a system memory area and the control data, such as the priority level of the streams and the ~ o metadata from targeted cameras, are also recorded.
  16. 16-CCTV system according to one of the preceding claims characterized in that a supervised mode of the system 15 allows an operator to select a particular video stream not necessarily priority and the server therefore responds by decreasing the overall bit rate and ordering the chosen camera a maximum rate.
FR0803052A 2008-06-03 2008-06-03 Intelligent videosurveillance system reconfigurable dynamically Active FR2932054B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
FR0803052A FR2932054B1 (en) 2008-06-03 2008-06-03 Intelligent videosurveillance system reconfigurable dynamically

Applications Claiming Priority (7)

Application Number Priority Date Filing Date Title
FR0803052A FR2932054B1 (en) 2008-06-03 2008-06-03 Intelligent videosurveillance system reconfigurable dynamically
PCT/EP2009/056684 WO2009147116A1 (en) 2008-06-03 2009-05-29 Dynamically reconfigurable intelligent video surveillance system
MX2010013320A MX2010013320A (en) 2008-06-03 2009-05-29 Dynamically reconfigurable intelligent video surveillance system.
EP09757497A EP2300997A1 (en) 2008-06-03 2009-05-29 Dynamically reconfigurable intelligent video surveillance system
BRPI0913386-0A BRPI0913386B1 (en) 2008-06-03 2009-05-29 Dynamically reconfigurable intelligent video surveillance system
MA33396A MA32380B1 (en) 2008-06-03 2010-12-03 video surveillance system intelligent dynamically reconfigurable
US13/218,807 US8922659B2 (en) 2008-06-03 2011-08-26 Dynamically reconfigurable intelligent video surveillance system

Publications (2)

Publication Number Publication Date
FR2932054A1 true FR2932054A1 (en) 2009-12-04
FR2932054B1 FR2932054B1 (en) 2010-08-13

Family

ID=40428275

Family Applications (1)

Application Number Title Priority Date Filing Date
FR0803052A Active FR2932054B1 (en) 2008-06-03 2008-06-03 Intelligent videosurveillance system reconfigurable dynamically

Country Status (6)

Country Link
EP (1) EP2300997A1 (en)
BR (1) BRPI0913386B1 (en)
FR (1) FR2932054B1 (en)
MA (1) MA32380B1 (en)
MX (1) MX2010013320A (en)
WO (1) WO2009147116A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120314127A1 (en) * 2011-06-09 2012-12-13 Inayat Syed Provisioning network resources responsive to video requirements of user equipment nodes

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1173020A2 (en) * 2000-07-10 2002-01-16 Mitsubishi Denki Kabushiki Kaisha Networked surveillance and control system
US20050091311A1 (en) * 2003-07-29 2005-04-28 Lund Christopher D. Method and apparatus for distributing multimedia to remote clients
WO2006010910A1 (en) * 2004-07-27 2006-02-02 2020 Imaging Limited Apparatus and method for capturing and transmitting images of a scene
US20060104345A1 (en) * 2004-11-12 2006-05-18 Pelco Method and apparatus for controlling a video surveillance display
US20070024705A1 (en) * 2005-08-01 2007-02-01 Richter Roger K Systems and methods for video stream selection
US7177448B1 (en) * 2001-04-12 2007-02-13 Ipix Corporation System and method for selecting and transmitting images of interest to a user
US20070107029A1 (en) * 2000-11-17 2007-05-10 E-Watch Inc. Multiple Video Display Configurations & Bandwidth Conservation Scheme for Transmitting Video Over a Network
US20070174881A1 (en) * 2004-03-02 2007-07-26 Yuichi Idehara Media delivering apparatus and media receiving apparatus

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1173020A2 (en) * 2000-07-10 2002-01-16 Mitsubishi Denki Kabushiki Kaisha Networked surveillance and control system
US20070107029A1 (en) * 2000-11-17 2007-05-10 E-Watch Inc. Multiple Video Display Configurations & Bandwidth Conservation Scheme for Transmitting Video Over a Network
US7177448B1 (en) * 2001-04-12 2007-02-13 Ipix Corporation System and method for selecting and transmitting images of interest to a user
US20050091311A1 (en) * 2003-07-29 2005-04-28 Lund Christopher D. Method and apparatus for distributing multimedia to remote clients
US20070174881A1 (en) * 2004-03-02 2007-07-26 Yuichi Idehara Media delivering apparatus and media receiving apparatus
WO2006010910A1 (en) * 2004-07-27 2006-02-02 2020 Imaging Limited Apparatus and method for capturing and transmitting images of a scene
US20060104345A1 (en) * 2004-11-12 2006-05-18 Pelco Method and apparatus for controlling a video surveillance display
US20070024705A1 (en) * 2005-08-01 2007-02-01 Richter Roger K Systems and methods for video stream selection

Also Published As

Publication number Publication date
FR2932054B1 (en) 2010-08-13
WO2009147116A1 (en) 2009-12-10
MA32380B1 (en) 2011-06-01
EP2300997A1 (en) 2011-03-30
BRPI0913386A2 (en) 2015-11-24
BRPI0913386B1 (en) 2019-07-02
MX2010013320A (en) 2011-02-24

Similar Documents

Publication Publication Date Title
US7460601B2 (en) Video data transmitting/receiving method and video monitor system
US9524437B2 (en) Automatic extraction of secondary video streams
EP2795600B1 (en) Cloud-based video surveillance management system
AU2012340862B2 (en) Geographic map based control
US20080270569A1 (en) Method and system for analyzing multimedia content
US7986810B2 (en) Mesh based frame processing and applications
US20050036659A1 (en) Method and system for effectively performing event detection in a large number of concurrent image sequences
JPWO2008035745A1 (en) Surveillance system, camera, and video encoding method
US20050123172A1 (en) Monitoring an environment
US8558903B2 (en) Accelerometer / gyro-facilitated video stabilization
US6989745B1 (en) Sensor device for use in surveillance system
US7649938B2 (en) Method and apparatus of controlling a plurality of video surveillance cameras
US9143759B2 (en) Intelligent image surveillance system using network camera and method therefor
US7342489B1 (en) Surveillance system control unit
JP2008502229A (en) Video flashlight / visual alarm
US6774905B2 (en) Image data processing
KR100918480B1 (en) Stereo vision system and its processing method
US6961447B2 (en) Image monitoring method, image monitoring apparatus and storage media
US8780199B2 (en) Networked security camera with local storage and continuous recording loop
Engelsberg et al. A comparative review of digital image stabilising algorithms for mobile video communications
US7421727B2 (en) Motion detecting system, motion detecting method, motion detecting apparatus, and program for implementing the method
US8675074B2 (en) Custom video composites for surveillance applications
EP2688296A1 (en) Video monitoring system and method
US8675065B2 (en) Video monitoring system
US10116904B2 (en) Features in video analytics

Legal Events

Date Code Title Description
PLFP Fee payment

Year of fee payment: 9

PLFP Fee payment

Year of fee payment: 10

PLFP Fee payment

Year of fee payment: 11