EP2329474B1 - Scene surveillance method and system - Google Patents

Abstract

The invention relates to a scene surveillance method comprising the following steps: (a) evaluating the scene state data (DES) corresponding to the state of the scene at time t; (b) storing the data; (c) checking if the reliability level of the newly obtained data is below the set acceptable operating threshold (SAFE); (d) if the reliability level is below the threshold, the method continues from step (a); (e) if the reliability level is above the threshold, an updating cycle is launched; (f) activation of the updating system (SYMAJ) actuates a sensor selection step; and (g) the data and/or metadata from the selected sensor are received and used to establish a new scene state calculation phase comprising the following steps, namely (h) calculating the scene state data (DES) corresponding to the state of the scene, (i) storing the data, (j) checking if the reliability level of the newly obtained real scene state data (DES) is below the set acceptable operating threshold (SAFE), (k) if the real reliability level is below the threshold, the method continues from step (a) and (l) if the real reliability level is above the threshold, the method continues from step (f).

Description

TECHNICAL FIELD OF THE INVENTION

The invention relates to a method of monitoring a scene for providing in a substantially real time a state of the scene and a monitoring device of a corresponding scene.

STATE OF THE PRIOR ART

Multiple examples of realization of monitoring methods of places, premises or buildings are known. In general, the known approaches make it possible simultaneously to retrieve all the information making it possible to improve the knowledge of the state of a system and to update this state at one time. Such an approach requires large computing capacities and / or particularly long computation times since the system to be monitored comprises a large number of sensors. However, many applications in large buildings with many parts require the use of a large number of sensors, usually cameras. Moreover, the complexity of such systems generates many false detections or alarms. Indeed, no particular attention is paid to the results of the data analysis methods of the sensors. These results are considered inaccurate but always correct which is not always the case since analysis errors can occur. Finally, the operator does not know the status of the system. It is for information only alarms or alerts, and / or a plurality of information transmitted on a plurality of outputs such as television screens. Moreover, a single event, for example an intrusion, can involve the excitation of several sensors. The known systems then generate a plurality of alerts, typically one per sensor. Finally, when the system is used with different types of sensors such as cameras, intrusion sensors, motion sensors, there is no cooperation between the sensors. That is, when the system uses a given sensor, it does not take into account the data it has received from other sensors. Some systems offer an alternative but which is limited to sharing the alerts provided by each sensor.

Such systems imply not only heaviness in terms of hardware and additional costs, but also cause difficulties of use due in particular to the many false alarms, with all the consequences that this may entail.

The document US 2006/059557 , represents the state of the art closest to the subject of the invention, and describes a method of monitoring a scene that contemplates the acquisition of data from a plurality of sensors in communication with the scene to be monitored and with a central monitoring system. This process uses meta-data in relation to the monitored object and determines the level of reliability relative to the meta-data, detecting a warning condition after comparing these meta-data with a model. The method described in this document also takes into account system state data.

The document WO 02/19077 , describes an intrusion detection system on a computer network using probabilistic correlation techniques to reduce the frequency of false alarms, and improve the information provided in intrusion detection systems. The document describes an anti-intrusion system comprising sensors of different types. The only data emitted by the sensors are alerts.

The intrusion detection system would benefit from having high-level information of the sensors that it could exploit to configure or parameterize the other sensors or methods of analysis of the sensor data. These data are correlated so that the state (i.e. alert or non-alert) of one sensor can affect the state of the other. This system also uses all available sensors simultaneously, requiring a powerful computing system and complex management. Indeed, the solicitation of a method of analysis of a sensor data requires firstly a necessary calculation time but also entails a risk of "pollution" of the final result because in some cases, the method can lead to a wrong analysis. Furthermore, only the states of the sensors are compared with one another: no comparison is made with respect to another value which may possibly be used as a reference.

Finally, the probability information used for the state of the sensors depends only on the states (ie alert or non-alert) of the other sensors. In the case of an intrusion detection surveillance system of a building used in difficult conditions (ie dust, brightness, rain, vegetation movement), the probabilistic information used for the sensor state should not be limited to the alerts raised by the other sensors but should be able to adapt dynamically to the changing conditions of perception of the sensors. the scene.

SUMMARY OF THE INVENTION

To avoid these disadvantages, the invention provides, in a monitoring device of a scene of at least two dimensions, delimited by a known and localized periphery, said device comprising a central monitoring system, a plurality of sensors in communication with the scene to be monitored and with the central monitoring system, said central system including an Update and Alert System (SYMAJA) for performing data acquisitions, providing metadata in relation to the monitored object, determine the level of reliability (and possibly precision) of the metadata and to detect a possible alert condition (corresponding to a supradonne) by comparing the metadata with at least one Condition Definition Model (MODECA), a comparator, to determine whether the level of reliability corresponding to the Scene State Data (DES) satisfies an Admissible Threshold sible of Operating Established (SAFE),

1. a) évaluer, à un instant donné t, les Données d'Etat de Scène (DESS) correspondant à l'Etat dans lequel se trouve la scène à l'intervalle t sur la base des données et métadonnées disponibles de l'intervalle précédent t-delta t;
   Cette prédiction étant effectuée uniquement sur la base des données préalablement mémorisées (d'Etat de Scène historiques) plus les données théoriques éventuelles relatives au fonctionnement du système ex ; tout capteur proprioceptif uniquement : capteur de vitesse, accélération, inclinaison ou toute autre donnée d'un fichier ou capteur extéroceptif tel qu'une cellule photo électrique)
2. b) mémoriser les Données d'Etat de Scène (DESS) nouvellement obtenues ;
3. c) vérifier, à l'aide du comparateur, si le niveau de fiabilité des Données d'Etat de Scène (DESS) nouvellement obtenues satisfait le Seuil Admissible de Fonctionnement Etabli (SAFE); (selon une variante de réalisation, la fiabilité est remplacée par la précision)
4. d) si le niveau de fiabilité satisfait le Seuil Admissible de Fonctionnement Etabli (SAFE), le Système de Mise à Jour (SYMAJ) est à l'état inactif, et, après un intervalle de temps prévu, le procédé se poursuit à l'étape a) ;
5. e) si le niveau de fiabilité ne satisfait pas le Seuil Admissible de Fonctionnement Etabli (SAFE), le Système de Mise à Jour (SYMAJ) passe à l'état actif et un cycle de mise à jour est lancé ;
6. f) l'activation du Système de Mise à Jour (SYMAJ) actionne une étape de sélection de capteurs permettant de sélectionner au moins un capteur permettant l'obtention de données ou de métadonnées dont les niveaux de fiabilité correspondants permettent de se rapprocher ou de dépasser les niveaux établis par le Seuil Admissible de Fonctionnement Etabli (SAFE);
7. g) les données et/ou métadonnées du capteur sélectionné sont reçues et prises en compte pour établir une nouvelle phase de calcul d'Etat de Scène selon les étapes suivantes :
8. h) calculer, à un instant donné (t), les Données d'Etat de Scène (DESS) correspondant à l'état dans lequel se trouve la scène sur la base des données et/ou métadonnées du capteur sélectionné obtenues à cet instant (t) donné ;
9. i) mémoriser les Données d'Etat de Scène Réel (DESS) nouvellement obtenus ;
10. j) vérifier, à l'aide d'une étape de comparaison, si le niveau de fiabilité des Données d'Etat de Scène (DESS) réelles nouvellement obtenues satisfait le Seuil Admissible de Fonctionnement Etabli (SAFE);
11. k) si le niveau de fiabilité réel satisfait le Seuil Admissible de Fonctionnement Etabli (SAFE), le Système de Mise à Jour (SYMAJ) passe à l'état inactif, et, après l'intervalle de temps prévu, le procédé se poursuit à l'étape a) (estimation);
12. l) si le niveau de fiabilité réel ne satisfait pas le Seuil Admissible de Fonctionnement Etabli (SAFE), le Système de Mise à Jour (SYMAJ) demeure à l'état actif, un nouveau cycle de mise à jour est lancé, et le procédé se poursuit à l'étape f).

a scene monitoring method for providing a real-time state of the scene, said method comprising the steps of:

1. a) evaluate, at a given time t, the Scene State Data (DESS) corresponding to the state in which the scene is located at the interval t on the basis of available data and metadata from the previous interval t -delta t;
   This prediction being performed solely on the basis of the previously stored data (historical Scene State) plus any theoretical data relating to the operation of the ex system; any proprioceptive sensor only: speed sensor, acceleration, inclination or any other data of a file or exteroceptive sensor such as a photoelectric cell)
2. b) store the newly obtained Scene State Data (DESS);
3. c) verify, with the aid of the comparator, whether the reliability level of the newly obtained Stage State Data (DESS) satisfies the Acceptable Set Operating Threshold (SAFE); (according to an alternative embodiment, reliability is replaced by precision)
4. d) if the reliability level meets the Acceptable Set Operating Threshold (SAFE), the Update System (SYMAJ) is in the idle state, and after a specified time interval, the process continues to step a);
5. e) if the level of reliability does not meet the Acceptable Operating Threshold (SAFE), the Update System (SYMAJ) goes into the active state and an update cycle is initiated;
6. f) activation of the Update System (SYMAJ) activates a step of selecting sensors for selecting at least one sensor for obtaining data or metadata whose corresponding reliability levels make it possible to approach or exceed the levels established by the Eligible Operating Threshold Established (SAFE);
7. g) the data and / or metadata of the selected sensor are received and taken into account to establish a new scene state calculation phase according to the following steps:
8. h) compute, at a given instant (t), the Scene State Data (DESS) corresponding to the state in which the scene is located on the basis of the data and / or metadata of the selected sensor obtained at that moment ( t) given;
9. i) memorize the newly obtained Real Scene State Data (DESS);
10. j) verify, using a comparison step, whether the reliability level of the newly obtained Real Scene State Data (DESS) satisfies the Acceptable Operating Threshold (SAFE);
11. k) If the actual reliability level satisfies the Established Safe Operating Threshold (SAFE), the Update System (SYMAJ) goes into the idle state, and, after the expected time interval, the process continues to step a) (estimate);
12. l) If the actual reliability level does not meet the Acceptable Set Operating Threshold (SAFE), the Update System (SYMAJ) remains in the active state, a new update cycle is initiated, and the process continues in step f).

Such a method makes it possible to reduce the number and / or the frequency of the false alarms, makes it possible to monitor a greater number of parameters while proposing a simplified exit facilitating the work of the operator. The calculation time and / or the computation power required are also considerably reduced because the number of sensor data to be processed is considerably reduced. According to an alternative embodiment, the expected time interval may change during the process depending for example on the risks that are variable in the course of the day.

According to the invention, the measurements of the state of a system are carried out in phases. Thus, the monitoring system is either in a rest phase or in an excited phase. It is in a rest phase when the state of the calculated system meets the requirements of the application. When this constraint is no longer validated, the system goes into an excited phase. This results in a solicitation of the sensors to recover their data and process them to improve the state to return to a new phase of rest. In the rest phase, the calculation of the state of the system is carried out by means of models, commands and / or inputs of the system, such as for example the actions performed on a "joystick" for controlling a manipulator arm, which indirectly make use of to sensors (eg sensors that measure the orientation of the "joystick").

According to an advantageous embodiment, the sensor selection step makes it possible to select the sensor that will provide a theoretical scene state which, in comparison with the Acceptable Operation Operating Threshold (SAFE), has the smallest deviation.

Thanks to these characteristics, it is possible to simultaneously solicit a single or a limited number of sensors to perform the first calculations, then complete a possible lack of reliability by additional data and / or metadata judiciously and rigorously selected. This step process simplifies management and operations for surveillance systems with a large number of sensors. According to an alternative embodiment, the selection step makes it possible to select a sensor signal processing method from among a plurality of available methods, either in addition to or instead of the sensor selection. An example of data that makes it possible to modify the state of a scene from one instant to another is the speed, which has the effect of modifying the position of an object as a function of time.

in another example, the sensor selection step, comparing the reliability levels of each of the data and / or metadata, and selecting the sensor and / or the sensor signal processing method whose reliability level is the higher.

Advantageously, at the sensor selection step, the difference with the Acceptable Operating Threshold Set (SAFE) is assigned a correction factor corresponding to the cost required to obtain said deviation.

The level of accuracy of the magnitudes which make up the system is advantageously added to the level of reliability considered.

The Scene State Data is preferably made available (at the output).

This can be done at any time, because there is always some knowledge of the current state of the scene, and even, in the case of the prediction stage, of the future state of the scene. Thus, it is useful to be able to provide an output indicating a scene in which the level of reliability is insufficient, or a scene in which this level is sufficient with a possible indication of an alert.

According to an advantageous embodiment, the scene to be monitored is complex and comprises a plurality of sub-scenes related to each other, and in which, during at least one of the steps of the method, the Scene State Data of a sub-scene are used for step a) of estimating the Scene State Data of another sub-scene, related to the first.

This supposes that the links (for example the layout plan of the rooms of a building) are known and taken into account. This feature is particularly advantageous because it allows the evaluation step a particularly high level of performance, thus avoiding calculations for a wide variety of cases. For example, a scene to be monitored may be an establishment having a plurality of speakers to monitor, the scene state data being presented on a synthetic visual representation of said institution. The establishment can be a museum, an airport, a bank, a shopping center, a public building, etc. The visual representation can be a schematic representation of the establishment in 2 or 3 dimensions. A single display can then represent a physical system or a complex scene, thus making the monitoring of this scene much simpler, ergonomic, efficient and safe.

In the event of an alarm or an alarm in an enclosure of the establishment, the enclosure concerned is indicated by a visual signal on the schematic representation of the establishment. The signal may be accompanied by an audible signal to alert an eventual operator of a message display, to identify the nature of the alert and any steps to be taken to verify the situation and / or to remedy it.

Alternatively, if the Scene State Data (ESD) evaluated from the first sub-scene reveals that no event related to monitoring is likely to occur at the second sub-scene, the this second sub-scene are put in deactivated mode.

Thanks to this characteristic, it is possible to place a plurality of sensors in "sleep" mode, greatly simplifying the operation of the system, limiting the number of operations to be performed, and contributing to increasing the speed of the system, even if the computer used does not have particularly important capabilities. In particular, this feature is particularly useful if the state of the system does not indicate any alert source in the second sub-scene and if this second sub-scene is accessible only by the first.

According to another variant, the sensors used are selected from the list comprising: a camera, a motion detector, a pressure sensor, a temperature sensor, a vibration sensor, a photoelectric cell, a laser beam, a thermal camera, a door opening detector or window or other access point that can be opened, an infrared sensor, an ultrasonic sensor, a radar, an acceleration sensor, an inclination sensor, a force sensor, an RFID sensor, an intrusion sensor such as broken glass sensors, an access badge reader, a magnetic sensor.

• un système central de calcul ;
• une pluralité de capteurs, en communication avec la scène à surveiller, chacun étant apte à surveiller un élément ou phénomène physique pour au moins une portion de ladite scène ;
• des moyens de transmission de données des capteurs vers le système central de calcul ;
• ledit système central comportant :
  • une interface d'entrée de données, permettant au système central de recevoir les données des capteurs et d'assurer la compatibilité avec le système ;
  • un Système de Mise à Jour et d'Alerte (SYMAJA), permettant de procéder à des acquisitions de données, de fournir des métadonnées en relation avec l'objet surveillé, de déterminer le niveau de fiabilité (et éventuellement de précision) relatif aux métadonnées et de déceler une éventuelle condition d'alerte en comparant les métadonnées avec au moins un Modèle de Définition de Condition d'Alerte (MODECA) ;
  • un comparateur, permettant de déterminer si le niveau de fiabilité correspondant aux Données d'Etat de Scène satisfait un Seuil Admissible de Fonctionnement Etabli (SAFE) ;
  • une interface de sortie, permettant de fournir une sortie en relation avec les Données d'Etat de Scène. (la sortie peut fournir des données, métadonnées, supradonnées, etc).

The invention also provides a device for monitoring a scene of at least two dimensions delimited by a known and localized periphery, said device comprising:

• a central system of calculation;
• a plurality of sensors, in communication with the scene to be monitored, each being able to monitor a physical element or phenomenon for at least a portion of said scene;
• means for transmitting data from the sensors to the central computing system;
• said central system comprising:
  • a data entry interface, enabling the central system to receive sensor data and ensure compatibility with the system;
  • an Update and Alert System (SYMAJA), enabling data acquisition, providing metadata related to the monitored object, determining the level of reliability (and possibly precision) relative to the metadata and detecting a possible alert condition by comparing the metadata with at least one Alert Condition Definition Model (MODECA);
  • a comparator, for determining whether the reliability level corresponding to the Scene State Data satisfies an Acceptable Set Operating Threshold (SAFE);
  • an output interface, for providing an output in relation to the Scene State Data. (The output can provide data, metadata, supradata, etc.).

Alternatively, the central computing system includes an interface for receiving data sources and / or metadata other than sensors.

For example, they may be data files, a clock, instructions and / or data from another system, an individual, etc. ,

The invention furthermore provides software comprising code elements programmed to implement the previously described method, when said software is loaded into a computer system and executed by said computer system.

In a variant, the software is in the form of a product recorded on a support readable by a computer system, comprising programmed code elements.

Here are some other advantages provided by the present invention. The computing power is reduced: in the idle phase, the system consumes very few resources; in the excited phase, only the useful sensors are solicited; in the excited phase, the system does not process all the data of the sensor but only the appropriate zone; unlike a conventional video surveillance system where the computing power required is a function of the number of sensors, the method and the system according to the invention make it possible to optimize the computing power.

The method and the device allow an independent approach of the type of sensors used. Any sensor to provide information on the state of the system is exploitable. That is to say any sensor for measuring a physical quantity that has been modified by the element that it is desired to detect or characterize.

The method and the device allow an approach integrating the reliability of the information returned by the sensors. Thus, the surveillance system will solicit a camera in a room where lighting is controlled but will not solicit it if it is dark because the sensor is no longer reliable to recognize individuals.

The method and apparatus allow an approach in which the sensors are not used independently of each other. Thus, in a conventional approach and for an intrusion detection function, there would be as many alarms as alerts returned by the sensors.

According to the invention, for the same intruder, there is only one alarm even if it passes in front of several sensors. Moreover, the method makes it possible to send back to the operator only the information that is useful with respect to its problematic. For example, rather than transmitting temperature increase information in a room, it is possible to indicate that a person is in the room. This information is obtained through a probabilistic correlation with models of temperature increase and the opening of the door.

DESCRIPTION OF THE FIGURES

• la figure 1 est une représentation schématique du principe de fonctionnement d'un système de surveillance de type connu;
• la figure 2 est une représentation schématique du principe de fonctionnement d'un système de surveillance selon l'invention;
• la figure 3 est une représentation schématique du système de surveillance selon l'invention;
• les figures 4 et 5 présentent les principales étapes du procédé de surveillance selon l'invention ;
• les figures 6 à 14 illustrent différents aspects relatifs à la notion de tryptique ;
• les figures 15 à 28 présentent un premier exemple de scène surveillée à l'aide du procédé de l'invention ;
• les figures 29 et 30 présentent un second exemple de scène surveillée à l'aide du procédé de l'invention ;
• la figure 31 présente un affichage type de surveillance selon un mode connu ;
• la figure 32 présente un exemple de présentation des résultats de la surveillance avec le procédé selon l'invention ;
• les figures 33 et 34 présentent un troisième exemple de scène surveillée à l'aide du procédé de l'invention.

All the details of realization are given in the description which follows, completed by the Figures 1 to 34 in which:

• the figure 1 is a schematic representation of the operating principle of a known type of surveillance system;
• the figure 2 is a schematic representation of the operating principle of a monitoring system according to the invention;
• the figure 3 is a schematic representation of the monitoring system according to the invention;
• the figures 4 and 5 present the main steps of the monitoring method according to the invention;
• the Figures 6 to 14 illustrate different aspects relating to the notion of tryptic;
• the Figures 15 to 28 present a first example of scene monitored using the method of the invention;
• the figures 29 and 30 present a second example of scene monitored using the method of the invention;
• the figure 31 presents a typical display of surveillance in a known mode;
• the figure 32 presents an example of presentation of the results of the monitoring with the method according to the invention;
• the figures 33 and 34 present a third example of scene monitored using the method of the invention.

DETAILED DESCRIPTION OF THE INVENTION

In the present description, the following terms are used in particular with the following meanings:

"Data" refers to information of a technical or physical nature provided by a sensor to the system. This is often a signal giving information of temperature, vibration, force, an image, an interruption signal of a laser beam, a propagation time of a wave, etc.

The term "metadata" refers to high-level information resulting from the analysis of at least a portion of a data item received from one or more sensors by a processing process. A metadata has a direct relationship with the object being monitored. For example, a metadata may for example correspond to the object of monitoring by a feature of that object that can be controlled or verified, such as a color, shape, profile, outline, etc. A camera is an example of a sensor providing raw non-exploitable data (image signal). This signal processed by an image processing process makes it possible to obtain one or more metadata.

The term "supradonne" refers to information related to the end result or objective of the surveillance. For example, there is intrusion or not, we stole the object under surveillance or not, etc.

By "monitor", we mean (for a sensor) the concepts and notions used to monitor, measure, detect, analyze an element (living or not, therefore including an animal, a plant, a virus, a micro-organism, a person or a group of people, etc.), or physical phenomenon.

The term "sensor" means a hardware device optionally incorporating processing tools that measure physical quantities or phenomena.

"Stage status" or "scene state data" or "physical state data" means information or data of a physical or technical nature representing the value of a magnitude or physical phenomenon present in the scene. By way of example of physical quantities or phenomena: presence or absence of a living or non living element, color, texture, temperature, speed, acceleration, luminosity, infrared, pressure, time, weight of an object or being the propagation time of a wave, the reflectivity of a surface, the position of an individual, etc. More particularly, the term "stage state" is used to describe the description stage technique on a particular date. This technical description contains a list of scene state data. One of the objectives of the method and the device consists in calculating the state of the scene, that is to say the set of state data of a scene. To calculate this state data, the system uses sensor data or metadata. It is understood that a scene is preferably static. However, certain "mobile" scenes, such as the cockpit or the cabin of an aircraft, the wagons of a train or the rooms of a ship, are not excluded from the method according to the invention.

"SYMAJA" means an Update and Alert System. This system allows for data acquisition, metadata related to the monitored object, determination of the level of reliability (and possibly precision) of the metadata, and detection of a possible alert condition by comparing metadata with at least one Alert Condition Definition Model (MODECA).

"MODECA" means an Alert Condition Definition Model. Such a model makes it possible to establish a direct or indirect relationship between the object of the surveillance and the metadata in order to detect a possible warning condition. For example, if one or more metadata matches elements of the model, an alert condition is given. If a certain number of metadata partially correspond to the model, or if the reliability level of certain metadata does not correspond to the requirements, the monitoring method may make it possible to obtain additional data or metadata to validate or not the alert condition.

The term "reliability" means the reliability variable of each quantity or set of quantities of the state of the calculated system. This value is intended to be compared to the reliability variable of the system state requested by the user for these same quantities. Whether the quantity considered is digital or not, the reliability is a numerical value between 0 and 1 and the precision is a numerical value greater than or equal to 0. Thus, when the comparison with the requirements is performed, the variables are compared. of reliability and precision and not the magnitudes to which are associated these variables. However, reliability and accuracy are always numerical values and can always be compared.

The purpose of this method is to establish the state of a system, particularly a scene. The state of a system corresponds to a set of numerical and possibly non-digital quantities characteristic of the system.

Thus, the method and the device make it possible to calculate these quantities (measure their value) with a certain precision and a certain reliability. The reliability of a quantity is the probability that the actual quantity belongs to the interval: value plus or minus its accuracy. If the quantity has no precision, then the reliability is the probability that the real quantity is equal to the value. Reliability can be defined not for a quantity but for a set of quantities. It then corresponds to the probability that each real quantity belongs to each interval or is equal to the value. In some cases, it is necessary to calculate additional quantities to achieve the desired accuracy and reliability of the quantities requested. In the following description, the level of reliability desired by the user, and in some cases the accuracy desired by the user correspond to the requirements of the application.

The figure 1 schematically illustrates the operating principle of a monitoring system of a device of known type. The sensors, usually cameras, are connected to video devices, allowing to view live, and possibly record what cameras can film scenes concerned. To visualize a lot of data, it is necessary to have a plurality of screens, making the consultations and monitoring tiresome and tiring for the operators. If other sensors are used, those are managed independently, and information obtained from different sources is provided separately to the operator. In practice, a single operator has difficulty controlling a large number of parameters at the same time. Often, to overcome this limitation, several operators are present.

The figure 2 schematically illustrates a scene monitoring system or device according to the invention. A central monitoring system 2 makes it possible to receive all the inputs of the sensors. A probabilistic approach, described later, makes it possible to predict a sequential solicitation of the sensors. The sensor data is interpreted to allow an overall analysis of the situation at the scene level. This makes it possible to produce alerts based on events rather than on the number of sensors involved. Thus, several sensors 30 participating in the detection of a single event will produce a reliable detection, and send a single signal to the operator, even if several sensors are involved. The process and system schematized at figure 2 will be presented in more detail with the aid of the following figures.

The figure 3 shows an example of a device or monitoring system 1 according to the invention. On the left portion of the figure, a series of sensors 30 is noted. These examples are presented to illustrate an infinite variety of sensors that can be used to improve system performance and / or to detect a wide variety of sensors. physical, mechanical, electrical, chemical, etc., or more simply to detect individuals, animals, objects, etc. For some applications it is also useful to receive data or information from other sources than sensors, such as data files from computers, data storage means, a machine, system or application. diverse.

A central surveillance system 2 makes it possible to receive the data and metadata of the various sensors 30 and of the other sources 40 via the specially adapted interface interfaces 21. These interfaces allow the central system 2 to receive the data and signals of the sensors by ensuring compatibility between these different elements. The central system 2 comprises an updating and warning system (SYMAJA) .22 and a comparator 23. These last two elements are advantageously implemented by means of a microprocessor or any other powerful means of calculation, with the adequate coded instructions. The data and metadata can be stored in a storage means 26 or memory, provided for this purpose. An output 24 can send to the operator or another system data, metadata, alerts, after interpretation and processing by the central system. The interface 25 makes it possible to receive the thresholds or adjustment parameters of the system if these parameters are not already parameterized in the central system.

The updating and alerting system 22 is provided for acquiring data from the sensors 30, providing metadata in relation to the monitored object, determining the level of reliability relating to the metadata, and detect any warning conditions. These alert conditions are identified by comparing the metadata with at least one Condition Definition Model (MODECA). The scene state data (DES) is compiled at its level, either by evaluation or by acquisition of new sensor data.

The comparator 23 makes it possible to determine whether the level of reliability corresponding to the Scene State Data satisfies an Acceptable Operating Threshold Set (SAFE).

The figure 4 presents a functional flowchart with the main steps of the method according to the invention. Firstly, an initialization phase makes it possible to specify, at the start of the process, the available data on the state of the system: value of digital or non-digital quantities, accuracy, reliability, etc. A prediction step makes it possible to pre-establish or evaluate the state of the system at a time or a date t, knowing the state of the system at a date passed t-delta t. The sensors of the device as well as any other data can be exploited to carry out this evaluation.

• Si l'état du système calculé est plus fiable (i.e. la variable fiabilité de chaque grandeur ou ensemble de grandeurs de l'état du système calculé a une valeur supérieure à la variable fiabilité de l'état du système demandé par l'utilisateur pour ces mêmes grandeurs) ou plus précis (i.e. la variable précision de chaque grandeur de l'état du système calculé a une valeur inférieure à la variable précision de l'état du système demandé par l'utilisateur pour ces mêmes grandeurs) que l'état du système demandé par l'utilisateur, alors ce résultat (i.e. l'état du système calculé) est fourni à l'utilisateur, et une nouvelle étape d'évaluation est lancée pour calculer l'état du système à l'itération suivante ;
• Si l'état du système calculé n'est pas assez fiable ou précis par rapport aux exigences et qu'il existe des informations potentiellement disponibles pour améliorer les données de l'état, en outre, des données provenant des capteurs, alors le processus passe à l'étape de mise à jour ;
• Si l'état du système calculé n'est pas assez fiable ou précis par rapport aux exigences et qu'il n'existe pas de données nouvelles pour améliorer la connaissance de l'état, alors le processus fournit ce résultat (i.e. l'état du système calculé) à l'utilisateur et reboucle à l'étape d'évaluation pour calculer l'état du système à l'itération suivante.

A comparison step to the requirements makes it possible to compare the state of the system previously established with the requirements of the application, namely the state of the system requested by the user. During this step, several cases may occur. So :

• If the state of the calculated system is more reliable (ie the reliability variable of each magnitude or set of magnitudes of the calculated system state has a value greater than the reliability variable of the system state requested by the user for these same quantities) or more precise (ie the variable precision of each magnitude of the state of the calculated system has a value lower than the variable precision of the state of the system requested by the user for these same quantities) that the state of the system requested by the user, then this result (ie the state of the calculated system) is provided to the user, and a new evaluation step is started to calculate the state of the system at the next iteration;
• If the state of the calculated system is not reliable enough or accurate against the requirements and there is potentially available information to improve the state data, in addition, data from the sensors, then the process passes at the update stage;
• If the state of the calculated system is not reliable enough or accurate with respect to the requirements and there is no new data to improve the state knowledge, then the process provides this result (ie the state of the calculated system) to the user and loops back to the evaluation step to calculate the state of the system at the next iteration.

The figure 5 presents the update step. This step consists in using available data to update the state of the system (for example a modification of the value of the digital quantities, a modification of the precision or a modification of the reliability). Unlike conventional methods with which all the information to improve the knowledge of the state of the system is recovered and to update this state at once, an iterative approach is provided. This approach consists not in extracting all the data at one time, but in retrieving and extracting a selected data item iteratively. The data is selected according to the contribution it makes to the improvement of the knowledge of the state of the system compared to the requirements defined by the user. This last point is schematized on the figure 6 .

• le gain qui évalue l'apport de l'information dans le calcul de la connaissance de l'état du système. Très souvent, il s'agit d'une différence entre l'estimation de l'état théorique du système et l'état du système précédent ;
• le coût qui peut être suivant l'application une estimation du temps nécessaire pour extraire l'information et/ou une estimation de la consommation énergétique nécessaire pour extraire l'information, etc ;
• la contribution qui pondère l'apport de l'information dans le calcul de la connaissance de l'état du système par rapport au coût d'obtention de l'information. Il s'agit d'une équation mathématique spécifique pour chaque application considérée et qui est fonction du gain et du coût.

Back to the figure 5 there is a step of enumerating the information, which consists in listing the potentially extractable data or metadata of the data provided by the sensors and which can improve the state of the system: either to increase the accuracy, or to increase the reliability, or both. For each potentially extractable information listed in the previous step, a comparator allows to calculate the theoretical state, namely what could be the state of the system updated by this data or metadata. Then the comparator determines, for each of the data or metadata:

• the gain that evaluates the contribution of the information in the calculation of the knowledge of the state of the system. Very often, it is a difference between the estimate of the theoretical state of the system and the state of the previous system;
• the cost that can be according to the application an estimate of the time necessary to extract the information and / or an estimate of the energy consumption necessary to extract the information, etc;
• the contribution that weights the contribution of the information in the calculation of the knowledge of the state of the system compared to the cost of obtaining the information. It is a mathematical equation specific for each application considered and which is a function of the gain and the cost.

Among the set of data or metadata listed potentially extractable, the comparator selects the data or metadata that has the greatest contribution. To recover it, the data signal from the sensor concerned is received and in the case of a metadata, the signal is analyzed by means of a signal processing method. This last step can provide two output results: either the data or metadata that was searched could be extracted, or it could not be extracted. In the case where the desired data or metadata could be extracted, the next step is to modify the value of the quantities of the state of the system, to update the precision associated with these values and to update the reliability for the quantities that have been modified. Since the device has managed to obtain the information compatible with what was wanted, then the reliability increases. However, before carrying out these actions, a backup is made to memorize the quantities of the state that will be modified as well as their accuracy, the reliability associated with all of these quantities and the temporal data such as date and current time.

In the case where the data or metadata sought could not be extracted, the next step consists in reducing the reliability of the magnitudes of the state that should have been modified, depending on the quality of the data of the sensor, the performances of signal processing method, etc. In the case where the data or metadata sought could not be extracted and if the updated reliability is below a threshold, then the process provides for recovering from the memory and deleting from the memory the quantities that would have been recovered. be modified with the information sought, their accuracy and reliability of this set. If the date of the recovered values differs from the current date, then the evaluation step is used to calculate these quantities at the current date.

Triptych concept

• probabilité que le triptyque réussisse à extraire (i.e. détecter) l'information (i.e. la métadonnée);
• dans le cas où le triptyque a réussit à extraire une information compatible avec l'information recherchée ;
• probabilité que l'information renvoyée correspond à l'information recherchée ;
• précision de l'information renvoyée.

In the case of CCTV, the system can use a camera to detect intruders in the room. The triptych (camera, processing algorithm, information sought, ie metadata) has a certain performance in terms of:

• probability that the triptych succeeds in extracting (ie detecting) information (ie metadata);
• in the case where the triptych has managed to extract information compatible with the information sought;
• probability that the information returned corresponds to the information sought;
• accuracy of the information returned.

This performance is evaluated online by the system. If it is day or night in the room, this last triptych will not have the same performance. Thus we can equip the room with a light sensor that can somehow "share its measurement" with the triptych to inform the camera that it works even better than the scene is adequately lit.

In reality, the brightness sensor will not act on the triptych (camera, algorithm, information) or on another triptych: it will improve the knowledge of the state of the system by informing an additional numerical quantity (ie a data of stage state). When the system wants to improve the knowledge of the state, it will list all the information (ie metadata) extractable and build the set of possible triptychs. Building does not mean using but rather enumerating and evaluating, that is, characterizing the performance of each triptych. Now the characterization of the performances takes into account the state of the system and therefore the quantity of light (ie numerical quantity of the state of the system which is indicated by the brightness sensor). For example, if the light is strong, the triptych will have high probabilities whereas when the lighting is weak, the triptych will have difficulties to extract the information and the returned information will have more risk of not corresponding to the information sought. Then for each triptych, one calculates the gain, the cost, the contribution: the triptych which has the most important contribution is selected.

Thus, a triptych defines a way of extracting useful information. In order to extract it, the system needs data, an algorithm for processing / searching for this data and for signing the information sought (i.e. the characteristics of the information). A triptych is a union of these elements: sensor, algorithm or process of processing, information sought (i.e. metadata).

Example ( figure 7 ): we want to develop an individual recognition system. For example, suppose the character to be recognized has a mustache. The surveillance system must reinforce working hypotheses (ie improve the reliability of the state of the system) by verifying that the individual under surveillance has a mustache (ie the information sought). Thus the triptych is: camera n ° 3, mustache recognition algorithm, a rectangular mustache of height 1 cm, width 3 cm, of black color, which must be in a certain zone of the image (called zone of interest), etc.

Suppose that the state of the system does not meet the requirements of the application and that a set of digital magnitudes of the system state need to be improved (i.e. not reliable enough and / or not accurate enough).

The system will define and select a triptych to improve these magnitudes. That is, it selects the sensor and the processing algorithm that it will use and specifies what information it will look for in a multi-dimensional area of interest. An area of interest is a subspace of the data of the sensor which makes it possible to restrict the search (ex: one looks for a straight line in an image which must belong to a rectangular zone of the image, the line has an inclination between x ° and y °, it has a thickness between a and b cm, etc.).

This triptych was selected because it had a gain, a cost, a contribution that was more interesting than the other triptychs ( figure 8 ).

Then, the system will use the triptych by applying the triptych processing algorithm to the area of interest defined with respect to the sensor data to recognize the information sought.

• l'algorithme ne réussit pas à trouver l'information recherchée alors que le système avait comme hypothèse que l'information était présente dans la zone d'intérêt. En conséquence, la fiabilité associée à l'ensemble des grandeurs numériques à améliorer diminue mais cela ne modifie pas leur valeur ni leur précision. (voir figure 9).
• l'algorithme réussit à trouver une information compatible avec l'information recherchée. Dans le cas de cet exemple, l'information serait : rectangle de couleur noire, de hauteur 1cm et de largeur 3cm à la position x, y dans la donnée image de la caméra. En conséquence, la fiabilité associée à l'ensemble des grandeurs numériques à améliorer augmente mais cela ne modifie pas leur valeur ni leur précision. (voir figure 10).

Two solutions are possible:

• the algorithm failed to find the information sought when the system assumed that the information was present in the area of interest. As a result, the reliability associated with the set of digital magnitudes to be improved decreases, but this does not modify their value or their accuracy. (see figure 9 ).
• the algorithm succeeds in finding information compatible with the information sought. In the case of this example, the information would be: rectangle of black color, height 1cm and width 3cm at the position x, y in the image data of the camera. As a result, the reliability associated with the set of digital magnitudes to be improved increases, but this does not change their value or their accuracy. (see figure 10 ).

Thanks to the information returned by the algorithm (or the triptych), it is possible to update the numerical values value, precision of the state of the system as well as the reliability (because the algorithm found the information but there is some uncertainty: is the detected black rectangle actually a mustache?). These new values (value, precision, reliability) are a function of the state of the system following the result of the triptych "I found the information" and performance of the triptych. (see figure 11 ).

When reliability increases or decreases, it depends on the performance of the triptych. These aspects are managed by the system in a probabilistic way.

• ■ γ la variable aléatoire binaire associée à l'événement « l'algorithme de traitement a détecté une information dans la donnée du capteur compatible avec l'information recherchée ».
  (c'est le résultat «information trouvée »).
• ■ H la variable aléatoire binaire associée à l'événement « les grandeurs numériques de l'état du système qui vont être modifiées suite à la tentative d'extraction de l'information sont fiables (i.e. justes) ».
  (la probabilité de H (i.e. P(H)) représente la fiabilité du cercle hachuré des figures 8 à 10).
• ■ Hnouveau la variable aléatoire binaire associée à l'événement « les grandeurs numériques de l'état du système calculées à partir de l'information extraite sont fiables (i.e. justes) ».
  (la probabilité de Hnouveau (i.e. P(Hnouveau)) représente la fiabilité du cercle quadrillé de la figure 11)).
• ■ G variable aléatoire binaire associée à l'événement « l'information retourné par l'algorithme de traitement est une information du même type que l'information recherché ». Dans le cas de cet exemple, le système souhaite détecter la moustache de l'individu positionnée devant la caméra. Cependant, si le champ de vision de la caméra est important, il est possible que d'autres individus moustachus soient filmés par cette même caméra. Si l'information renvoyée par l'algorithme est une moustache, il s'agit effectivement d'une information du même type que l'information recherchée. En revanche, si l'information renvoyée est un sourcil, il ne s'agit pas d'une information du même type à savoir d'une moustache. En effet, un sourcil et une moustache peuvent avoir des caractéristiques très proches : assimilable à un rectangle de couleur noire, cependant il s'agit de deux types d'informations.

We define for this:

• ■ γ the binary random variable associated with the event "the processing algorithm has detected information in the sensor data compatible with the information sought".
  (This is the result "information found").
• ■ H the binary random variable associated with the event "the numerical quantities of the system state that will be modified following the attempt to extract the information are reliable (ie, accurate)".
  (the probability of H (ie P (H)) represents the reliability of the hatched circle of Figures 8 to 10 ).
• ■ Hnouveau the binary random variable associated with the event "the numerical quantities of the state of the system computed from the information extracted are reliable (ie just)".
  (The probability of Hnouveau (ie P (Hnouveau)) represents the reliability of the squared circle of the figure 11 )).
• ■ G binary random variable associated with the event "the information returned by the processing algorithm is information of the same type as the information sought". In the case of this example, the system wishes to detect the mustache of the individual positioned in front of the camera. However, if the field of view of the camera is important, it is possible that other moustached individuals are filmed by the same camera. If the information returned by the algorithm is a mustache, it is indeed information of the same type as the information sought. On the other hand, if the information returned is an eyebrow, it is not a question of information of the same type namely of a mustache. Indeed, an eyebrow and a mustache may have very similar characteristics: similar to a black rectangle, however there are two types of information.

• ■ la probabilité que l'algorithme du triptyque détecte dans la zone d'intérêt de la donnée du capteur, une information compatible avec l'information recherchée en considérant que la connaissance de l'état du système utilisée est juste (i.e. fiable) : P(Y|H).
• ■ la probabilité que l'information retournée par l'algorithme soit une information du même type que l'information recherchée : P(G|Y,H).
• ■ la probabilité que le résultat retourné par l'algorithme soit l'information recherchée en supposant que l'information retourné est une information du même type que l'information recherchée P(Hnouveau|G).
• ■ la précision de l'information renvoyée.

Thus, to characterize a triptych (even before attempting detection) is to calculate at least the following quantities:

• The probability that the triptych algorithm detects information in the area of interest of the sensor data that is compatible with the information sought by considering that the knowledge of the state of the system used is accurate (ie reliable): P (Y | H).
• The probability that the information returned by the algorithm is information of the same type as the information sought: P (G | Y, H).
• The probability that the result returned by the algorithm is the information sought, assuming that the information returned is information of the same type as the information sought P (Hnew | G).
• ■ the accuracy of the information returned.

The gain, the cost are defined in relation to these quantities but others can be taken into account.

Detection of information is performed by applying the algorithm in the area of interest. If we consider that the event H is true, then the area of interest necessarily contains the information sought. But it can also contain other information or any other elements that we assimilate to noise. To be able to list the elements present in this zone, we divide it into samples to be analyzed. The purpose of the algorithm is to find the sample corresponding to the information sought.

Let nbln be the total number of samples in the area of interest. Among all these samples, there is the information sought (ie the mustache of the individual present in front of the camera) represented by the variable nbAr (ie nbAr = 1). There may also be other information of the same type as the information sought (whiskers belonging to other individuals could also be filmed) represented by the variable nbAc. These have a signature very close to the information to be detected. Also, the probability that the algorithm returns the information sought or information of the same type is almost identical. Finally, all the remaining samples are treated as a uniform noise represented by the nbBr variable (the eyebrows, a black shirt collar, a black thick bezel border are a whole set of samples that may have been filmed by the camera and that can disrupt the processing algorithm that might confuse them with whiskers). So we have nbBr = nbln - nbAr - nbAc.

To recognize the information sought, the algorithm searches for the signature of the latter in its area of interest. For this, he compares the footprint of each sample with the signature he has built. The returned sample is the one with the highest degree of resemblance while having a level above a threshold.

The fingerprint of each sample is extracted from a sensor measurement, so is likely to be noisy. In the case of the signature, it is constructed from an information modeling, thus incomplete and simplified. As a result, the detection of information can pose some difficulties. We propose here to characterize statistically the performances of the algorithm.

• la première Pamer mesure l'aptitude de l'algorithme à reconnaître l'information recherchée. Il s'agit donc de la probabilité qu'il identifie un échantillon comme étant l'information recherchée lorsque l'échantillon testé est effectivement l'information recherché. Dans le cas de notre exemple, il s'agit de savoir quelle est la probabilité que l'algorithme de détection de moustache reconnaisse une moustache lorsqu'on en lui présente une.
• la seconde Pbruit est la probabilité que l'algorithme identifie un échantillon comme étant l'information alors que ce dernier n'en est pas un. Dans le cas de cet exemple, il s'agit de savoir quelle est la probabilité que l'algorithme de détection de moustache reconnaisse un sourcil, une bordure de lunette, un col de chemise, etc, comme une moustache.

Two probabilities are posed:

• the first Pamer measures the ability of the algorithm to recognize the information sought. It is therefore the probability that it identifies a sample as being the information sought when the sample tested is indeed the information sought. In the case of our example, the question is what is the probability that the mustache detection algorithm recognizes a mustache when presented with one?
• the second Pbruit is the probability that the algorithm identifies a sample as the information while the latter is not one. In the case of this example, it is a question of what is the probability that the mustache detection algorithm recognizes an eyebrow, a rim of a telescope, a shirt collar, etc., like a mustache.

To calculate these last two probabilities, it is useful to proceed as follows:

Take a database of samples corresponding to the information sought (eg if the objective is to measure the performance of a face detection algorithm, we take a set of pictures of faces of different people and each photo corresponds to a sample). For each sample, the algorithm is applied and the degree of resemblance to the model is recovered. This gives the histogram of the figure 12 .

In an ideal world, all samples should have a degree of resemblance of 1.

We do the same with a database of samples that have nothing to do with the information we are looking for and we get the result of the figure 13 .

• Pamer = nombre d'échantillons au dessus du seuil sur la courbe 1 divisé par le nombre d'échantillons total de la base de donnée 1.
• Pbruit = nombre d'échantillons au dessus du seuil sur la courbe 2 divisé par le nombre d'échantillons total de la base de donnée 2.

Thanks to these two curves, we can define the threshold and we have:

• Pamer = number of samples above the threshold on curve 1 divided by the total number of samples in database 1.
• Pbruit = number of samples above the threshold on curve 2 divided by the total number of samples in database 2.

Estimate of probabilities of the triptych: it is now possible to calculate the probabilities characterizing a triptych. The first P (Y | H) represents the probability that the detector will return information when the H event is true. In the case of this example, it corresponds to the probability that the algorithm detects a mustache (ie event Y) while the person in front of the camera is mustachioed (ie event H). To calculate it, we determine the probability that no sample will respond positively to the detector. In the area of interest, there are nbBr noise samples. Each of them has a probability (1 - Pbruit) of not being recognized. There are also (nbAr + nbAc) samples that correspond to either the information sought or to information of the same type. We can consider that each of these samples has the same probability (1 - Pamper) of not being recognized by the detector. As a result, the probability P (Y | H) is: $$\mathrm{P}\left(\mathrm{Y}\mathrm{|}\mathrm{H}\right)\mathrm{=}\mathrm{1}\mathrm{-}\mathrm{P}\left(\mathrm{no\; Y}\mathrm{|}\mathrm{H}\right)\mathrm{=}\mathrm{1}\mathrm{-}{\left(\mathrm{1}\mathrm{-}\mathrm{Pnoise}\right)}^{\mathrm{\wedge }}\mathrm{NBQB}\mathrm{\times }{\left(\mathrm{1}\mathrm{-}\mathrm{Pamer}\right)}^{\mathrm{\wedge }}\left(\mathrm{NBAR}\mathrm{+}\mathrm{NBAC}\right)$$

The second P (G | Y, H) is the probability that the result returned by the detector is information of the same type as the information sought. In the case of this for example, this corresponds to the probability that the detected information (ie event Y: the rectangle of black color) corresponds to a mustache (ie event G). However, the area of interest contains (nbAr + nbAc) samples that correspond to information of the same type as the information sought. Taking Pamer into account, there are ((nbAr + nbAc) Pamer) of these samples that responded positively to the algorithm. And among all the samples in the area, there are ((nbAr + nbAc) Pamer + nbBr Pbruit) that responded positively. Also, P (G | Y, H) is the ratio of these two quantities: $$\mathrm{P}\left(\mathrm{BOY\; WUT}\mathrm{|}\mathrm{Y}\mathrm{,}\mathrm{H}\right)\mathrm{=}\left(\left(\mathrm{NBAR}\mathrm{+}\mathrm{NBAC}\right)\mathrm{\cdot }\mathrm{Pamer}\right)\mathrm{/}\left(\left(\mathrm{NBAR}\mathrm{+}\mathrm{NBAC}\right)\mathrm{\cdot }\mathrm{Pamer}\mathrm{+}\mathrm{NBQB}\mathrm{\cdot }\mathrm{Pnoise}\right)$$

The third P (Hnouveau | G), is the probability that the result returned by the detector is the information sought assuming that the event G is true. In the case of this example, this corresponds to the probability that the mustache detected (ie event G) corresponds to the mustache that the system wished to detect, that is to say the individual present exactly in front of the camera. As in the previous formula, it is a question of calculating the ratio of the number of favorable cases on the number of possible cases. If the event G is true, there are ((nbAr + nbac) Pamer) samples that may have been returned by the detector but only (nbAr - Pamer), we are interested. The ratio is: $$\mathrm{P}\left(\mathrm{Hnouveau}\mathrm{|}\mathrm{BOY\; WUT}\right)\mathrm{=}\left(\mathrm{NBAR}\mathrm{\cdot }\mathrm{Pamer}\right)\mathrm{/}\left(\left(\mathrm{NBAR}\mathrm{+}\mathrm{NBAC}\right)\mathrm{\cdot }\mathrm{Pamer}\right)$$

To calculate the last probability: P (Ylnon H), it is enough to be inspired by the method used for the computation of P (Y | H) considering that nbAr = nbAc = 0 so that nbBr = nbln. $$\mathrm{P}\left(\mathrm{Y}\mathrm{|}\mathrm{no\; H}\right)\mathrm{=}\mathrm{1}\mathrm{-}\mathrm{P}\left(\mathrm{no\; Y}\mathrm{|}\mathrm{no\; H}\right)\mathrm{=}\mathrm{1}\mathrm{-}{\left(\mathrm{1}\mathrm{-}\mathrm{Pnoise}\right)}^{\mathrm{\wedge }}\mathrm{NBQB}$$

To simplify the demonstration, it was assumed that the noise was evenly distributed in the area of interest. We could imagine a different distribution. For the information sought, we have defined an area of interest. We could very well have taken into account a probability density on the presence of information. For example, say that the information sought is twice as likely to be in the center of the area than on the sides.

Thus, in a summarized way, we characterize each triptych, then we calculate the gain, the cost and the contribution, we select the most interesting triptych (ie contribution max), if the triptych is not able to extract / find the information, the reliability associated with the digital quantities that had to be modified fall in the following way (Bayes formula): $$\mathrm{P}\left(\mathrm{H}\mathrm{|}\mathrm{no\; Y}\right)\mathrm{=}\left(\mathrm{P}\left(\mathrm{no\; Y}\mathrm{|}\mathrm{H}\right)\mathrm{.}\mathrm{P}\left(\mathrm{H}\right)\right)\mathrm{/}\mathrm{(}\mathrm{P}\left(\mathrm{no\; Y}\right)\mathrm{.}$$

If the triptych succeeds in extracting the information, the reliability of the hatched circle increases: $$\mathrm{P}\left(\mathrm{H}\mathrm{|}\mathrm{Y}\right)\mathrm{=}\left(\mathrm{P}\left(\mathrm{Y}\mathrm{|}\mathrm{H}\right)\mathrm{.}\mathrm{P}\left(\mathrm{H}\right)\right)\mathrm{/}\mathrm{(}\mathrm{P}\left(\mathrm{Y}\right)\mathrm{.}$$

Then we calculate the state of the system following the result of the triptych (ie the grid circle of the figure 11 ) using a data fusion technique such as the Kalman filter and calculating the reliability associated with this circle. $$\mathrm{P}\left(\mathrm{Hnouveau}\right)\mathrm{=}\mathrm{P}\left(\mathrm{Hnouveau}\mathrm{|}\mathrm{BOY\; WUT}\right)\mathrm{.}\mathrm{P}\left(\mathrm{BOY\; WUT}\mathrm{|}\mathrm{H}\mathrm{,}\mathrm{Y}\right)\mathrm{.}\mathrm{P}\left(\mathrm{H}\mathrm{|}\mathrm{Y}\right)$$

We have seen previously how to characterize the algorithms. This characterization is defined for a certain context of use (ie Pâmer and Pbruit were defined from a learning in certain condition). To the figure 14 we see that if we use this characterization in other conditions (eg the use of the camera at night) the system will work less well. If in the state of the system, a numerical quantity specifies the brightness of the room. It is possible to use this value to prevent the system from using the characterization of the algorithms with respect to the night context for example. In this case, it is necessary to provide a set of learning of the quantities Pâmer and Pbruit for all the contexts that one is likely to meet and use the knowledge of the system to switch on the right context during the operation of the process (ie very context light, dark context, dark context). We can also consider a characterization via a mathematical function for example: Pamer = f (amount of light in the room).

First example of scene

• une caméra dans le coin supérieur droit de la pièce,
• un capteur de mouvement au dessus de la fenêtre,
• un capteur de franchissement (i.e. une cellule photoélectrique) au niveau de la porte.

The Figures 15 to 28 show an embodiment of the method and the monitoring system for monitoring a simple scene, corresponding to a room. The purpose of this application is to detect and locate all individuals who enter the room. To do this, several sensors have been placed in this room:

• a camera in the upper right corner of the room,
• a motion sensor above the window,
• a crossing sensor (ie a photocell) at the door.

The figure 15 schematizes all of this information. These different sensors, in combination with a surveillance system presented to the figure 3 , allow to implement this example. The planned output is used to periodically, or where necessary, provide the status of the scene to the user or to another application.

All these elements and the method according to the invention make it possible on the one hand to obtain the data and / or metadata relating to the presence or absence of individuals in the room and on the other hand to obtain the data and or metadata about detected individuals to improve the tracking and tracking process of these individuals.

• pour chaque portion de la scène, d'une variable indiquant si la portion en question est libre ou occupée par un ou plusieurs intrus avec la fiabilité associée à cette variable. Pour cette modélisation, on utilise des outils de discrétisation afin de réaliser un découpage de la scène sous forme de cellules rectangulaires, tel qu'illustré à la figure 16. Par ailleurs, la variable indiquant si la cellule est libre et la fiabilité de cette variable peuvent être réunies au sein d'une même probabilité. Ainsi, à chaque cellule est associée la probabilité qu'un individu soit présent dans la cellule.
• pour chaque intrus, les données concernant la position de l'intrus dans la pièce, la précision associée à cette localisation, la vitesse de déplacement de l'intrus, la précision associée à cette vitesse et la fiabilité, à savoir la probabilité que l'individu soit réellement présent dans la pièce.

For example, consider that the state of the system consists of the following numerical quantities:

• for each portion of the scene, a variable indicating whether the portion in question is free or occupied by one or more intruders with the reliability associated with this variable. For this modeling, we use discretization tools to cut the scene in the form of rectangular cells, as illustrated in figure 16 . In addition, the variable indicating whether the cell is free and the reliability of this variable can be combined within the same probability. Thus, each cell is associated with the probability that an individual is present in the cell.
• for each intruder, data on the position of the intruder in the room, the accuracy associated with that location, the speed of the intruder's movement, the accuracy associated with that speed and the reliability, ie the probability that the intruder individual is actually present in the room.

The following table summarizes these data: Numeric quantities of the state possible values Precision Reliability Probability of presence of an individual in zone Z1 [0, 1] X X Probability of presence of an individual in zone Z2 [0, 1] X X ... ... X X Probability of presence of an individual in the ZM zone [0. 1] X X Position of the first detected individual {} X1.Y1 YES YES Movement speed of the first detected individual [0.3 m / s] YES ... ... Position of the last detected individual {XN, YN} YES YES Movement speed of the last detected individual [0.3 m / s] YES

The monitoring system can use two types of inputs: sensors and data.

• Capteur : caméra
• Donnée renvoyée : image couleur
• Information recherchée : est ce qu'un individu est présent dans la cellule Zi (i appartenant à [1, M]) ?
• Procédé de traitement : comparaison de l'image obtenue avec une image prise lorsque la pièce était vide (i.e. aucun individu).
• Information renvoyée : présence d'un individu avec sa précision de localisation et la fiabilité de cette information sinon absence d'un individu et la fiabilité de cette information.
• Capteur : capteur de mouvement
• Donnée renvoyée : signal 2D de mesure du rayonnement infrarouge
• Information recherchée : est ce qu'un individu est présent dans la cellule Zi (i appartenant à [1, M]) ?
• Procédé de traitement : comparaison du signal infrarouge avec une mesure prise lorsque la pièce était vide (i.e. aucun individu).
• Information renvoyée : présence d'un individu avec sa précision de localisation et la fiabilité de cette information sinon absence d'un individu et la fiabilité de cette information.

Let us analyze more closely the sensors and more particularly the triptych formed by the sensor, the signal processing method of this sensor and the extractable metadata used in the updating step. In order to preserve this simple example, a single sensor signal processing method is considered, thus effectively removing a possible step of selecting the most relevant method.

• Sensor: camera
• Data returned: color image
• Information sought: is an individual present in cell Zi (i belonging to [1, M])?
• Processing method: comparison of the image obtained with an image taken when the room was empty (ie no individual).
• Information returned: presence of an individual with its location accuracy and the reliability of this information otherwise absence of an individual and the reliability of this information.
• Sensor: motion sensor
• Data returned: 2D signal for measuring infrared radiation
• Information sought: is an individual present in cell Zi (i belonging to [1, M])?
• Processing method: comparison of the infrared signal with a measurement taken when the room was empty (ie no individual).
• Information returned: presence of an individual with its location accuracy and the reliability of this information otherwise absence of an individual and the reliability of this information.

Regarding the crossing sensor, its role is different from other previously presented sensors. Indeed, it does not indicate if an intruder is in the room. Nor does it provide data on the position of intruders in the room. On the other hand, whether or not he is solicited provides valuable information as to whether an intruder has entered or left the room. Accompanied by its reliability, it is used in the evaluation stage. In additional input, the use of a map of the room indicating where intruders are likely to come and go can be particularly useful for this application. As in the case of the crossing sensor, these data are used in particular during the evaluation stage.

Some settings can be left to the user's choice, or preset in a fixed way. Possible settings include how often the system status is updated. In the case of this example, a frequency of 10Hz is a period of 100 ms is selected.

The requirements of the application can also be fixed or adjustable. The figure 3 provides a parameter setting interface. These requirements define the minimum level of knowledge required by the user. This knowledge is all the more important as the accuracy of the state of the system and its reliability is great. The following two tables illustrate an example of setpoint or threshold values that correspond to the requirements of this application. Numeric quantities of the state Desired values Probability of presence of an individual in zone Z1 [0, 0.1 [union] 0.9, 1] ... ... Probability of presence of an individual in the ZM zone [0, 0.1 [union] 0.9, 1] Numeric quantities of the state Desired precision Desired reliability Position of the first detected individual [0, 0.3m [ ] 0.9, 1] Movement speed of the first detected individual [0, 0.3 m / s [ ... ... Position of the last detected individual [0, 0.3m [ ] 0.9, 1] Movement speed of the last detected individual [0, 0.3 m / s [

Illustration of the process with 4 iterations: Initialization phase

Suppose that initially, there is no individual in the room. As a result, the probability of each cell in the area is zero.

Prediction loop / update n ° 1

After the prediction step, we get the result shown on the figure 17 . Although the crossing sensor did not detect an interruption of the laser beam, an individual could still enter the room by passing over or under the crossing sensor. However, this case is relatively unlikely. Given the speed of movement of people, if an individual entered the room, he necessarily entered through the only entrance to the room ie the door and could not reach the other end of the room for a while. expected time interval, ie for 100ms.

The evaluation stage was able to point out that there is a risk, though low with a probability of 0.16, that an individual entered the room. In order to meet the requirements of the application, the system update and alert (SYMAJA) solicits a sensor. After calculating the contribution for each sensor, it appeared relevant to use the camera to search in the part of the image corresponding to the environment zone having a probability of 0.16 if an individual is present. The image processing method associated with the camera makes it possible to indicate that no person has been detected in the part of the analyzed image. By taking into account the reliability of this new information, the system is updated. In the end, the probability of an individual being present in the cells of the room located in front of the front door increases to 0.07 (see figure 18 ). The comparator then verifies that the state of the system meets the requirements of the application, which is the case. Thus, no further action is taken until the next update, which occurs after the expiration of the expected time interval.

Prediction loop / update n ° 2

At the next evaluation stage, the result illustrated on the figure 19 is obtained. As in the previous case, the crossing sensor did not detect any entry or exit of individuals from the room. In contrast, the shadow of the tree outside the room is projected into the same room near the front door. The state of the system after the prediction does not meet the requirements of the application. After calculating the contribution of each potentially extractable information, it is established that it is more advantageous to use the camera to search the part of the image corresponding to the area of the environment having a probability of 0.17 if an individual is present. If no verification is done, the monitoring system could indicate that a person has been detected in the portion of the scanned image. In reality, the system did not detect a person but the shadow of the tree. However, this shadow is, at this point, confused with a person. Also, the state of the system is updated with this information accompanied by its reliability and its proposal. On the one hand, the probability of cells in the room where no individual has been detected increases to 0.071. On the other hand, two new digital quantities are added in the state to describe the detected individual, namely the position and the speed of movement of the individual. Always taking into account the information extracted from the camera, a reliability is associated with these new quantities and is worth in the case of this example 0.34, as indicated in the figure 20 .

The comparator then verifies that the state of the system meets the requirements of the application, which is not the case because the reliability relative to the detection of the individual is too low.

However, the system update and alert (SYMAJA) has the ability to extract information from the motion sensor. This sensor indicates that no individual is present in the part of the signal corresponding to the area of the room where an individual is likely to be. Accurate with its reliability, this data makes it possible to update the state of the system. Thus, the reliability associated with the detected individual increases to 0.013 (see figure 21 ).

As specified in the application requirements, this reliability belongs to the interval [0, 0.1 [union] 0.9, 1]. However, this reliability is in the lower part of the interval. It therefore reflects a very strong certainty concerning the absence of any individual in this zone. The numerical quantities associated with the detected individual are thus removed from the state of the system and finally the following result (see figure 22 ) is obtained, which meets the requirements of the user.

Prediction loop / Update n ° 3

At the next iteration, the crossing sensor detected an interruption of its laser beam. This means that there is a very high probability that an individual has entered the room. However, his position is not known (see figure 23 ). Clearly, the state of the system does not meet the requirements of the application. Among the potentially extractable information for improving the knowledge of the state of the system, the motion sensor or the camera may be used. After calculating the contributions of this potentially extractable information, it is established that the camera, associated with its image processing method, is the most relevant. The result following the solicitation of the sensor is illustrated on the figure 24 . Thanks to the information provided by the image processing method that made it possible to analyze the portion of the image concerned, it is found that an individual is present in front of the door. By taking into account the accuracy and reliability of this data, new digital quantities are added in the state to describe the detected individual. The reliability associated with the detected individual is 0.76. On each side of the detected individual, the camera did not detect any individual, hence the probability of 0.17 of these cells. Finally it should be noted that the camera could not perform image perception behind the detected individual, which explains why the probabilities of these cells were not modified.

The system state still does not meet the requirements of the application. The update system has the ability to extract information from the motion sensor. The result of the system updated by this last sensor is represented at the figure 25 . This state still does not meet the requirements of the application, because of cells having a probability greater than 0.1, the accuracy relative to the detected individual, and the reliability about the detected individual. However, the update system no longer has the ability to retrieve new additional information from the sensors to improve the state.

Prediction loop / update n ° 4

At this iteration, the crossing sensor has detected no entry or exit of individuals from the room. Also, the evaluation stage will make it possible to calculate the state of the system by considering that an individual could enter the room by the door with a low probability and that an individual is likely to be in the cells. of the piece that have a probability greater than 0.1. Indeed, the system considers that there is no individual in the cells that have a probability less than 0.1, that there is an individual in the cells that have a probability higher than 0.9 and that there is may have an individual for the remaining cells. However, if an individual is likely to be in a cell, this individual could possibly move, which the evaluation step must also take into account.

Regarding the individual that has been detected at the previous iteration, the evaluation step will make it possible to estimate the area where this individual is likely to be taking into account his speed of movement. This is an area that integrates all the cells where the individual was able to go. The figure 26 presents the state of the system following the evaluation.

To improve the state of the system that does not meet the requirements of the application, the update system extracts additional data. The potentially extractable data that seems to be the most relevant to improve the state of the system can be obtained from the motion sensor. These data make it possible to indicate that there is no individual in front of the door and that the previously detected individual has moved in front of the window. This information extracted from the motion sensor could improve the location accuracy of the individual. In return, this has reduced reliability (see figure 27 ).

Since the state of the system still does not meet the requirements of the application, the update system then extracts information from the camera which leads to the following result (see figure 28 ). The state of the system responds here to the requirements of the application. An alert can therefore be sent to the operator of the surveillance system, who will be informed of the facts established with a reliability higher than the threshold requested. It is therefore found that the use of sensors capable of providing complementary data makes it possible to effectively overcome the lack of reliability associated with the first data of the first sensors.

Example with a plurality of scenes

The figure 29 presents the premises of a laboratory working on the development of vaccines against deadly viruses. To carry out his research, the laboratory stores these viruses in a cold room (ie zone 5 on the plane). This laboratory is on two levels. It is accessed by level 0 which overlooks the zone 1. By taking the elevator, one arrives at the reception point zone 2. From this last zone, one can reach various rooms notably zone 3. The Authorized persons who have a badge may pass in zone 4. Zones 4 and 6 are closed outer zones. It is possible to go to zone 6 by illegally crossing the fence. The cold room finally corresponds to zone 5.

• Il est interdit d'apporter et de déposer des objets dans la chambre froide, ceci pour garantir qu'aucun explosif ne sera disposé dans la pièce.
• Il est interdit de dérober des vaccins ou des virus.
• Les zones 4, 5 et 6 ne sont autorisées qu'aux personnes habilitées.

The laboratory wants to meet safety standards. For this, it must answer the following problems:

• It is forbidden to bring and deposit objects in the cold room, this to ensure that no explosive will be placed in the room.
• It is forbidden to steal vaccines or viruses.
• Zones 4, 5 and 6 are only allowed to authorized persons.

To meet these safety standards, the laboratory has been equipped with various sensors. Anyone can access zones 1, 2 and 3 as long as the laboratory is open. Zone 1 is equipped with a camera, a sensor provides information on the position of the elevator and a motion sensor is located in zone 2. The door allowing access to zone 4 is equipped with a badge reader and a breakage detection sensor of this glazed door. Zone 4 is equipped with a camera and a brightness sensor. A photocell is positioned at the entrance to zone 5. Finally, a thermal camera is placed in the cold room as well as numerous pressure sensors in the floor of this room to measure the weight of objects and individuals in the room. The figure 30 schematically shows the corresponding monitoring system.

With all these measuring devices, the laboratory responds to these security issues. Indeed, at zone 4, the camera will be able to detect the intruders who have entered the site through zone 6. Thanks to the brightness sensor, the system will dynamically adapt the reliability (ie in a probabilistic way) of the information. returned by the camera.

The photoelectric cell at the entrance to zone 5 makes it possible to know precisely when an individual enters or leaves the cold room. Only authorized individuals can enter this room and they must first identify themselves with their badge at the access control door of zone 4. The laboratory requires that these individuals weigh themselves regularly which allows to know precisely when these same individuals leave the cold room if they steal products.

Zone 5 pressure sensors are used to detect any object left in the chamber. As for the thermal camera, it is used to detect and locate all the individuals in the room. Indeed, in the middle of the night (i.e. brightness zero) the camera of the zone 4 can not detect the intruders and the system knows it. In this case, the thermal camera coupled to the photocell is the device for detecting intruders in the chamber. It should also be noted that the laboratory could have used a conventional color camera rather than a thermal camera. The sensors were chosen based on their ability to provide information on the state of the system. Also any sensor for measuring a physical quantity that will be modified by the element that we want to detect / characterize can be exploited. An individual in a cold room has a temperature different from the air in the room and is therefore detectable by a thermal camera and this much more reliable than with a color camera.

Either the following scenario: an individual enters the building through the main entrance. Immediately, it is detected by the camera that locates and follows it. The individual climbs into the elevator and goes down to the lower level. At this point, the zone 1 camera focuses its attention only on the entrance to the building since the only individual in zone 1 has left.

When the elevator arrives at zone 2, the system learns this information and then requests the motion sensor to observe the elevator door. The individual then moves to Zone 4.

In Zone 4, the system must ensure that there are only authorized individuals. To access this zone, intruders can come from zone 6 or zone 2 after having crossed the access control sensor. This is why the zone 4 camera only monitors the zone 4 border with zone 6 because there is a non-zero risk that an individual crosses the fence. Note that the system uses the information of the brightness sensor to know the reliability of detection of individuals with the camera.

Rather than submit to the control of his access badge, the individual breaks the glass door and enters the zone 4. The information of the broken door raised by the sensor indicates to the system that an individual is in the process to enter zone 4. Also, the system solicits the zone 4 camera to detect and locate the individual. From now on, this camera will no longer monitor only the zone 4 border with zone 6 but also the zone 2 border with zone 4 because there is a non-zero risk that another individual enters zone 4 by broken door.

All sensors in zone 5 are currently at rest. The system will solicit them only when an individual approaches the door of zone 5. As soon as the intruder crosses the photocell, the system solicits the thermal camera to detect it.

The individual then drops an object into the room and runs away. The system detects this object from the pressure sensors of the part and deduces that it is not an individual by soliciting the thermal sensor. Indeed, the temperature, the weight and the shape of the object are incompatible with that of an individual.

• Aptitude à solliciter et utiliser uniquement les informations (capteurs) nécessaires et pertinente pour répondre à la problématique de l'utilisateur : les exigences de l'application (i.e. état repos et excité du système) ;
• Gestion des capteurs de manière coopérative (ex : le capteur de luminosité et la caméra, le capteur de l'ascenseur et le capteur de mouvement, etc) ;
• Toutes les informations qui transitent dans le système sont probabilisées, de même que l'état du système.

In the case of this example, the following features of the system could be highlighted:

• Ability to solicit and use only the information (sensors) necessary and relevant to answer the problem of the user: the requirements of the application (ie state of rest and excited system);
• Collaborative sensor management (eg the brightness sensor and the camera, elevator sensor and motion sensor, etc.);
• All information passing through the system is probabilized, as is the state of the system.

• d'afficher l'ensemble des mesures ou alarmes renvoyées par les différentes capteurs, (vidéos des caméras de surveillance, alarmes des capteurs d'intrusion, des capteurs de mouvement, etc). Parfois ces données étaient affichées de manière cyclique lorsque leur nombre est trop important.
• de placer éventuellement ces mesures ou alarmes sur un plan ou une carte par rapport à la localisation des capteurs.
• d'inclure des informations de haut niveau sur les mesures renvoyées par les capteurs (rectangle autour des personnes détectées sur les vidéos de surveillance, trajectoire des personnes détectées, etc) (i.e. outils de réalité augmentée).

Representation of the result: previously, the solutions available were solutions called "augmented reality" (see figure 31 ). Namely, it was:

• to display all the measurements or alarms returned by the various sensors, (videos of surveillance cameras, alarms of intrusion sensors, motion sensors, etc.). Sometimes these data were displayed cyclically when their numbers are too large.
• to possibly place these measurements or alarms on a plane or a map with respect to the location of the sensors.
• to include high-level information on the measurements returned by the sensors (rectangle around the people detected on the surveillance videos, trajectory of the detected persons, etc.) (ie tools of augmented reality).

• le capteur de bris de glace,
• la caméra de la zone 4,
• la cellule photoélectrique,
• les capteurs de pression,
• la caméra thermique,

qui correspondent à une seule et même intrusion.So in the case of the previous example, we would have had 5 intrusion detection alarms sent back by:

• the ice breaking sensor,
• the camera of zone 4,
• the photocell,
• pressure sensors,
• the thermal camera,

which correspond to one and the same intrusion.

• L'opérateur n'a plus à interpréter les signaux capteurs. Si l'opérateur souhaite savoir si un individu a pénétré dans le site alors la représentation va consister en une vue 3D du site (i.e. réalité virtuelle) où le système indiquera les zones où des intrus sont présents mais les informations bas niveaux, (i.e. les données des capteurs) ne seront pas affichées.

The solution according to the invention makes it possible to offer the operator an interactive and intelligent visualization corresponding to his surveillance problem. One of the advantages is to facilitate the visualization and analysis of complex data to accelerate their understanding, communication and decision-making. The means of the invention provide an interactive data visualization technology based on "virtual reality" tools. Thus, as shown in the visualization example of the figure 32 , the system responds to the problem of the operator:

• The operator no longer has to interpret the sensor signals. If the operator wants to know if an individual has entered the site then the representation will consist of a 3D view of the site (ie virtual reality) where the system will indicate the areas where intruders are present but the low level information (ie sensor data) will not be displayed.

• Par rapport à la problématique de surveillance, le système informe l'opérateur sur les zones de l'environnement qu'il ne peut percevoir afin que ce dernier n'imagine pas qu'aucun intrus est dans la pièce alors qu'elle n'est simplement pas équipée de capteurs. De même que pour la détection des individus, le système informe l'opérateur s'il a un doute (fiabilité réduite) sur la présence de l'individu.

The system informs the user about its limitations:

• With regard to the problem of surveillance, the system informs the operator of the areas of the environment that he can not perceive so that the latter can not imagine that no intruder is in the room when he is not simply not equipped with sensors. As for the detection of individuals, the system informs the operator if he has a doubt (reduced reliability) on the presence of the individual.

• Par rapport à la problématique de l'opérateur, le système va attirer son attention sur les zones pertinentes : intrusion, regroupement d'individus, individus agités, etc, ainsi que sur les zones où les individus ont été détectés avec une fiabilité faible.
• Le seul moyen de lever les doutes est de contrôler les données des capteurs. Pour cela, uniquement les capteurs en lien avec la représentation 3D sont affichés. Par ailleurs, seul quelques capteurs sont affichés : les plus pertinents, c'est-à-dire ceux qui apportent une information nouvelle. Le système n'affiche pas les capteurs qui fournissent des informations redondantes par rapport à d'autres capteurs.

The system can dialogue with the operator by suggesting to him to control suspicious events with regard to his problematic or results that he has not been able to analyze:

• In relation to the problem of the operator, the system will draw his attention to the relevant areas: intrusion, grouping of individuals, restless individuals, etc., as well as in areas where individuals have been detected with low reliability.
• The only way to remove doubts is to control the sensor data. For this, only the sensors related to the 3D representation are displayed. In addition, only a few sensors are displayed: the most relevant, that is to say those that bring new information. The system does not display sensors that provide redundant information compared to other sensors.

• Si l'opérateur veut voir une zone particulière ou suivre un individu, il peut le demander au système en sélectionnant l'information sur la représentation 3D ou sur les donnes capteurs.

The system is interactive and responds to requests from the operator:

• If the operator wants to see a particular area or follow an individual, he can request it from the system by selecting the 3D representation information or the sensor data.

• une réalité virtuelle du site,
• focalisée sur le lieu où l'intrus se trouve,
• tenant compte que la porte badgée est brisée,
• intégrant les zones non perceptibles du site,
• couplée à une réalité augmentée des mesures des capteurs,
• avec uniquement les mesures des capteurs en lien avec la réalité virtuelle,
• et en priorité les mesures des capteurs les plus pertinentes, c'est-à-dire ceux permettant de vérifier l'analyse qu'a réalisé le système.

Thus in the case of the previous example, a single alarm is displayed and made available to the operator. The system displays:

• a virtual reality of the site,
• focused on where the intruder is,
• taking into account that the badged door is broken,
• integrating the non-perceptible areas of the site,
• coupled with augmented reality sensor measurements,
• with only sensor measurements related to virtual reality,
• and, in priority, the most relevant sensor measurements, ie those that make it possible to verify the analysis performed by the system.

Second example of scene: surveillance system of a jewel

For this example, a jewel is exposed to the public in a gallery. The jewel is presented on a base and is lit by halogen light spots. Several sensors are located nearby. A color camera and a thermal camera film the jewel. The latter is placed on a pressure sensor. A brightness sensor measures the lighting level. Indeed, when the light spots work, they can start to flash. A clock measures the time. The central monitoring system has access to a computer file that specifies the time slots during which the halogen spotlights operate. Finally, a belt of ten ultrasonic sensors surrounds the base to indicate if individuals are near the jewel.

This application aims to check the presence of the jewel on its base. It is therefore a device against theft.

The preparation of the surveillance system involves providing a modeling of the element that we want to monitor and recognize, namely the jewel. In the case of this example, the jewel is a metal crown. The model is the following: cylindrical object diameter 15cm, yellow color, weight 300gr, metal.

The state of the scene is composed on the one hand of the variable that one wishes to calculate namely: the presence of the jewel on its base. On the other hand, it can be considered that the more individuals around the jewel, the greater the risk that an individual will steal the object. Also, the state of the system includes the number of ultrasonic sensors having detected at least one individual. This last variable will play a particularly important role important in the step of estimating the state of the system at the date t from the historical state of the system at the date t-delta t. Finally a last variable will be used to indicate the level of luminosity around the jewel. This last variable will play a particularly important role in determining the reliability of the metadata extracted from the data (ie color images) of the camera.

The following table shows an example of the Stage State: Numeric quantities of the state Possible values Presence of the jewel {Yes No} Number of ultrasonic sensors [0, 10] Brightness [0, 1000 lux]

For the initialization phase, it is indicated that initially, the jewel is on its base, there is no individual is in the room and the level of brightness is not known, as indicated in this table : Numeric quantities of the state Value Precision Reliability Presence of jewel Yes X 1 Number of ultrasonic sensors 0 0 1 Brightness 500 500 0.5

It is observed that there is no precision associated with the variable "presence of jewel", which is quite normal because it is a discrete binary variable. We also note that reliability is associated with each digital quantity. In the case of some applications, there might be reliability not for each size but for a group. For brightness, the level is not known. The probability is set to 0.5. As the value is not known, it is initialized to the median value of the possible values, ie 500. Precision makes it possible to construct ranges of estimation of the quantities. The interval is built with terminals [value - precision, value + precision]. As stated previously, the value has been fixed at 500. For the precision, it is made so that it can allow to construct an estimated interval for the numerical quantity "luminosity" encompassing the range of the set of possible values for the magnitude [0, 1000lux]. So the precision is 500.

This latter table is an example of a result provided by the monitoring method: the state of the system is periodically provided to the user, to another application or to another device.

Concerning the requirements of the application, suppose that the state requested by the user has the characteristics presented in the following table: Sizes of the state Precision Reliability Presence of jewel X 0.95 Number of ultrasonic sensors 3 0.7 Brightness 100 0.8

Prediction step : This step is very dependent on the digital quantities considered, the application, its context of use. In this example, the following modeling is proposed: State at the date t - δt State at the date t Presence of jewel on date t - δt: Presence of jewel on date t: • _Presence value • _Presence value • Reliability _presence • Max (0.5, _Presence reliability - ( _ultrasound value +1) .0.17) Number of ultrasonic sensors on date t- δt: Number of ultrasonic sensors at date t: • _Ultrasonic value • _Ultrasonic value • _Ultrasound precision • • _Ultrasound precision _Ultrasonic reliability • Max (0.5, _Ultrasonic Reliability - 0.2) Brightness at the date t- δt: Brightness at date t: • _Brightness value • _Brightness value • _Brightness accuracy • _Brightness accuracy • _Brightness reliability • Max (0.5, _Brightness Reliability - 0.45)

This table should be interpreted as follows: for the number of ultrasonic sensors, it is considered that people can approach the base and others move away. We do not know at all how this varies. Thus, the reliability level of this magnitude is decreased to indicate that this information is less reliable. The reasoning is identical for the brightness.

For the size "presence of jewel", it is considered more this time that the closer the number of individuals close to the base, the more the jewel may be stolen.

For example, it is assumed that at a certain date, the state of the system is as follows: Sizes of the state Value Precision Reliability Presence of jewel Yes X 0.68 Number of ultrasonic sensors 6 4 0.73 Brightness 700 230 0.7

The comparison step consists in determining whether there exists one of the numerical magnitudes of the calculated state which has a precision greater than the precision indicated in the state desired by the user (ie requirements of the application) for this same state. magnitude or if there is one of the magnitudes of the calculated state that has a reliability lower than the reliability indicated in the state desired by the user (ie application requirements) for that same magnitude. In this case, the calculated state does not meet the requirements and the system goes into the excited state.

For this example, the reliability of the quantities "presence of jewelry" and "brightness" is insufficient, and the accuracy of the quantities "number of ultrasonic sensors" and "brightness" is also insufficient.

Update step

• Sensor: camera
• Data returned: color image of the top of the base where is the jewel.
• Metadata searched: detection of yellow in the image corresponds to the perception of the jewel.
• Image processing method: measuring the amount of yellow color in a part of the image.
• Result returned: if the number of yellow pixels exceeds a threshold then the image processing method returns metadata "detection of a yellow color that seems to match the jewel" otherwise "metadata searched not obtained".
• Metadata searched: detection of a cylindrical object of diameter 15cm in the image.
• Image processing method: detection of contours and shapes.
• Result returned: if a cylindrical object could be recognized then the following result is sent: "detection of a cylindrical object of diameter 15cm which seems to correspond to the jewel", otherwise "metadata sought not obtained".

• Sensor: pressure sensor.
• Data returned: weight of the object placed on the sensor.
• Information sought: is the data returned by the sensor worth 300gr?
• Result returned: if the measured data is worth 300gr then the following result is sent: "measurement of a weight compatible with the weight of the jewel", otherwise "weight incompatible with the jewel".

• Sensor: thermal camera.
• Data returned: 2D image of the measured temperature.
• Information sought: does the temperature of the object filmed by the thermal camera have a temperature compatible with that which the jewels should have?
• Result returned: by using the information on the time and the time slots of lighting of jewels and on the thermal release produced by the spots, the system can deduce at what temperature the jewels are at a certain time. If the heat measured by the sensor is compatible with that which should have the jewels, the following result is sent: "the thermal heat released by the filmed object is compatible with the heat that should clear the jewelry", otherwise "heat incompatible with jewelry ".
• Metadata sought: detection of a cylindrical object of diameter 15cm in the thermal image of the sensor.
• Processing method: detection of contours and shapes in a thermal image.
• Result returned: if a cylindrical object could be recognized, then the following result is sent: "detection of a cylindrical object of diameter 15cm which seems to correspond to the jewel", otherwise "metadata searched not obtained".

• Sensor: Brightness sensor.
• Data returned: brightness of the scene.

• Sensor: Ultrasonic sensor belts.
• Data returned: number of ultrasonic sensors having detected an obstacle.

In any case, the result is returned with its precision and reliability. The accuracy and reliability depend on the sensor and the treatment process considered. In the case of processing methods that work on the image of the color camera, the results returned are all the more reliable as the brightness of the scene is important. Thus, the reliability evaluation for metadata extracted from the camera image takes into account the brightness level of the scene available in the stage state as a numeric magnitude.

For example, it is accepted that at a certain date, the state of the system is as follows: Sizes of the state Value Precision Reliability Presence of jewel Yes X 0.68 Number of ultrasonic sensors 6 4 0.73 Brightness 700 230 0.7

Given the characteristics of the state, it is necessary to improve the reliability of the quantities "presence of jewelry" and "brightness" and the accuracy of the sizes "ultrasonic sensors" and "brightness".

• Métadonnée : détection de jaune dans l'image couleur ;
• Métadonnée : détection d'un objet cylindrique de diamètre 15cm dans l'image couleur ;
• Donnée : poids de l'objet posé sur le capteur de pression ;
• Donnée : température de l'objet filmé par la caméra thermique ;
• Métadonnée : détection d'un objet cylindrique de diamètre 15cm dans l'image thermique du capteur.

Suppose the update system comes to receive a new measurement for each sensor. It therefore has potentially available information to improve knowledge of the state. To enhance the "Presence of jewel" size, that is, to confirm the presence of the jewel, the monitoring system may use the following data or metadata:

• Metadata: detection of yellow in the color image;
• Metadata: detection of a 15cm diameter cylindrical object in the color image;
• Data: weight of the object placed on the pressure sensor;
• Data: temperature of the object filmed by the thermal camera;
• Metadata: detection of a 15cm diameter cylindrical object in the thermal image of the sensor.

• Donnée : nombre de capteurs ultrasons ayant détecté un obstacle.

To improve the size of "number of ultrasonic sensors", ie to determine how many ultrasonic sensors have detected individuals, the system can use the following data or metadata:

• Given: number of ultrasonic sensors that detected an obstacle.

• Donnée : luminosité de la scène

To improve the "brightness" size, the system can use the following data or metadata:

• Data: brightness of the scene

Reliability and accuracy of data is an intrinsic feature of sensors, while those of metadata are calculated dynamically by the update and alert system.

The determination of the data or metadata that the system must use to improve the state of the scene is performed by means of a comparison function which measures a difference between the theoretical state of the scene produced by a given piece of data or metadata. and the state of the scene desired by the user (ie application requirements).

The theoretical state produced by a data item or a metadata corresponds to the state that the system expects to obtain if it requests the recovery of the data or the extraction of the metadata. For this the system takes into account the reliability and the theoretical accuracy of the data and the metadata. It should be noted that at this stage, the update and alert system has not yet recovered the sensor data and has not performed any analysis or processing of the data.

As far as the comparison function is concerned, it consists of measuring the difference between the accuracy and reliability of each magnitude of the theoretical state with the accuracy and reliability of the state desired by the user. Each of these deviations can be weighted by a coefficient to indicate for example that the brightness accuracy is a more important characteristic than for the other quantities. For example, the following comparison function is used:

• Calculer l'état théorique $$\begin{array}{l}\mathrm{F}\left(\mathrm{etat\_theorique},\mathrm{etat\_souhaite}\right)\mathrm{=}\\ \begin{array}{l}\mathrm{4.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Fiabilité}}_{\mathrm{présence}}\left(\mathrm{souhaité}\right)\mathrm{-}{\mathrm{Fiabilité}}_{\mathrm{<figure-callout\; id="1"\; label="présence\; théorique\; +"\; filenames="imgf0004.png,imgf0021.png"\; state="\{\{state\}\}">présence</figure-callout>}}\left(\mathrm{théorique}\right)\right)\mathrm{+}\mathrm{}\\ \mathrm{1.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Précision}}_{\mathrm{ultrasons}}\left(\mathrm{théorique}\right)\mathrm{-}{\mathrm{Précision}}_{\mathrm{ultrasons}}\left(\mathrm{souhaité}\right)\right)\mathrm{+}\\ \mathrm{3.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Fiabilité}}_{\mathrm{ultrasons}}\left(\mathrm{souhaité}\right)\mathrm{-}{\mathrm{Fiabilité}}_{\mathrm{ultrasons}}\left(\mathrm{théorique}\right)\right)\mathrm{+}\\ \mathrm{1.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Précision}}_{\mathrm{luminosité}}\left(\mathrm{théorique}\right)\mathrm{-}{\mathrm{Précision}}_{\mathrm{luminosité}}\left(\mathrm{souhaité}\right)\right)\mathrm{+}\\ \mathrm{2.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Fiabilité}}_{\mathrm{luminosité}}\left(\mathrm{souhaité}\right)\mathrm{-}{\mathrm{Fiabilité}}_{\mathrm{luminosité}}\left(\mathrm{théorique}\right)\right)\end{array}\end{array}$$
  

Fin pourFor each data and metadata do:

• Calculate the theoretical state $$\begin{array}{l}\mathrm{F}\left(\mathrm{etat\_theorique},\mathrm{etat\_souhaite}\right)\mathrm{=}\\ \begin{array}{l}\mathrm{Four.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Reliability}}_{\mathrm{presence}}\left(\mathrm{wish}\right)\mathrm{-}{\mathrm{Reliability}}_{\mathrm{presence}}\left(\mathrm{<figure-callout\; id="1"\; label="theoretical\; +"\; filenames="imgf0004.png,imgf0021.png"\; state="\{\{state\}\}">theoretical\; </figure-callout>},\right)\right)\mathrm{+}\mathrm{}\\ \mathrm{1.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Precision}}_{\mathrm{ultrasonic}}\left(\mathrm{theoretical}\right)\mathrm{-}{\mathrm{Precision}}_{\mathrm{<figure-callout\; id="3"\; label="ultrasonic\; wish\; +"\; filenames="imgf0004.png,imgf0021.png"\; state="\{\{state\}\}">ultrasonic\; </figure-callout>}}\left(\mathrm{wish}\right)\right)\mathrm{+}\mathrm{}\\ \mathrm{3.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Reliability}}_{\mathrm{ultrasonic}}\left(\mathrm{wish}\right)\mathrm{-}{\mathrm{Reliability}}_{\mathrm{ultrasonic}}\left(\mathrm{<figure-callout\; id="1"\; label="theoretical\; +"\; filenames="imgf0004.png,imgf0021.png"\; state="\{\{state\}\}">theoretical\; </figure-callout>},\right)\right)\mathrm{+}\mathrm{}\\ \mathrm{1.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Precision}}_{\mathrm{brightness}}\left(\mathrm{theoretical}\right)\mathrm{-}{\mathrm{Precision}}_{\mathrm{<figure-callout\; id="2"\; label="brightness\; wish\; +"\; filenames="imgf0001.png,imgf0002.png"\; state="\{\{state\}\}">brightness\; </figure-callout>}}\left(\mathrm{wish}\right)\right)\mathrm{+}\mathrm{}\\ \mathrm{2.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Reliability}}_{\mathrm{brightness}}\left(\mathrm{wish}\right)\mathrm{-}{\mathrm{Reliability}}_{\mathrm{brightness}}\left(\mathrm{theoretical}\right)\right)\end{array}\end{array}$$
  

End for

Thus, the comparison function is 0 since the accuracy of the numerical quantities of the theoretical state is less than the accuracy of the quantities the desired state and that the reliability of the magnitudes of the theoretical state is greater than the reliability of the magnitudes of the desired state.

The computation time to obtain a metadata can be quite important due to the processing time of the sensor data. It is possible to integrate the estimated time required to obtain data or metadata or the amount of memory estimated necessary to obtain data or metadata, etc., in a cost variable. This cost can be taken into account in the comparison function to reduce the importance of a given datum or metadata. For this, it is possible to add this cost to the comparison function or to multiply the comparison function by the cost, etc.

It should be noted that the comparison function presented previously takes into account all the differences in reliability and accuracy for each quantity between the theoretical state and the desired state. However, these differences were weighted by multiplying coefficients (i.e. 4, 1, 3, 1 and 2). It is possible to use a comparison function which only takes into account the difference of a few quantities and is limited to the precision characteristic (ie the reliability characteristic will not be used). Thus, we obtain the following comparison function:

• Calculer l'état théorique $$\begin{array}{l}\mathrm{F}\left(\mathrm{etat\_theorique},\mathrm{etat\_souhaite}\right)\mathrm{=}\\ \begin{array}{l}\mathrm{0.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Fiabilité}}_{\mathrm{présence}}\left(\mathrm{souhaité}\right)\mathrm{-}{\mathrm{Fiabilité}}_{\mathrm{<figure-callout\; id="3"\; label="présence\; théorique\; +"\; filenames="imgf0004.png,imgf0021.png"\; state="\{\{state\}\}">présence</figure-callout>}}\left(\mathrm{théorique}\right)\right)\mathrm{+}\mathrm{}\\ \mathrm{3.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Précision}}_{\mathrm{ultrasons}}\left(\mathrm{théorique}\right)\mathrm{-}{\mathrm{Précision}}_{\mathrm{ultrasons}}\left(\mathrm{souhaité}\right)\right)\mathrm{+}\\ \mathrm{0.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Fiabilité}}_{\mathrm{ultrasons}}\left(\mathrm{souhaité}\right)\mathrm{-}{\mathrm{Fiabilité}}_{\mathrm{ultrasons}}\left(\mathrm{théorique}\right)\right)\mathrm{+}\\ \mathrm{1.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Précision}}_{\mathrm{luminosité}}\left(\mathrm{théorique}\right)\mathrm{-}{\mathrm{Précision}}_{\mathrm{luminosité}}\left(\mathrm{souhaité}\right)\right)\mathrm{+}\\ \mathrm{0.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Fiabilité}}_{\mathrm{luminosité}}\left(\mathrm{souhaité}\right)\mathrm{-}{\mathrm{Fiabilité}}_{\mathrm{luminosité}}\left(\mathrm{théorique}\right)\right)\end{array}\end{array}$$
  

Fin pour
ce qui revient, après simplification, à la fonction de comparaison suivante :For each data and metadata do:

• Calculate the theoretical state $$\begin{array}{l}\mathrm{F}\left(\mathrm{etat\_theorique},\mathrm{etat\_souhaite}\right)\mathrm{=}\\ \begin{array}{l}\mathrm{0.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Reliability}}_{\mathrm{presence}}\left(\mathrm{wish}\right)\mathrm{-}{\mathrm{Reliability}}_{\mathrm{presence}}\left(\mathrm{<figure-callout\; id="3"\; label="theoretical\; +"\; filenames="imgf0004.png,imgf0021.png"\; state="\{\{state\}\}">theoretical\; </figure-callout>},\right)\right)\mathrm{+}\mathrm{}\\ \mathrm{3.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Precision}}_{\mathrm{ultrasonic}}\left(\mathrm{theoretical}\right)\mathrm{-}{\mathrm{Precision}}_{\mathrm{<figure-callout\; id="0"\; label="ultrasonic\; wish\; +"\; filenames="imgf0009.png,imgf0010.png"\; state="\{\{state\}\}">ultrasonic\; </figure-callout>}}\left(\mathrm{wish}\right)\right)\mathrm{+}\mathrm{}\\ \mathrm{0.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Reliability}}_{\mathrm{ultrasonic}}\left(\mathrm{wish}\right)\mathrm{-}{\mathrm{Reliability}}_{\mathrm{ultrasonic}}\left(\mathrm{<figure-callout\; id="1"\; label="theoretical\; +"\; filenames="imgf0004.png,imgf0021.png"\; state="\{\{state\}\}">theoretical\; </figure-callout>},\right)\right)\mathrm{+}\mathrm{}\\ \mathrm{1.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Precision}}_{\mathrm{brightness}}\left(\mathrm{theoretical}\right)\mathrm{-}{\mathrm{Precision}}_{\mathrm{<figure-callout\; id="0"\; label="brightness\; wish\; +"\; filenames="imgf0009.png,imgf0010.png"\; state="\{\{state\}\}">brightness\; </figure-callout>}}\left(\mathrm{wish}\right)\right)\mathrm{+}\mathrm{}\\ \mathrm{0.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Reliability}}_{\mathrm{brightness}}\left(\mathrm{wish}\right)\mathrm{-}{\mathrm{Reliability}}_{\mathrm{brightness}}\left(\mathrm{theoretical}\right)\right)\end{array}\end{array}$$
  

End for
which comes back, after simplification, to the following comparison function:

• Calculer l'état théorique $$\begin{array}{l}\mathrm{F}\left(\mathrm{etat\_theorique},\mathrm{etat\_souhaite}\right)\mathrm{=}\\ \begin{array}{l}\mathrm{3.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Précision}}_{\mathrm{ultrasons}}\left(\mathrm{théorique}\right)\mathrm{-}{\mathrm{Précision}}_{\mathrm{ultrasons}}\left(\mathrm{souhaité}\right)\right)\mathrm{+}\\ \mathrm{1.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Précision}}_{\mathrm{luminosité}}\left(\mathrm{théorique}\right)\mathrm{-}{\mathrm{Précision}}_{\mathrm{luminosité}}\left(\mathrm{souhaité}\right)\right)\end{array}\end{array}$$
  

Fin pourFor each data and metadata do:

• Calculate the theoretical state $$\begin{array}{l}\mathrm{F}\left(\mathrm{etat\_theorique},\mathrm{etat\_souhaite}\right)\mathrm{=}\\ \begin{array}{l}\mathrm{3.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Precision}}_{\mathrm{ultrasonic}}\left(\mathrm{theoretical}\right)\mathrm{-}{\mathrm{Precision}}_{\mathrm{<figure-callout\; id="1"\; label="ultrasonic\; wish\; +"\; filenames="imgf0004.png,imgf0021.png"\; state="\{\{state\}\}">ultrasonic\; </figure-callout>}}\left(\mathrm{wish}\right)\right)\mathrm{+}\mathrm{}\\ \mathrm{1.}\max \left(\mathrm{0}\mathrm{,}{\mathrm{Precision}}_{\mathrm{brightness}}\left(\mathrm{theoretical}\right)\mathrm{-}{\mathrm{Precision}}_{\mathrm{brightness}}\left(\mathrm{wish}\right)\right)\end{array}\end{array}$$
  

End for

In the case of this example, it is considered that the cost represents an estimate of the necessary processing time in milliseconds to obtain the data or the metadata.

Suppose that the metadata "detection of a cylindrical object with a diameter of 15 cm in the image" that the system can potentially obtain by applying an image detection algorithm on the camera data, would make it possible to obtain the following theoretical state : Sizes of the state Value Theoretical precision Theoretical reliability Presence of jewel Yes X 0.80 Number of ultrasonic sensors 6 4 0.73 Brightness 700 230 0.7

The comparison function for this metadata would be F = 3.max (0, 4 - 3) + 1.max (0, 230 - 100) = 133

Estimation of the processing time needed to obtain the metadata: 73 milliseconds. $$\mathrm{Comparison\; function}+\mathrm{cost\; of\; this\; metadata}:133+73=206$$

Suppose that the data "brightness of the scene" provided by the brightness sensor would obtain the theoretical state: Sizes of the state Value Theoretical precision Theoretical reliability Presence of jewel Yes X 0.68 Number of ultrasonic sensors 6 4 0.73 Brightness 700 110 0.84

The comparison function for this data would be F = 3.max (0, 4 - 3) + 1.max (0, 110-100) = 13

Estimation of the processing time required to obtain the data: 16 milliseconds.

Comparison function + cost of this data: 13 + 16 = 29

Suppose that the data "number of ultrasonic sensors having detected an obstacle" provided by the "ultrasonic sensor belt" sensor, makes it possible to obtain the following theoretical state: Sizes of the state Value Theoretical precision Theoretical reliability Presence of jewel Yes X 0.68 Number of ultrasonic sensors 6 2 0.95 Brightness 700 230 0.7

The comparison function for this data would be F = 3.max (0, 2 - 3) + 1.max (0, 230 - 100) = 130

Estimation of the processing time required to obtain the data: 3 milliseconds. $$\mathrm{Comparison\; function}+\mathrm{cost\; of\; this\; data}:130+3=133$$

The data or metadata that would be selected is the one that has a minimum "comparison function + cost" value, that is, the data "brightness of the scene".

In these various examples, some comparison functions have been presented as well as an example of cost. However, it is possible to offer any type of function.

Once the data or metadata is obtained, it is used to improve the state. This operation can be carried out in a conventional manner by using, for example, the data fusion tools (ie Kalman filter, particulate filter) to update the value and the accuracy of the state while the reliability of the digital quantities of the state is updated using the Bayes rule.

The figures and their descriptions made above illustrate the invention rather than limiting it. In particular, the invention and its various variants, defined by the appended claims, have just been described in connection with particular examples of sometimes very simple scenes for illustrative purposes. Nevertheless, it is obvious to a person skilled in the art that the invention can be extended to many other cases of scenes, in dwellings, shops, industries, office buildings, banks, airports, car parks, stations, etc.

The verbs "understand" and "include" do not exclude the presence of elements other than those listed in the claims. The word "a" preceding an element does not exclude the presence of a plurality of such elements.

Claims (11)

1. Scene surveillance method for a scene surveillance device with at least two dimensions, delimited by a known boundary and localised, said device comprising a central surveillance system, a plurality of sensors communicating with the scene under surveillance and with the central surveillance system, said central system comprising an update system allowing to carry out acquisitions of data, to provide metadata related to the object under surveillance, to determine the confidence level relative to the metadata and to detect an eventual alert condition, a comparator allowing to detect whether the confidence level corresponding to data representative of physical values present in the scene fulfils and admissible threshold, said method allowing the provision essentially in real time of a state of the scene; and including the steps consisting of:

   a) evaluate at a given time t, data representative of physical values present in the scene corresponding to the state in which the scene finds itself at the interval t based on data and metadata available at the preceding interval t-Δt;

   b) memorise the data representative of newly obtained physical values present in the scene;

   c) verify with the aid of a comparator whether the confidence level of the data representative of physical values present in the scene fulfils the admissible threshold;

   d) if the confidence level fulfils the admissible threshold, the update system is in the inactive state and, after a preset time interval, the method continues at step a);

   e) if the confidence level does not fulfil the admissible threshold, the update system switches into the active state and an update cycle is triggered;

   f) the activation of the update system actions a step of sensors selection allowing the choice of at least one sensor allowing gathering of data or metadata whose corresponding confidence levels allows to approach or exceed the levels determined by the admissible threshold;

   g) data and/or metadata of the chosen sensor are received and considered to determine a new scene state computation step according to the following steps:

   h) compute, at a given time (t), the data representative of physical values present in the scene corresponding to the state in which the scene finds itself based on data and/or metadata of the chosen sensor obtained at this given time (t);

   i) memorise the real data of the physical values newly obtained;

   j) verify, with the aid of a comparison step, whether the confidence level of the real data of the physical values newly obtained fulfils the admissible threshold;

   k) if the real confidence level fulfils the admissible threshold, the update system switches to the inactive state and, after the preset time interval, the method continues at step a);

   l) if the real confidence level does not fulfil the admissible threshold, the update system remains in the inactive state, a new update cycle is triggered and the method continues at step f).
2. Surveillance method according to claim 1, wherein the step of sensor selection allows the choice of the sensor that will provide a theoretical scene state that, compared with the admissible threshold, exhibits the lowest difference.
3. Surveillance method according to one of claims 1 or 2, wherein, in the step of sensor selection, the confidence level of each data or metadata is compared, and the sensor whose confidence level is the highest is selected.
4. Surveillance method according to one of claims 2 or 3, wherein, in the step of sensor selection, the difference with the admissible threshold is treated with a correction factor corresponding to the cost needed to obtain said difference.
5. Surveillance method according to one of claims from 1 to 4, wherein the precision level of the quantities composing the system is added to the considered confidence level.
6. Surveillance method according to one of claims from 2 to 4, wherein the data representative of physical values present in the scene are made available.
7. Surveillance method according to any of the preceding claims, wherein the scene under surveillance is complex and comprises a plurality of sub-scenes connected with one another, and wherein, in at least one of the method steps, the data representative of physical values present in the scene of a sub-scene are used in step a) of estimate the data representative physical values present in the scene of another sub-scene, connected with the first.
8. Surveillance method according to claim 7, wherein, if the data representative physical values present in the scene estimated from the first sub-scene reveal that no surveillance-related event is likely to eventuate at the level of the second sub-scene, the sensors of said second sup-scene are put into inactive mode.
9. Surveillance method according to any of the preceding claims, wherein the sensors used are selected from a list including: a camera, a motion detector, a pressure sensor, a temperature sensor, a vibration sensor, a photoelectric cell, a laser beam, a thermal camera, a detector of opening of a door or a window or another access point that can be opened, an infrared sensor, a ultrasound sensor, a radar, an accelerometer, an inclinometer, a force sensor, a RFID sensor, an intrusion sensor like a glass-crash sensor, a badge reader, a magnetic sensor.
10. Software comprising the programmed code elements to the execution of the method according to any of claims 1 to 9, when said software is loaded into an information system and executed by said information system.
11. Software according to claim 10, as a recorded product on a support readable by an information system, including programmed code elements.

Images