FR3135948A1 - device and method for monitoring a hardware infrastructure - Google Patents

Abstract

The invention relates to a method for monitoring, by a monitoring device (10), a hardware infrastructure (2) comprising a first hardware component (8a) and at least one second hardware component (5, 9, 8b, 14, 16), the method comprising: obtaining video data (DT1) acquired by at least one camera (12); generation of state data representative of respective states of the first and second hardware components over time; detection, from the state data of the first hardware component, of an event associated with the first hardware component at a time t1; and generation of an alert (AL1) representative of the event, said alert being associated with context data (DT4) comprising at least in part the state data of the first and/or second hardware components, defining as as context of the event a temporal evolution of the states of the first and/or second material components. Figure for abstract: Fig. 1.

Description

device and method for monitoring a hardware infrastructure

The invention relates to the field of video surveillance of infrastructures and in particular concerns a device and a method for monitoring a hardware infrastructure. The invention applies in particular, and not exclusively, to the monitoring of railway networks or railway furniture, such as for example level crossings.

Prior art

Monitoring technical equipment is a key element in various fields such as industry, port or railway activities, etc., to ensure proper functioning of the equipment in question, avoid breakdowns and ensure the safety of users and professionals. . For example, the monitoring and maintenance of infrastructure in the field of railway networks constitutes an important issue due in particular to the resources required in human and material terms, but also in view of the associated financial costs and security considerations.

Rail networks involve numerous material elements that must be monitored regularly in order to prevent breakdowns and carry out maintenance operations if necessary.

For example, level crossings are infrastructures located at intersections between at least one traffic lane (road or pedestrian) and at least one railway track. These infrastructures include a certain number of hardware components which may vary depending on the case. Some level crossings are for example equipped with barrier guards (or barriers) preventing access to the railway track during a time interval corresponding to the passage of a railway vehicle, and also include signal lights and bells to warn of the passage of a railway vehicle, etc. Various failures or incidents can affect these hardware components so that monitoring is necessary.

Today, there are several ways to perform damage detection on technical railway equipment. Traditionally, the commonly used manual method consists of entrusting users to notify competent personnel when they notice a breakdown or malfunction on railway equipment, such as a level crossing for example. Typically, once informed, the equipment manager takes the necessary steps to check the damage and decides on the maintenance actions to be carried out. This method, still widely used today, however has drawbacks in that its effectiveness depends on the goodwill of the users. In particular, it is not guaranteed that users will always report the information, particularly when the impacted equipment is not essential for the users in question. Furthermore, the time between the appearance of the breakdown and the reporting of information by a user can be relatively long, which can cause numerous problems, including inconvenience for users and security risks, see even cause additional damage. Another problem lies in the fact that users are not always qualified to note the breakdowns of certain equipment (in particular this equipment is relatively complex), nor to precisely describe these breakdowns to those responsible.

In some cases, the task of monitoring technical equipment, such as level crossings for example, is entrusted to members of maintenance staff who make surveillance rounds to check the condition of the equipment. However, the delays in detecting damage can also be relatively long and are a direct function of the frequency of the surveillance rounds implemented (not counting errors of assessment, oversights, etc.). In addition, the regular intervention of qualified personnel generates significant costs, particularly in financial and time terms.

In addition, the manual methods described above often do not make it possible to precisely understand the origin of failures, which prevents effective preventive or curative actions from being carried out.

More rarely, video surveillance is used on certain technical sites to check the state of material infrastructure. An operator can possibly view a video in real time to observe a failure, or view the video after observing the failure to try to understand the causes. However, this technique requires significant resources, particularly in terms of human and time, and does not make it possible to effectively monitor certain infrastructures, particularly when they are numerous or complex.

Generally speaking, there is a need for a solution making it possible to reliably and effectively monitor material infrastructures or technical equipment, such as for example railway equipment, technical means of production, port infrastructures, and more generally all equipment comprising hardware components likely to encounter any events, such as breakdowns, malfunctions, incidents, etc.

One of the objects of the present invention is to solve at least one of the problems or deficiencies of the technological background described above.

Another object of the present invention is to offer a solution making it possible to reliably and effectively monitor material infrastructures or technical equipment, such as for example railway equipment, technical means of production, port infrastructures, and more generally all equipment comprising hardware components likely to encounter breakdowns, damage, malfunctions, incidents, passage into a predefined state, or other types of events.

According to a first aspect, the present invention relates to a monitoring method, implemented by a monitoring device, for monitoring a hardware infrastructure comprising a first hardware component and at least one second hardware component, said method comprising:

- obtaining video data acquired by at least one camera monitoring the hardware infrastructure;

- generation, by image analysis applied to video data, of state data representative of respective states of the first and second hardware components over time;

- detection, by application of at least a second decision rule taking as input the state data of the first hardware component, of an event associated with the first hardware component at a time t1; And

- generation of an alert representative of the event, said alert being associated with context data comprising at least in part the state data of the first and/or second hardware components, defining as context of the event a temporal evolution of the states of the first and/or second hardware components.

According to a particular embodiment, said event is detected by application of said at least one second decision rule taking as input the state data of the first hardware component combined with the context data.

According to a particular embodiment, during image analysis, a first neural network previously subjected to machine learning performs the following steps:

- detection of objects in images defined by video data; And

- object classifications for detecting objects corresponding to the first and second hardware components in respective states.

According to a particular embodiment, during image analysis, a second neural network previously subjected to machine learning takes as input at least a first decision rule and results of the first neural network to determine as output the data 'state.

According to a particular embodiment, the method further comprises:
- reception of sensor data acquired by at least one sensor monitoring the hardware infrastructure, other than said at least one camera, said sensor data being representative of at least one characteristic of the first and/or second hardware component over time ;
wherein said event is detected by applying said at least one second decision rule based on the state data of the first hardware component in combination with the sensor data.

According to a particular embodiment, the context data defines as context of the event a temporal evolution of the respective states of the first and/or second hardware components during a period of time preceding the moment t1.

According to a particular embodiment, the method further comprises:
- sending the alert in association with the context data to a remote control unit to cause subsequent event analysis.

According to a particular embodiment, the monitoring device comprises a non-volatile memory, the method further comprising:
- recording as event history, in non-volatile memory, of the alert in association with the context data to allow subsequent event analysis.

According to a second aspect, the different stages of the monitoring method according to the first aspect of the invention are determined by computer program instructions.

Consequently, the invention also relates to a computer program on an information medium (or recording medium), this program being capable of being implemented in a monitoring device or more generally in a computer, this program comprising instructions adapted to the implementation of the steps of a monitoring process as defined in this document.

Thus, the method of the invention can be implemented by means of a non-volatile memory storing computer program instructions and by means of a processor executing these instructions.

This computer program may use any programming language, and be in the form of source code, object code, or intermediate code between source code and object code, such as in partially compiled form, or in no matter what other desirable shape.

According to a third aspect, the invention also aims at an information medium (or recording medium) readable by a computer (or by at least one processor), and comprising instructions for a computer program according to the second aspect. and as defined in this document.

The information carrier can be any entity or device capable of storing the program. For example, the medium may comprise a storage means, such as a rewritable non-volatile memory or ROM, for example a CD ROM or a microelectronic circuit ROM, or even a magnetic recording means, for example a hard disk, USB stick, etc.

On the other hand, the information carrier may be a transmissible medium such as an electrical or optical signal, which may be carried via an electrical or optical cable, by radio or by other means. The program according to the invention can in particular be downloaded onto an Internet type network.

Alternatively, the information carrier may be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in executing the method in question.

According to a fourth aspect, the present invention also targets a monitoring device configured to implement the monitoring method according to the first aspect of the invention and as defined in this document. In particular, the invention according to the fourth aspect relates to a monitoring device for monitoring a hardware infrastructure comprising a first hardware component and at least a second hardware component, said device comprising a memory associated with at least one processor configured for the implementation of the steps of the monitoring method according to the first aspect of the invention.

According to a particular example, the invention according to the fourth aspect aims at a monitoring device for monitoring a hardware infrastructure comprising a first hardware component and at least a second hardware component, said device comprising:

- obtaining module configured to obtain video data acquired by at least one camera monitoring the hardware infrastructure;

- analysis module configured to generate, by image analysis applied to video data, state data representative of respective states of the first and second hardware components over time;

- detection module configured to detect, by application of at least a second decision rule taking as input the state data of the first hardware component, of an event associated with the first hardware component at a time t1; And

- generation module configured to generate an alert representative of the event, said alert being associated with context data comprising at least in part the state data of the first and/or second hardware components, defining as context of the event a temporal evolution of the states of the first and/or second hardware components.

Note that the different embodiments defined above (as well as those described below) in relation to the monitoring method according to the first aspect of the invention as well as the associated advantages apply analogously to the monitoring device. monitoring according to the fourth aspect of the invention.

For each step of the monitoring process, the monitoring device of the invention may comprise a corresponding module configured to carry out said step.

According to one embodiment, the invention is implemented by means of software and/or hardware components. With this in mind, the term “module” can correspond in this document to a software component as well as to a hardware component or to a set of hardware and software components.

A software component corresponds to one or more computer programs, one or more subprograms of a program, or more generally to any element of a program or software capable of implementing a function or a set of functions, as described in this document for the module concerned. Such a software component can be executed by a data processor of a physical entity (terminal, server, gateway, router, etc.) and is capable of accessing the hardware resources of this physical entity (memories, recording media, communication bus, electronic input/output cards, user interfaces, etc.).

In the same way, a hardware component (referring to a “module”) corresponds to any element of a hardware assembly capable of implementing a function or a set of functions, according to what is described in the this document for the module concerned. It may be a programmable hardware component or one with an integrated processor for executing software, for example an integrated circuit, a smart card, a memory card, an electronic card for executing firmware ( firmware), etc.

The present invention also relates to a surveillance system comprising a surveillance device as defined in this document as well as at least one camera with which the surveillance device cooperates to carry out the steps of the surveillance method.

The present invention makes it possible to carry out reliable and effective monitoring of an infrastructure comprising multiple hardware components likely to be affected by normal or abnormal events, such as for example damage, malfunction, passage into a predefined state, etc. This monitoring can be carried out automatically by an autonomous monitoring device. This effective monitoring is possible without resorting to human intervention as previously described.

In particular, thanks to the context data which is associated with the alerts produced by the monitoring device, it is possible not only to detect a particular event affecting a hardware component of the infrastructure, but also to analyze and understand the context in which said event occurred. It is thus possible to quickly detect (for example in real time) events affecting a hardware component and to carry out curative and/or preventive maintenance actions in order to guarantee that the infrastructure remains in good condition.

The use of a particular architecture of neural networks makes it possible in particular to effectively detect the state of the various hardware components of an infrastructure to be monitored and to deduce particular events that may require maintenance action.

Description of figures

Other characteristics and advantages of the present invention will emerge from the description of the particular and non-limiting examples of embodiment of the present invention below, with reference to the attached Figures 1 to 7, in which:

There schematically represents a monitoring device for monitoring a hardware infrastructure, according to at least one embodiment of the invention;

There schematically represents a monitoring device according to at least one embodiment of the invention of the invention;

There schematically represents modules implemented by a monitoring device according to at least one embodiment of the invention of the invention;

there represents in the form of a diagram the steps of a monitoring method according to at least one embodiment of the invention;

there schematically represents part of a monitoring method according to at least one embodiment of the invention;

there schematically represents part of a monitoring method according to at least one embodiment of the invention; And

there schematically represents an architecture of neural networks carrying out part of a monitoring method according to at least one embodiment of the invention.

detailed description

As already indicated, the invention relates in particular to a method and a monitoring device for monitoring hardware infrastructures, as well as a corresponding monitoring system and a corresponding computer program.

Examples of implementations of the invention will now be described in what follows with reference jointly to Figures 1 to 7. Unless otherwise indicated, the elements common or similar to several figures bear the same reference signs and present identical or similar characteristics, so that these common elements are generally not described again for the sake of simplicity.

The terms “first(s)” (or first(s)), “second(s)”, etc.) are used in this document by arbitrary convention to help identify and distinguish different elements (such as operations, threshold values, etc.) implemented in the embodiments described below.

Unless otherwise indicated, elements common or similar to several figures bear the same reference signs and have identical or similar characteristics, so that these common elements are generally not described again for the sake of simplicity.

The invention proposes in particular to implement monitoring of a hardware infrastructure by an autonomous video surveillance device capable of automatically detecting events affecting a hardware component of the infrastructure, this event possibly being damage, a malfunction, an incident, a change of state, a predefined state or any other event likely to affect said hardware component. This monitoring aims, for example, to enable control of the infrastructure, analysis of behavior, establishment of a diagnosis, performance of maintenance actions (preventive and/or curative), etc.

The invention, according to its different embodiments, aims in particular at a monitoring method, implemented by a monitoring device, for monitoring a hardware infrastructure comprising a first hardware component and at least a second hardware component, said method comprising: obtaining video data from one or a plurality of cameras; generation, from the video data, of state data representative of respective states of the first and second hardware components over time; detection, from the state data of the first hardware component, of an event associated with the first hardware component at a time t1; and generation of an alert representative of the event, said alert being associated with context data comprising at least in part the state data of the first and/or second hardware components, defining as context of the event a temporal evolution of the states of the first and/or second hardware components.

As described more below in particular embodiments, the video data are for example acquired by at least one camera monitoring (or filming) the hardware infrastructure. Status data can be generated by image analysis applied to video data, the nature of this analysis may vary depending on the case. In addition, the aforementioned event can be detected, for example, by application of at least one decision rule taking as input the state data of the first hardware component.

Thanks in particular to the context data which are associated with the alerts produced by the monitoring device 10, it is possible not only to detect a particular event affecting a hardware component of the infrastructure, but also to analyze and understand the context in which said event has occurred. The invention offers a reliable and effective solution for automatically monitoring a hardware infrastructure and thus improving its condition by carrying out, for example, curative and/or preventive maintenance actions.

Other aspects and advantages of the present invention will emerge from the exemplary embodiments described below with reference to the drawings mentioned above.

In this document, examples of implementation of the invention are described in the context of monitoring a railway infrastructure, for example railway equipment (also called "furniture") such as a level crossing by example. Note, however, that the invention does not apply exclusively to the monitoring of railway infrastructures but applies more generally to any hardware infrastructure (or hardware equipment) comprising hardware components (or hardware parts) that are desired. monitor visually for event detection and possibly maintenance purposes. By way of non-limiting examples, the invention also applies to the monitoring of industrial infrastructures (such as production lines, production machines, etc.), port infrastructures, road infrastructures, etc. We understand, for example, in view of the following embodiments that the invention can be applied to the monitoring of a mechanical parts assembly line (motor vehicle production line, etc.).

In this document, the type of events that can be detected by means of the invention may vary depending on the case and may concern for example damage, breakdowns, malfunctions, incidents, but also other types of events such as changes status, transition to a predefined state, state respecting predefined criteria, etc. Thus, the invention is not limited to the detection of damage but can also be applied to monitoring the behavior of an infrastructure over time, which can make it possible, for example, to carry out preventive maintenance to improve the state of a hardware infrastructure even before a failure occurs.

In this document, cameras can be used to acquire video data from the infrastructure that one wishes to monitor. For this purpose, a camera within the meaning of the present invention can be any image acquisition device capable of acquiring an image sequence (for example in the form of a video stream).

There schematically represents an environment comprising a monitoring device 10 configured to monitor a hardware infrastructure 2, according to a particular embodiment of the invention. In this example, hardware infrastructure 2 is a level crossing comprising a certain number of hardware components described below.

As already indicated, a level crossing is a railway infrastructure arranged at a crossing between at least one traffic lane 6 (a road and/or pedestrian lane for example) and at least one railway track 4. By definition, this crossing is at level in the sense that the intersecting tracks are arranged at the same height. The material infrastructure 2 (also called below “infrastructure” or “level crossing”) comprises in this example two barriers 8a and 8b (collectively designated 8), also called barrier guards, located on either side of a section 5 of the railway track to control access to the level crossing from the traffic lane 6. To this end, the barriers 8 can be positioned in two different states, namely either in a “closed” state in which the barriers 8 block access to the level crossing, or in an “open” state in which the barriers 8 authorize access to the level crossing. Individuals 16 and vehicles 17 are likely to move on the traffic lane 6 and cross the level crossing (in theory only when the barriers 8 are raised). The barriers 8 are configured to prevent access to the railway track during a time interval corresponding to the passage of a railway vehicle (not shown).

Note that the arrangement of a level crossing may vary depending on the case. Variant embodiments are particularly possible for monitoring level crossings which are devoid of barriers.

Infrastructure 2 also includes traffic lights 14 and bells 16 to warn users (individuals 16, vehicle occupants 17, etc.) of the passage of a rail vehicle. The traffic lights 14 are for example configured to flash at a predefined frequency to indicate the passage of the barriers 8 to the “closed” state. The bells 16 are configured to emit a ringtone to accompany the flashing of the traffic lights 14.

Electrical equipment 9, here taking the form of an electrical cabinet, is also installed to control and ensure the electrical supply of the various components of the infrastructure 2 requiring such a power supply.

Section 5 of railway track (including rails), barriers 8, electrical cabinet 9, traffic lights 14 and bells 16 each constitute material components of infrastructure 2 in the example considered here. Each of these hardware components is likely to be affected by various events and therefore require monitoring. As already indicated, breakdowns, incidents, damage, malfunctions or any other specific events can have an impact on the proper functioning of a hardware component and therefore of the hardware infrastructure 2 as a whole.

Thus, a barrier 8 may encounter a malfunction which prevents it from switching normally between the open and closed positions. Another event may possibly affect a barrier 8, potentially causing a subsequent malfunction (imminent or potential). According to yet another example, section 5 of the railway track may show signs of failure or wear. The presence of an individual 16 or even an object on or near this section 5 of railway track can also constitute an event affecting said section and being able to have an influence on the state of the infrastructure 2.

For monitoring purposes, the monitoring device 10 is connected to cameras 12a and 12b (collectively denoted 12). The number of cameras 12 with which the monitoring device 10 cooperates can be adapted depending on the case and can vary between a single camera and a plurality of cameras. These cameras 12 are configured to monitor (film, observe) the level crossing 2. Each camera 12 is positioned to have in its field of vision at least part of the hardware components of the infrastructure 12, so that each hardware component is monitored by at least one camera 12.

The surveillance device 10 forms with the cameras 12 a surveillance system denoted SY1.

According to a particular example, the cameras 12 transmit a stream of video data DT1 to the monitoring device 10 via appropriate connection means 15, such as wired connections or wireless connections for example.

According to a particular example, the monitoring device 10 also cooperates with at least one sensor 18 to recover DT2 sensor data acquired by said sensor. By way of example, it is assumed that a sound sensor 18, placed in or near the infrastructure 2, is configured to capture sounds in the environment of the infrastructure 2 and transmit data to the monitoring device 10 DT2 sensors representative of these sounds. This sound sensor 18 makes it possible, for example, to pick up a ringing sound emitted by the bells 16 during the passage of a railway vehicle. Embodiments are however possible without sensor 18.

According to a particular example, the cameras 12 and/or the sensors 18 also constitute hardware components of the infrastructure 2 and can therefore as such be monitored by the monitoring device 10. Each camera 12 can for example be monitored by at least another camera 12 so that it can be detected when an event affects said camera 12. Thus, the monitoring device 10 can be configured to detect when the lens of a camera is obstructed by any element (due to for example weather conditions). Likewise, the sound sensor 18 for example can be placed in the field of vision of at least one camera 12 for surveillance purposes.

According to a particular example, the monitoring device 10 itself is part of the infrastructure 2 and can therefore also be monitored in accordance with the monitoring method. To do this, the monitoring device 10 is positioned in the field of vision of at least one camera 12 for monitoring purposes.

According to a particular example, the monitoring device 10 is configured to cooperate with a remote control unit 20, also called "central" hereinafter. This central 20 can include various analysis and processing equipment, for example in the form of at least one server. As described below, the monitoring device 10 can transmit to the central 20 alerts associated with data, in particular context data characterizing the context in which an event causing the alert occurred. For this purpose, the monitoring device 10 can be connected to the central unit 20 by any suitable communication link (wired or wireless type).

There schematically represents the structure of the monitoring device 10 according to a particular embodiment of the invention. More specifically, the monitoring device 10 comprises in this example at least one processor 30, a volatile memory (or RAM) 32 and a non-volatile memory 34.

In this example, the memory 34 is a rewritable non-volatile memory (for example flash, SSD, SSHD, etc.), this memory constituting a recording medium (or information medium) conforming to a particular, readable embodiment. by the processor 30 (and more generally by the monitoring device 10), and on which a computer program PG1 conforming to a particular embodiment is recorded. This computer program PG1 includes instructions for executing the steps of a monitoring process according to a particular embodiment. The main stages of this process are described below, in a particular embodiment of the invention, with reference to Figures 4-7.

As described below, the non-volatile memory 34 is also capable of storing certain data, in particular decision rules RL1 and RL2, data denoted DT1-DT4 as well as alerts AL1, see also historical data H1, H2 . The configuration and use of this data is described later. According to a particular example, the computer program PG1 is recorded in a read only memory (ROM) of the device 10, other than memory 34.

The processor 30 is configured to execute the instructions of the computer program PG1 in order to carry out steps of the monitoring process. For this purpose, the processor 30 can include integrated memory, an input/output interface, and various circuits known to those skilled in the art. In particular, the processor 30 can use the volatile memory 32 to carry out the various operations and functions necessary for the operation of the monitoring device 10, including to execute the computer program PG1 during the implementation of the monitoring method. 'invention.

According to a particular example, the monitoring device 10 comprises a first communication interface 36 for receiving DT1 video data coming from the cameras 12, or if necessary for receiving DT2 sensor data coming from the sensor 18. For this purpose, the first communication interface 36 makes it possible to establish communication with the cameras, and possibly with the sensor 18, via communication channels through communication links 15. The communication interface 36 corresponds for example to an interface connected to a wired network, for example an interface operating according to the “Modbus” or “Profibus” protocol, or even CAN type interfaces (from the English “Controller Area Network” or in French “Réseau de controllers”), CAN FD (from English “Controller Area Network Flexible Data-Rate” or in French “Network of controllers with flexible data rate”), FlexRay (standardized by ISO 17458), Ethernet (standardized by ISO/IEC 802-3) or LIN (from English “Local Interconnect Network”, or in French “Réseau interconnecté local”).

According to a particular example, the monitoring device 10 comprises a second communication interface 38 for sending data to the remote control unit 20 (the central unit), in particular alerts AL1 associated with context data DT4. To do this, the second communication interface 38 ensures the connection with the central 20 via for example a cellular network (4G, 5G communication link or equivalent) or via any other appropriate communication network (Wi-Fi access, etc.) .

The first and second communication interfaces can optionally form a single interface. According to a particular example, the first and/or second communication interfaces 36, 38 comprise at least one of a Wi-Fi interface, a serial port, an Ethernet port or a cellular network interface.

The monitoring device 10 can be positioned within or near the infrastructure 2 to be monitored, but can also be further away. In practice, the monitoring device 10 can take various forms. For example, the monitoring device 10 may comprise a box (or a computer bay, for example in a server room type environment) in which its various constituent components are positioned.

According to a particular example, the monitoring device 10 comprises the components 30-38 forming a calculation unit (called a calculator) and may include additional components cooperating with this calculation unit. By way of example, the monitoring device 10 may further comprise a power supply subsystem 40 to ensure the power supply of the components of the monitoring device 10 ( ). This electrical power subsystem 40 may include, for example, a connection to the outside to collect an electrical supply. According to a particular example, the electrical power subsystem 40 comprises an electrical protection component comprising at least one circuit breaker or fuse to electrically protect the monitoring device 10, which can be particularly useful if the monitoring device 10 is installed in an unstable or unreliable electrical environment.

According to a particular example, the electrical power subsystem 40 comprises an electrical monitoring device which acts as a sensor, capable of switching between two different states, to detect when a loss of power (electrical outage) occurs, this sensor also making it possible to determine if necessary whether this loss of current occurs in the monitoring device 10 itself or upstream. If the electrical outage occurs upstream of the monitoring device 10, the electrical monitoring device remains in a first so-called “normal” state. If, on the other hand, the electrical outage occurs in the monitoring device 10, the electrical monitoring component is in a second so-called “non-functional” state. If the electrical power supply to the monitoring device 10 is restored after a power outage, the monitoring device 10 can then detect the state in which the electrical monitoring component is found and determine, from this state, the origin of the power outage. The monitoring device 10 can be configured to generate DT3 status data and/or DT4 context data representative of the state of the electrical monitoring component, which subsequently makes it possible to determine the origin of the electrical failure.

According to a particular example, the electrical monitoring component mentioned above comprises a microcontroller, a voltage monitoring relay and a battery (or any other suitable source of electrical energy). If the electrical protection component is based on a circuit breaker, an auxiliary contactor configured to indicate the state in which the circuit breaker switch is found, namely either in an open state or in a closed state, can be added. The monitoring relay is responsible for detecting the power outage. Finally, the microcontroller is then configured to read the states of the voltage monitoring relay, the battery and, where applicable, the auxiliary contactor, in order to determine the state of the electrical power subsystem 40 and send this information to the processor 30 (or more generally to the calculation unit).

According to a particular example, the electrical subsystem 40 comprises a power supply interface via Ethernet cable (called PoE for “Power Over Ethernet” in English) which makes it possible to provide electrical power, in addition to data, to the monitoring device 10.

According to a particular example, the monitoring device 10 (or more particularly its calculation unit) comprises an APU (for “Accelerated Processing Unit” in English) or GPU (for “Graphic Processing Unit” in English) type unit configured to implement neural networks, as described below in particular examples.

According to a particular example, the monitoring device 10 is configured to monitor a plurality of infrastructures 2 by implementing the method of the invention as described in this document for each of the infrastructures. In this case, the surveillance system SY1 comprises cameras 12 deployed on the plurality of infrastructures to be monitored so that the surveillance device 10 can apply the surveillance method based on video data coming from each site. It is thus possible to monitor multiple infrastructures simultaneously.

It will be understood that certain elements generally present in a calculator, monitoring device or the like have been deliberately omitted because they are not necessary for understanding the present invention. The monitoring device 10 shown in Figures 1-2 constitutes only exemplary embodiments, other implementations being possible within the framework of the invention. Those skilled in the art understand in particular that certain elements of the monitoring device 10 are only described here to facilitate understanding of the invention, alternative embodiments being possible without these elements.

As shown in according to a particular embodiment, the processor 30 controlled by the computer program PG1 implements for example a certain number of modules, namely: an MD2 obtaining module, an MD4 analysis module, a detection module MD6 and an MD8 generation module.

The obtaining module MD2 is configured to obtain video data DT1 acquired by at least one camera 12 monitoring the hardware infrastructure 2.

The analysis module MD4 is configured to generate, by image analysis applied to video data DT1, state data DT3 representative of respective states of the first and second hardware components over time. As described below, the qualification of “first” hardware component and “second” hardware component can be adapted depending on the case. We can consider by way of example that the barrier 8a constitutes a first material component and that the other material components of the infrastructure 2 (including the other barrier 8b, the section 5 of the railway track, the traffic lights 14, the doorbells 16, etc.) constitute second material components within the meaning of the invention. This qualification of first/second component is for example adapted according to the component for which an event has been detected and for which a corresponding alert is generated as described below.

The MD4 analysis module determines, for example, the DT3 state data by applying first decision rules RL1 defined in a decision model ML1, as described later.

The detection module MD6 is configured to detect, by application of at least one decision rule RL2 taking as input the state data DT3 of the first hardware component, an event EVT1 associated with the first hardware component, in this example 8a, to a time t1.

The generation module MD8 is configured to generate an alert AL1 representative of the event EVT1, this alert being associated with context data DT4 comprising at least in part the state data DT3, of the first and/or second hardware components, defining as the context of the event a temporal evolution of the states of the first and/or second hardware components to allow subsequent event analysis.

According to a particular example, the processor 30 controlled by the program PG1 also implements a control module MD10 and/or an electrical monitoring module MD12.

The control module MD10 is configured to record as history certain data obtained by the monitoring device 10. This module can also trigger the sending of an alert AL1 associated with context data DT4, for example to the remote control unit 20 to cause subsequent event analysis.

The configuration and operation of the MD2-MD12 modules of the monitoring device 10 will appear more precisely in the exemplary embodiments described below with reference to Figures 4-7.

Note that the MD2-MD12 modules as represented in represent only non-limiting examples of implementation of the invention. Generally speaking, for each step of the monitoring method of the invention, the monitoring device 10 of the invention may comprise a corresponding module configured to carry out said step.

With reference to Figures 4-7, a method for monitoring a hardware infrastructure is now described, namely infrastructure 2 as previously described, according to particular embodiments. The monitoring method is for example implemented by the monitoring device 10 previously described with reference to Figures 1-3. To do this, the processor 30 executes the computer program PG1 previously described.

Generally speaking, the monitoring device 10 is capable of monitoring the different hardware components of the hardware infrastructure 2 and of detecting events affecting each of them, for purposes of, for example, diagnosis, monitoring, maintenance, etc. By way of example, it is subsequently assumed that barrier 8a constitutes the first material component – denoted CP1 – within the meaning of the invention and that the other material components – denoted CP2 – of infrastructure 2 (including the second barrier 8b, the railway section 5, the signal lights 14, the bells 16, etc.) constitute second material components within the meaning of the invention. As already indicated, the qualification of “first” hardware component CP1 and “second” hardware component CP2 can be adapted depending on the case. Alternatively, it is thus possible to select as first hardware component CP1 any one of the hardware components of infrastructure 2, the other hardware components then corresponding to the second hardware components CP2.

During a step S2 of obtaining ( ), the monitoring device 10 obtains video data DT1 acquired by at least one camera (namely the cameras 12 in this example) monitoring the hardware infrastructure 2.

For example, the monitoring device 10 can receive this video data DT1 in the form of a stream of video data coming from the cameras 12, this stream possibly comprising a sequence of video data packets. This stream is for example received in real time, so that the monitoring device 10 is able to associate a given instant with each packet of video data received. According to a particular example, the monitoring device 10 implements a clock function indicating a current time, which makes it possible to assign a moment in association with each image (or packet of video data) subsequently processed from the video data DT1.

The monitoring device 10 can optionally receive the video data DT1 from at least one intermediate server which is configured to retrieve the video data DT1 from the cameras 12 and retransmit them to the monitoring device 10.

According to a particular example, the monitoring device 10 recovers (S2) the video data DT1 which are stored beforehand in its memory (memory 34 for example). To do this, the DT1 data can be loaded in any way into the memory then the latter is consulted by the processor 30 to obtain the DT1 video data. In this case, the DT1 video data can be recorded in the form of data packets associated with temporal data indicating a respective instant for each packet. The monitoring device 10 can thus determine a time in association with each image (or video data packet) subsequently processed from the DT1 data.

According to a particular example, the monitoring device 10 also obtains in S2 sensor data DT2 generated by at least one sensor 18, namely the sound sensor 18 in this example. This DT2 sensor data can be obtained in an analogous manner to the DT1 state data as described above.

According to a particular example, the video data DT1, and possibly also the sensor data DT2, are recorded (S3) as historical data in a monitoring history H1, for example in the memory 34 ( ).

During a generation step S4 illustrated in particular in , the monitoring device 10 generates, by an image analysis applied to (or on the basis of) video data DT1, state data DT3 representative of respective states of the first and second hardware components CP1, CP2 during the time. To do this, the analysis carried out in S4 can be based on a calculation model ML1 configured to detect the state of the hardware components CP1, CP2. These calculation models ML1 can in particular be based on first decision rules RL1 which make it possible to determine the state of hardware components CP1, CP2 according to various elements detected in the images, such as the number, the position and, where appropriate the orientation, of elements detected in the images (see also additional factors that depend on the hardware components to be monitored). The implementation of these RL1 decision rules uses mathematical and/or statistical functions. As described below in particular examples, these functions can be implemented by neural networks (artificial intelligence based on machine learning).

For example, the S4 analysis makes it possible to detect whether the barrier 8a constituting the first hardware component CP1 is in an open, closed or transient state (between the open and closed positions). Likewise, we can detect the state in which the following second CP2 hardware components are found:

• section 5 of railway track: detection if state of wear or not, or detection if presence of an individual 16, a vehicle 17 or any other object (for example, section 5 of railway track is detected as being in a state “object positioned on the section of railway track”);
• second barrier 8b: detection if in an open, closed or transient state (between the open and closed positions);
• traffic lights 14: detection of whether they flash or not, see also detection, where applicable, of the flashing frequency of the lights;
• doorbells 16: detection of whether they emit a ringtone or not, see also detection of the frequency of the ringtone if necessary;
• cabinet 9: detection if normal or abnormal visual state; etc.

The state of a first or second component detected in S4 can characterize various aspects of said component at the instance considered, in particular the presence or absence of an object (individual, vehicle or other) occupying, or being on, said component material. For example, in the case where a camera 12 constitutes a second hardware component, of the infrastructure 2, which is monitored by the monitoring device 10, then the state of this camera detected in S4 can be a function for example of the quality images acquired by said camera (for example, brightness level, color saturation, contrast, etc.). If, for example, the lens of one of the cameras 12 is obstructed at least partially due to weather conditions, the quality of the images acquired by this camera may be detected as insufficient (below a minimum quality threshold).

According to a particular example, the state data DT3 further defines a time in association with each state of a respective hardware component (CP1 or CP2). To this end, the monitoring device 10 assigns for example, to each state of a hardware component, temporal data representative of a point in time where said state occurred. As already indicated, these points in time can be determined from a clock function implemented by the monitoring device 10 or by means of temporal data contained in the video data DT1. According to a particular example, the video data DT1 represents images comprising temporal inscriptions (of date and/or time) and the monitoring device 10 performs character recognition to determine, from these temporal inscriptions, the temporal data to be associated to each state in the DT3 state data.

The image processing analysis carried out in S4 can be implemented in various ways. According to a particular example, the MD4 analysis module is an image processing module configured to perform normalization of the DT1 video data recovered as input into a series of images encoded in a predefined format. Thus, processing is applied to the DT1 video data to convert it into the appropriate format, namely the input format of the calculation model(s) subsequently executed during the S4 analysis.

Thus, according to a particular example, the MD4 analysis module carries out encoding (or re-encoding) of the image sequences received in the DT1 video data. This encoding consists in particular of transforming the DT1 video data from their original format as provided by the cameras 12 (this original format may vary depending on the case) into a sequence of images in a predefined format.

According to a particular example, the conversion carried out by the analysis module MD4 in S4 comprises at least one of colorimetric processing and resizing.

As illustrated for example in , during colorimetric processing, the colorimetric representation of the images is modified (or converted), in particular the range of intensity values of the pixels of the images, so as to extend the range of values (conversion from a range of values between 0 to n to a range of values in 0 and m; where m and n are integers such that m > n). If necessary, this conversion makes it possible to reduce certain phenomena such as glare or a lack of contrast for example, so as to improve the images.

During resizing, the size of the frames contained in the DT1 video data are converted into a predefined format suitable for the calculation model(s) used during S4 image analysis. It is in fact advantageous to resize the images so that they correspond to the desired format, for example to the input format of the first neural network RS1 described below. To this end, we can use, for example, a resizing technique called “Letterboxing” or “Pillarboxing” which allows images to be resized without losing their original ratio, so that these images are as detailed as possible without being distorted.

According to a particular example, the state data DT3 generated in S4 are recorded (S5) as historical data in a monitoring history H1, for example in the memory 34 ( ).

During a detection step S6, the monitoring device 10 detects, by application of at least a second decision rule RL2 taking as input the state data DT3 of the first hardware component CP1, of an event EVT1 associated with the first hardware component CP1 at a time t1.

Each decision rule RL2 can define a type of event EVT1 which is detected for a given hardware component if the state of the latter meets a predefined criterion at time t1. An RL2 decision rule can optionally define that an EVT1 event is detected if the states detected for a given hardware component respect at least one predefined criterion over time.

By way of example, an EVT event representing a malfunction or a breakdown can be detected if the barrier 8a constituting the first component CP1 is in the open state in state t1 while a railway vehicle is on the point of entering the level crossing.

According to a particular example, during the detection step S6, the monitoring device 10 detects the event EVT1 by application of at least a second decision rule RL2 taking as input, not only the state data DT3 of the first component CP1, but also the state data DT3 of at least a second hardware component CP2. In other words, said at least one decision rule RL2 takes as input the data DT1 of the first component CP1 combined with the state data DT1 of the second components CP2. For example, an EVT1 event indicating a breakdown of barrier 8a is detected if the states of barriers 8a and 8b are different (one of the barriers is in the open position while the other is in the closed position). By taking into account the state of the second hardware components, we can determine more precisely when an event EVT1 affects the first component CP1.

According to a particular example, the monitoring device 10 extracts (S6) state data DT3 from the previously recorded state data DT3 (S5) as monitoring history H1 over time in S4. The state data DT3 extracted from the history H1 can then be used as input data during the application (S8) of said at least one decision rule RL2 to detect the event EVT1, as described below. -After.

According to a particular example, the monitoring device 10 further obtains (S2) DT2 sensor data acquired by at least one sensor 18 monitoring the hardware infrastructure, other than the cameras 12, this DT2 sensor data being representative of at least one minus one characteristic of the first and/or second hardware component CP1, CP2 over time. During the detection step S6, the monitoring device 10 then detects the event EVT1 by application of said at least one decision rule RL2 taking as input the state data DT1 of the first component CP1 (and possibly also of at least a second component CP2) in combination with the sensor data DT2.

For example, the monitoring device 10 takes into account in S6 DT2 sensor data generated by the sound sensor 18, these DT2 data being representative of a sound environment of the hardware infrastructure 2. By an analysis of sounds extracted from the sensor data DT2, the monitoring device 10 determines whether or not a ringtone is emitted at time t1 and, optionally, determines whether the frequency of the ringtone conforms to a predefined frequency. Depending on the result of this sound analysis, the monitoring device 10 detects whether an event EVT1, representative of a malfunction of the barrier 8a (first component CP1), occurs at time t1. For example, a Fourier transform frequency analysis can be performed based on sounds extracted from DT2 sensor data. The frequency at which the doorbells 16 are supposed to ring is for example defined according to a given standard. Thus, if the detected sounds do not meet predefined criteria based on this standard, an EVT1 event representing a malfunction can be detected in S6.

Generally speaking, one or more sensors 18 can be installed to acquire different characteristics of the infrastructure 2 over time and thus make it possible to enrich the input data used by the decision rules RL2 to detect the occurrence of a EVT1 event. These DT2 sensor data are metrics that depend on the sensors used. For example, a ringtone is detected in the sound environment of infrastructure 2 if ringing sounds extracted from DT2 sensor data reach at least a threshold sound intensity.

During a generation step S8, the monitoring device 10 generates an alert AL1 (or at least one alert AL1) representative of the event EVT1 detected in S6. This alert indicates for example that the barrier 8a is defective or presents a particular condition which requires verification or maintenance action (preventive or curative). In addition, this alert AL1 is associated (S8) with context data DT4 comprising at least part of the state data DT3 previously generated by the monitoring device 10, namely state data DT3 of the first hardware component CP1 and/or state data DT3 of at least one second hardware component CP2. These context data DT4 define, as context of the alert AL1 (and therefore of the event EVT1), a temporal evolution of the states of the first and/or second hardware components CP1, CP2. In other words, the context data DT4 defines a temporal evolution, either of the first component CP1 only, or of one or more second components CP2 only, or of the first component CP1 and at least one second component CP2 (see all components CP1 and CP2).

Thus, the context data DT4 used in S8 includes for example certain state data DT3, components CP1 and/or CP2 over time, these data DT4 being selected to contextualize the occurrence of the alert AL1 (and therefore of the EVT1 event), particularly over time. The monitoring device 10 can determine the context data DT4 before the alert generation step S8, see before the step S6 of detecting the event EVT1.

According to a particular example, the context data DT4 used in S8 define as context of the event EVT1 a temporal evolution of the respective states of the first and/or second hardware components CP1, CP2 during a period of time preceding the 'time t1. For example, the DT4 context data defines an evolution of the state of components CP1 and/or CP2 during a period of 1 hour (3600 seconds) preceding t1, or ending with t1.

In the case where the context data DT4 defines in particular a temporal evolution of the state of at least a second component CP2, these data provide not only a temporal but also a spatial context to the extent that these data characterize the behavior of hardware components other than CP1 in infrastructure 2.

According to a particular example, the AL1 alert and the associated DT4 context data are incorporated or recorded (S8) in a file. In other words, the monitoring device 10 generates a file in which it inserts the alert AL1 in association with the context data DT4.

The DT4 context data used in S8 are intended, for example, to enable subsequent event analysis to better understand the situation and, if necessary, trigger corrective or preventive actions on the hardware infrastructure 2. Various uses of the DT4 context data can thus be considered to enable effective maintenance of the hardware infrastructure 2.

According to a particular example, the monitoring device 10 sends (S10) the alert AL1 in association with the context data DT4 to the remote control unit 20 (the central unit) to cause a subsequent event analysis. This data may be sent in the form of alert notifications in any suitable format (e.g. by email, SMS, etc.). The control unit 20 may include computer means adapted to analyze the alert AL1 in light of the context represented by the context data DT4. The central unit 20 may in particular have greater processing resources than the monitoring device 10 and may also be configured to receive and process alert notifications from a plurality of monitoring devices 10 each monitoring at least one hardware infrastructure.

According to a particular example, the monitoring device 10 records (S12) as alert history H2 the alert AL1 in association with the context data DT4, for example in its non-volatile memory 34 ( ). It is thus possible to subsequently carry out an event analysis from AL1 and DT4 consulted in memory 34.

According to a particular example, if no connection is possible with the control unit 20 at a current time, the monitoring device 10 can record S12 of the data AL1 and DT4 in the alert history H2 and subsequently transmit this data to the central 20 upon detection that the connection (communication link) is reestablished.

According to a particular example, during the detection step S6 ( ), the monitoring device 10 detects the event EVT1 by application of at least one decision rule RL2 on the basis of the state data DT3 of the first component CP1 combined with the context data DT4. To do this, the DT4 context data must be determined before the detection state S6. We can thus advantageously refine the detection of an event EVT1 by taking into account not only the state of the first hardware component CP1 (the barrier 8a in this example) but also the hardware context in which the state of the first hardware component CP1 occurs. within infrastructure 2, so as to further improve event detection (in particular to avoid false event detections).

According to a particular example, the monitoring device 10 detects, by means of the electrical subsystem 40 ( ), a power cut (or loss of power) in the hardware infrastructure 2 and/or in the monitoring device 10. This may be a power cut in the infrastructure 2 at the current time t1, or again of a power cut which occurred at time t1 in the monitoring device 10 but which is only detected by the latter later once its power supply is restored (as described above, by example from the closed or open state of the circuit breaker of the electrical subsystem 40). The monitoring device 10 can then insert into the context data DT4 electrical data representative of the power outage, in order to further enrich the context of the alert AL1 and therefore of the event EVT1.

These electrical data can for example define an evolution over time of a state of an electrical supply in the hardware infrastructure 2 (upstream of the monitoring device 10) and/or in the monitoring device 10 itself, this evolution including said power cut. It is thus possible to also obtain a context over time of the electrical state of the infrastructure 2 and/or the monitoring device 10.

The monitoring device 10 detects for example if an electrical characteristic (in voltage, current, etc.) of the infrastructure 2 or of the monitoring device 10 itself is lower or higher than a predefined threshold, and deduces a state therefrom. electrical which, if necessary, serves as a context for the AL1 alert.

As indicated previously, the DT3 state data is generated in S4 by performing an image analysis applied to the DT1 video data, and possibly also to the DT2 sensor data (if applicable). This S4 image analysis illustrated in particular in can be based on a neural network system using calculation models obtained by learning.

In reference to the , the image analysis S4 implemented by a neural network system 60 is now described, according to a particular embodiment. In this example, the neural system 60 implemented by the monitoring device 10 comprises at least two neural networks, namely a first neural network RN1 and a second neural network RN2, which are configured in series to cooperate together so as to produce the data DT3 status previously described. To do this, these neural networks RN1, RN2 can be configured to implement mathematical and/or statistical functions from respective calculation models.

More precisely, the first neural network RN1, for example of the convolutional type, detects objects in images defined by the video data DT1 obtained in S2 then classifies these objects to detect objects corresponding respectively to the first hardware component CP1 and to the second hardware components CP2. For this purpose, the first neural network takes for example as input the image data DT1, and possibly also the sensor data DT2, and carries out from these input data the aforementioned detection and classification steps on the basis of 'a calculation model (or detection model) ML1. The first neural network RN1 thus classifies in its results RS1 objects OB1 as a given material component among the possible material components (namely the first component CP1 or a second component CP2).

For example, the RS1 results for an iteration i define OB1 objects as “n-tuples”, namely couplets [classification CP1 or CP2, time t], or even triplets [classification CP1 or CP2, time t, spatial coordinates CD1]. The dimension n of the tuples depends on the number of metadata associated with the same OB1 object. The same object OB1 can be defined by several couplets or triplets corresponding to several viewing angles of different cameras 12 on said object.

According to a particular example, each n-tuple may include a unique identifier associated with the OB1 object considered.

The CP2 classification may possibly include subclassifications CP2a, CP2b, CP2c, etc. identifying the respective subclass of each second hardware component 5, 8b, 9, 14, 16, etc. RS1 results can thus define, for example, an object OB1 as being the first component CP1 (namely the barrier 8a) located at coordinates CD1 given at time t.

According to a particular example, the neural system 60 further comprises a third neural network RN1a (not illustrated) configured to take as input all or part of the results RS1 of the first neural network RN1 and to carry out an image analysis from this input. , such as character recognition. For example, the third neural network RN1a can perform character recognition on a license plate of a vehicle present in the images as object OB1. It is thus possible to identify a vehicle 17 crossing the level crossing 8 and to analyze its behavior over time, in particular to determine whether this vehicle is the cause of a detected event (breakdown, malfunction, etc.). ). According to another example, the third neural network RN1a can perform character recognition to determine an instant in time from a time and/or date inscription present in the images defined in the video data DT1.

According to a particular example, the first neural network RN1 is previously subjected to a machine learning phase during which it generates the calculation model ML1 from a sampling base DS1. This model ML1 is for example generated from a sampling base (or “dataset” in English) DS1 comprising a plurality of image samples (synthetic or not) for which objects corresponding to material components of infrastructure are identified. The first network RN1 can thus train itself to detect and classify objects corresponding to hardware components of an infrastructure.

For example, to specifically detect level crossing barriers, a sampling base can be constructed by embedding images of barriers (in multiple positions and angles) into various image samples (a range , a forest, a supermarket, etc.) so that the model fits only the specific shape of the barrier.

The results RS1 of the image analysis carried out by the first neural network RN1 are then provided as input to the second neural network RN2. These RS1 analysis results can include the classified OB1 objects (for example each object is associated with an object identifier and a class identifier CP1 or CP2) as well as, possibly, the DT1 video data (variants are however possible in (which DT1 results do not include DT1 video data as such). According to a particular example, the analysis results RS1 of the first neural network RN1 further include spatial coordinates CD1 of the objects OB1 to allow localization of these objects relative to the hardware infrastructure 2.

Still with reference to the , the second neural network RN2 takes as input the results RS1 of the first neural network RN1, and possibly also the sensor data DT2 (if applicable). The second neural network RN2 determines from the input data the respective state of the objects OB1 (as classified in the analysis results RS1) on the basis of a decision model ML2 to produce results as output. RS2 comprising the state data DT3 representative of the respective states, at a current time t1, of said objects OB1. Thus, the DT3 state data represents respective states in which the OB1 objects classified as the first or second hardware component are located in the RS1 analysis results. This ML2 decision model reflects in particular the first decision rule(s) RL1 mentioned previously, which are used to produce the DT3 state data.

To produce this DT3 state data, the second neural network RN2 can optionally also take into account the DT2 sensor data (if applicable).

For example, the RS2 results for an iteration i define triplets [classification CP1 or CP2, time t, state], or even quadruplets [classification CP1 or CP2, time t, state, spatial coordinates CD1]. RS2 results can thus define for example an object OB1 as being the first component CP1 (namely the barrier 8a) in the state either "open" or "closed" or "transient", located at coordinates CD1 given to the instant t.

According to a particular example, the second neural network RN2 is of the recursive type ( ). Due to its recurrent nature, the second neural network RN2 also takes as input part of its own past RS2 results. In other words, the second network RN2 takes into account part of its result history to produce RS2 as output. More precisely, the second neural network RN2 may comprise a regulation loop which reinjects past RS2 results to calculate the RS2 results of a current iteration. For example, during a current iteration i+1, the second neural network RN2 produces as output a result RS2 _i+1 by taking as input the results RS1 _i+1 of the first neural network RN1 for the current iteration i+1 combined with the past results RS2 _i of the second neural network RN2 for the previous iteration i. The second neural network RN2 can thus use its past results as a reference to calculate its results for subsequent iterations. To do this, the monitoring device 10 keeps in memory its old RS2 i results for a past iteration _i in order to be able to use these old results for the calculation of the RS2 results of the next iteration i + 1 and thus refine the parameters of its ML2 decision model. This recurrence makes it possible, for example, to reduce false positives based on maintenance study feedback or to avoid retraining the second neural network RS2 to improve its model.

According to a particular example, the second neural network RN2 is first subjected to a machine learning phase during which it generates the decision-making model ML2 from a sampling base (or “dataset” in English) DS2. This ML2 model is for example generated from a sampling base DS2 comprising a plurality of samples comprising objects classified according to a given class CP1/CP2 (or even a subclass among the class CP2).

Generally speaking, the ML1 and ML2 models are adaptable depending on the hardware infrastructure 2 that you wish to monitor, which makes it possible to optimize the monitoring results.

The present invention makes it possible to carry out reliable and effective monitoring of an infrastructure comprising multiple hardware components likely to be affected by normal or abnormal events, such as for example damage, malfunction, passage into a predefined state, etc. This monitoring can be carried out automatically by an autonomous monitoring device. This effective monitoring is possible without resorting to human intervention as previously described.

In particular, thanks to the context data which is associated with the alerts produced by the monitoring device, it is possible not only to detect a particular event affecting a hardware component of the infrastructure, but also to analyze and understand the context in which said event occurred. It is thus possible to quickly detect (for example in real time) events affecting a hardware component and to carry out curative and/or preventive maintenance actions in order to guarantee that the infrastructure remains in good condition.

The use of a particular architecture of neural networks makes it possible in particular to effectively detect the state of the various hardware components of an infrastructure to be monitored and to deduce particular events that may require maintenance action.

Such maintenance action may consist of repairing or replacing all or part of a material component, expelling an individual present on the level crossing or even carrying out checks to confirm or not the alert.

As already indicated, the present invention can be applied to various types of hardware infrastructures in very diverse contexts (industrial or not).

As those skilled in the art understand, the embodiments and variants described above constitute only non-limiting examples of implementation of the invention. In particular, those skilled in the art may consider any adaptation or combination of the embodiments and variants described above, in order to meet a very specific need in accordance with the claims presented below.

Claims (10)

Monitoring method, implemented by a monitoring device (10), for monitoring a hardware infrastructure (2) comprising a first hardware component (CP1) and at least one second hardware component (CP2), said method comprising:
- obtaining (S2) video data (DT1) acquired by at least one camera (12) monitoring the hardware infrastructure;
- generation (S4), by image analysis applied to video data, of state data (DT3) representative of respective states of the first and second hardware components over time;
- detection (S6), by application of at least a second decision rule (RL2) taking as input the state data of the first hardware component, of an event (EVT1) associated with the first hardware component at a time t1; And
- generation (S8) of an alert (AL1) representative of the event, said alert being associated with context data (DT4) comprising at least in part the state data of the first and/or second hardware components, defining as context of the event a temporal evolution of the states of the first and/or second hardware components. Method according to claim 1, in which said event is detected by application of said at least one second decision rule (RL2) taking as input the state data (DT3) of the first hardware component (CP1) combined with the context data (DT4 ). Method according to claim 1 or 2 in which, during the image analysis, a first neural network (RN1) previously subjected to machine learning performs the following steps:
- object detection (OB1) in images defined by video data; And
- object classifications for detecting objects corresponding to the first and second hardware components (CP1, CP2) in respective states. Method according to claim 3 in which, during the image analysis, a second neural network (RN2) previously subjected to machine learning takes as input at least a first decision rule (RL1) and results (RS1) of the first neural network to determine the state data at output (DT3). Method according to any one of the preceding claims, further comprising:
- reception of sensor data (DT2) acquired by at least one sensor (18) monitoring the hardware infrastructure, other than said at least one camera, said sensor data being representative of at least one characteristic of the first and/or second material component over time;
wherein said event (EVT1) is detected by applying said at least one second decision rule (RL2) based on the state data (DT3) of the first hardware component in combination with the sensor data (DT2). Method according to any one of the preceding claims, in which the context data (DT4) defines as context of the event a temporal evolution of the respective states of the first and/or second hardware components (CP1, CP2) during 'a period of time preceding time t1. Method according to any one of the preceding claims, the method further comprising:
- sending the alert (AL1) in association with the context data (DT4) to a remote control unit (20) to cause subsequent event analysis. Method according to any one of the preceding claims, wherein the monitoring device comprises a non-volatile memory (34), the method further comprising:
- recording (S12) as event history (H2), in the non-volatile memory, of the alert in association with the context data to allow subsequent event analysis. Computer program (PG1) comprising instructions for executing the steps of a method according to any one of the preceding claims when said program is executed by a computer. Monitoring device (10) for monitoring a hardware infrastructure (2) comprising a first hardware component (CP1) and at least one second hardware component (CP2), said device comprising:
- obtaining module (MD2) configured to obtain video data (DT1) acquired by at least one camera (12) monitoring the hardware infrastructure;
- analysis module (MD4) configured to generate, by image analysis applied to video data, state data (DT3) representative of respective states of the first and second hardware components over time;
- detection module (MD6) configured to detect, by application of at least a second decision rule (RL2) taking as input the state data of the first hardware component, of an event (EVT1) associated with the first hardware component to be a time t1; And
- generation module (MD8) configured to generate an alert (AL1) representative of the event, said alert being associated with context data (DT4) comprising at least in part the state data of the first and/or second hardware components, defining as event context (EVT1) a temporal evolution of the states of the first and/or second hardware components.