FR3115594A1 - Electronic system for implementing a critical function, associated method and computer program - Google Patents

Abstract

Electronic system for implementing a critical function, method and associated computer program This electronic system (10) for implementing a critical function comprises a first processing module (14) for calculating a first intermediate datum ( SA) from a first input data (A) and a reinforcement learning algorithm; a second processing module (16) for calculating a second intermediate datum (SB) from said algorithm and a second input datum (B); an auxiliary processing module (18) for calculating an intermediate reference datum (SR) from the first and/or second input datum and a reference algorithm; and a monitoring device (20) for checking whether the intermediate data belongs to a security domain and for comparing them in order to detect any inconsistency; and to generate an output datum (C) as a function of the intermediate data if they belong to the security domain and without inconsistency, the output datum being the reference intermediate datum otherwise. Figure for abstract: Figure 1

Description

Electronic system for implementing a critical function, associated method and computer program

The present invention relates to an electronic system configured to implement a critical function.

The invention also relates to a method for implementing a critical function by such an electronic system.

The invention also relates to a computer program comprising software instructions which, when executed by a computer, implement such a method.

The electronic system is in particular embedded in an installation or in a device. The apparatus is preferably a vehicle, such as an air vehicle, in particular an aircraft, a railway vehicle or a motor vehicle. Alternatively, the installation is for example a factory or a power plant.

In particular, the invention relates to the implementation of critical functions, that is to say which are critical for the safety of the device or installation. Vehicle piloting controls, such as the flight controls of an aircraft, the braking system of the vehicle, the emergency shutdown of a chemical plant or a power plant are examples of such critical functions.

In the avionics field, a critical function is for example defined by the ARP-4754A ( Aerospace Recommended Practice ) standard.

By implementation of a critical function, we mean the performance of one or more calculations making it possible to generate at least one output datum associated with this critical function, from at least one input datum.

In order to ensure the safety of the device or installation, it is therefore necessary to ensure that the implementation of the critical function is carried out without failure. In particular, such a critical function is conventionally associated with a security usage domain defining a range of authorized value(s) for the output data associated with this critical function. Output data outside the range of authorized value(s) will be refused, and an alert will then be issued. In the avionics field, this range of authorized value(s) is for example values comprised between a minimum limit and a maximum limit such as for example maximum and minimum altitude limits of the aircraft. As a variant, the range of value(s) corresponds to a maximum limit or a minimum limit not to be exceeded, such as a limit of rate of climb of the aircraft or a maximum power of the engine of the aircraft. As a further variant, the range of value(s) corresponds to a single value, typically 0 or 1, associated with a Boolean indicator, such as a working state of an avionics system of the aircraft.

Reinforcement learning algorithms are known which aim to gradually learn a function or a behavior model, by carrying out successive experiments and with feedback from a performance indicator of the function. Once the learning has been carried out, these reinforcement learning algorithms make it possible to improve the performance of the implementation of the critical function. In addition, reinforcement algorithms make it possible to model critical functions that cannot be modeled conventionally, for example via physical laws, due to the excessive complexity of the system.

However, these algorithms have vulnerabilities that affect the security of the electronic system. Indeed, the implementation of the critical function by these reinforcement learning algorithms gives, in certain cases, a result that does not belong to the security domain associated with the critical function, which possibly leads to significant consequences on the safety of the installation or device in which the system is embedded. These vulnerabilities thus greatly limit the use of these algorithms for the implementation of critical function(s).

There is therefore a need to obtain a more efficient electronic system for implementing a critical function while being sufficiently safe.

To this end, the subject of the invention is an electronic system configured to implement a critical function, the electronic system being able to receive first and second input data and to deliver output data associated with said critical function, the output datum being associated with a predetermined security domain, the security domain being in the form of a range of predefined values,

the electronic system comprising:

- a first processing module configured to calculate a first intermediate datum from the first input datum and a reinforcement learning algorithm, for the implementation of the critical function for the first input datum,

- a second processing module configured to calculate a second intermediate datum from the second input datum and from the reinforcement learning algorithm, for the implementation of the critical function for the second input datum, the second input data being different from the first input data,

- an auxiliary processing module configured to calculate an intermediate reference datum from the first input datum and/or from the second input datum and from a reference algorithm different from the reinforcement learning algorithm , for the implementation of the critical function for the first input datum and/or the second input datum via the reference algorithm,

- a monitoring device comprising:

+ a control module configured to check whether the first and second intermediate data belong to the security domain and to compare the first and second intermediate data with each other in order to detect any inconsistency;

+ a generation module configured to generate the output data as a function of the first and second intermediate data if the first and second intermediate data belong to the security domain and if no inconsistency is detected, the output data being equal to the data reference intermediary otherwise.

According to other advantageous aspects of the invention, the electronic system comprises one or more of the following characteristics, taken in isolation or in all technically possible combinations:

- the set of implementations of the critical function by the reference algorithm defines an arrival domain of the reference algorithm, the arrival domain being included in the security domain;

- the monitoring device comprises a reward module configured to calculate a first reward for the implementation of the critical function by the first processing module and a second reward for the implementation of the critical function by the second processing module ,

the first processing module being configured to implement the reinforcement learning algorithm based on a first plurality of algorithmic parameters, the first processing mold being configured, at each iteration, to receive the first reward and to modify the first plurality of algorithmic parameters according to the first reward received in order to maximize the next first reward received,

the second processing module being configured to implement the reinforcement learning algorithm based on a second plurality of algorithmic parameters, the second processing module being configured, at each iteration, to receive the second reward and to modify the second plurality of algorithmic parameters as a function of the received second reward to maximize the next received second reward;

the reward module is configured to penalize each first reward and/or respectively each second reward when the first intermediate datum and/or respectively the second intermediate datum does not belong to the security domain;

- when the first and second intermediate data belong to the security domain and in the absence of detected inconsistency, the generation module is configured to deliver, as output data, the intermediate data associated with the highest reward calculated by the reward module;

- the control module is further configured to compare the first and second rewards with each other in order to detect any inconsistency,

and in which the electronic system further comprises an alert module configured to generate an alert signal based on each inconsistency between the intermediate data and/or between the rewards;

- the control module is further configured to count the number of times respectively where the first and second intermediate data do not respect the security domain and to reset respectively the first plurality of algorithmic parameters and the second plurality of algorithmic parameters when said number is greater than a predetermined threshold; And

- the system is an avionic system suitable for being onboard in an aircraft control station, in particular an aircraft navigation system.

The invention also relates to a method for implementing a critical function by an electronic system, the electronic system being configured to implement a critical function, the electronic system being capable of receiving first and second input data and in delivering an output datum associated with said critical function, the output datum being associated with a predetermined security domain, the security domain being in the form of a range of predefined value(s),

the method comprising the following steps:

- calculation of a first intermediate data from the first input data and a reinforcement learning algorithm, for the implementation of the critical function for the first input data,

- calculation of a second intermediate datum from the second input datum and the reinforcement learning algorithm, for the implementation of the critical function for the second input datum, the second datum of input being different from the first input data,

- calculation of an intermediate reference datum from the first input datum and/or from the second input datum and from a reference algorithm different from the reinforcement learning algorithm, for the implementation implementation of the critical function for the first input datum and/or the second input datum via the reference algorithm,

- verification whether the first and second intermediate data belong to the security domain and comparison of the first and second intermediate data with each other in order to detect any inconsistency;

- generation of the output data according to the first and second intermediate data if the first and second intermediate data belong to the security domain and if no inconsistency is detected, the output data being equal to the reference data otherwise.

The invention also relates to a computer program comprising software instructions which, when executed by a computer, implement a method as defined above.

These characteristics and advantages of the invention will appear more clearly on reading the following description, given solely by way of non-limiting example, and made with reference to the appended drawings, in which:

there is a schematic representation of an electronic system according to the invention,

there is a schematic representation of the implementation of a learning reinforcement algorithm, and

there is a flowchart of a method, according to the invention, for implementing a critical function by the electronic system of the .

An electronic system 10 is represented on the . The electronic system 10 is configured to implement at least one critical function.

In the example of the , the electronic system 10 is on board an aircraft. The aircraft is typically an airplane, a helicopter, or even a drone. In other words, the aircraft is a flying machine that can be controlled by a pilot via a control station, the control station being located inside the aircraft or else at a distance from the aircraft, in particular in the case of a drone.

In this example, the electronic system 10 is then an avionics system configured to implement an avionics critical function.

The avionics critical function is then typically chosen from the group consisting of: a calculation of the trajectory of the aircraft, a flight control of the aircraft, the strategy for avoiding an obstacle on the trajectory of the aircraft, a aircraft braking function, etc.

Those skilled in the art will understand that the invention also applies to other critical functions, such as a reactivity control process in a nuclear reactor, temperature control in a furnace of a factory , to a navigation control of an autonomous motor vehicle, to a speed control of a railway vehicle, etc.

The electronic system 10 is able to receive first and second input data, denoted A and B hereafter, and to deliver an output data C associated with said critical function.

The second input data B is distinct and different from the first input data A.

Each input data A, B is provided by a respective upstream electronic device 12A, 12B.

The upstream electronic device 12A associated with input data A is different from the upstream electronic device 12B associated with input data B.

Each upstream electronic device 12A, 12B is for example on board the aircraft. Each upstream electronic device 12A, 12B is in particular a sensor on board the aircraft, for example a camera, a position sensor or a temperature sensor. As a variant, each upstream electronic device 12A, 12B is another system, in particular another avionic system.

As a further variant, each upstream electronic device 12A, 12B is a system external to the aircraft, such as a control tower communicating the input data A, B associated with the aircraft.

Advantageously, the first and second input data A, B are of the same type. In other words, the type of the values likely to be taken by the first and second data is identical for these first and second data.

In particular, this type is chosen from the group consisting of: an image, a video stream, a measurement or a succession of measurements of one or more operating parameters of the electronic system 10, a measurement signal from a sensor, a command from a user of the electronic system 10, a text (that is to say a succession of alphanumeric characters) from information or voice recognition notices .

By way of example, the first and second input data A, B are images of the environment of the aircraft originating from two different on-board cameras, aircraft location data originating from two different sensors, avionic parameters, such as the fuel level or the operating state of one or more avionic systems, or else are flight commands from two different redundant piloting systems.

Alternatively, the first and second input data A, B are of different types.

By way of example, the first input data A is an image of the environment of the aircraft from a camera on board the aircraft, and the second input data B is a digital representation of the terrain overflown by the aircraft from a geographic database.

The output data C is advantageously a numerical value or Boolean indicator, typically equal to 0 or 1.

The output data C is for example a value of a flight command, a speed or altitude setpoint for the aircraft, an activation/deactivation setpoint for an avionics system, etc.

The output data C is associated with a predetermined security domain, the security domain being in the form of a range of predefined value(s).

Thus, when the output data C is a digital value, the security domain is for example an interval of values between two predetermined limits or an open interval defined by a minimum or maximum threshold value or even a set of such intervals. When the output data is a Boolean flag, typically equal to 0 or 1, then the security domain is a single value.

The electronic system 10 comprises a first processing module 14, a second processing module 16, an auxiliary processing module 18 and a monitoring device 20.

As an optional supplement, the electronic system further comprises an alert module 21 and a display module 23.

The first processing module 14, the second processing module 16 and the auxiliary processing module 18 are for example each embedded in a respective separate electronic computer.

Each electronic computer advantageously comprises its own power supply and its own calculation unit. The electronic computer is, for example, in the form of an electronic module independent of other electronic module(s) and capable of being installed in a rack, not shown, or even in the form of a card electronic, independent of other electronic card(s), and able to be installed in an electronic cabinet. As a variant, the electronic computer has its own electronic box, and is then the only computer placed inside a protective box associated with the box.

The first processing module 14 is configured to calculate a first intermediate datum SA from the first input datum A and from a reinforcement learning algorithm for the implementation of the critical function.

In other words, the first processing module 14 is configured to implement the critical function for the first input datum A, by applying the reinforcement learning algorithm to the first input datum A, this in order to obtain the first intermediate datum SA.

The operation of a reinforcement learning algorithm is represented schematically on the . In a known manner, reinforcement learning consists, for an autonomous agent X such as an algorithm, in a current state S, in learning actions F to perform, from experiences, so as to optimize a quantitative reward R over time. Agent X is immersed in the environment E, and makes its decisions according to its current state S. In return, the environment E provides agent X with a positive or negative reward R and the agent X evolves into a new current state S'. The agent X seeks, through iterated experiments, to optimize a decisional behavior called strategy or policy, and which is a function associating the current state S with the action F to be executed, in order to maximize the sum of the rewards R over time.

Thus, in such a problem, we say that the agent X interacts with the environment E to find the optimal solution. Reinforcement learning differs fundamentally from supervised and unsupervised problems by this interactive and iterative side: agent X tries several solutions (we speak of "exploration"), observes the reaction of the environment E and adapts its behavior to find the best strategy.

The main families of reinforcement learning algorithms are in particular the so-called "Q-learning" methods, the variants of Q-learning such as the so-called "Deep Q-learning" methods, the so-called "Actor-Critic" methods , bandit algorithms, Markovian decision algorithms and game trees.

As will be explained below, a reward is determined by the control device 20 in response to the first intermediate datum SA and is sent back to the first processing module 14. The first processing module 14 is then capable of changing the reinforcement learning algorithm based on this reward for calculating the next first intermediate datum SA in order to maximize the next reward received.

In particular, the first processing module 14 is configured to preferentially apply a so-called “Q-learning” method, a so-called “Actor-Critic” method or a variant of these methods known to those skilled in the art.

Advantageously, the first processing module 14 is configured to implement the reinforcement learning algorithm according to a first plurality of algorithmic parameters.

Each algorithmic parameter makes it possible in particular to control the importance of a variable or a function in the calculations performed by the reinforcement learning algorithm. Thus, the modification of one or more algorithmic parameters makes it possible to evolve and adjust the reinforcement learning algorithm during its exploitation.

The so-called “Q-learning” method typically aims to build a table making it possible to link each possible input data to a cost, by adjusting the costs progressively, corresponding to the values of the table.

The first processing module 14 is configured to implement the reinforcement learning algorithm with the so-called “Q-learning” method if the number of possible input data, or even considered or taken into account, for the first input data A is low, typically if this number of input data is less than a predefined threshold. The first processing module 14 is typically configured to implement the reinforcement learning algorithm with the so-called “Q-learning” method if a combination of input data for the first input data A is low, such as lower than a corresponding predefined threshold.

The so-called "Deep Q-learning" method typically builds a neural network which calculates the cost from each possible input datum, with an adjustment of the parameters, such as weights, of the neural network, for example by a method of gradient descent.

The first processing module 14 is configured to implement the reinforcement learning algorithm with the so-called “Deep Q-learning” method if the number of possible input data, or even considered or taken into account, for the first input data A is high, in particular too large to produce a table, typically if this number of input data is greater than the predefined threshold. The first processing module 14 is typically configured to implement the reinforcement learning algorithm with the so-called “Deep Q-learning” method if the combination of input data for the first input data A is high , such as greater than the predefined corresponding threshold.

For the so-called “Actor-Critic” method, the principle is to estimate a function of cost and value. These functions are generally modeled by neural networks, by adjusting the weights of these neural networks.

The first processing module 14 is typically configured to implement the reinforcement learning algorithm with the so-called “Actor-Critic” method if the number of possible input data, or even considered or taken into account, for the first input data A is high, in particular too large to produce a table, typically if this number of input data is greater than the predefined threshold.

The algorithmic parameter is then typically the number of possible input data, considered or taken into account; or even the combination of input data for the first input data A.

The first processing module 14 is further configured to send the first intermediate data item SA to the monitoring device 20.

The second processing module 16 is configured to calculate a second intermediate datum SB from the second input datum B and the reinforcement learning algorithm for the implementation of the critical function.

In other words, the second processing module 16 is configured to implement the critical function for the second input datum B, by applying the reinforcement learning algorithm to the second input datum B, this in order to obtain the second intermediate datum SB.

The critical function is therefore implemented by the second processing module 16 with the same algorithm as that used by the first processing module 14, but with different input data.

Advantageously, the second processing module 16 is configured to implement the reinforcement learning algorithm according to a second plurality of algorithmic parameters.

Those skilled in the art will understand that the first plurality of algorithmic parameters and the second plurality of algorithmic parameters are similar and make it possible to adjust the same variables or functions, but possibly have different numerical values for each algorithmic parameter depending respectively on the first and second awards received.

The second processing module 16 is further configured to send the second intermediate datum SB to the monitoring device 20.

The auxiliary processing module 18 is configured to calculate an intermediate reference datum SR from the first input datum A and/or from the second input datum B and from a reference algorithm for the implementation of the critical function.

In other words, the auxiliary processing module 18 is configured to implement the critical function for the first input datum A and/or the second input datum B, by applying the reference algorithm to the first datum of input A and/or to the second input datum B, in order to obtain the intermediate reference datum SR.

The auxiliary processing module 18 is for example configured to implement the critical function only from the first input data A or only from the second input data B. In particular, the auxiliary processing module 18 is configured to take as input the first input data A as a priority, and otherwise, in particular in the event of failure of the first upstream electronic device 12A, to take as input the second input data B.

Alternatively, the auxiliary processing module 18 is configured to put the critical function from both the first input data A and the second input data B.

As a further variant, the auxiliary processing module 18 is configured to implement the critical function only from the first input datum A; and separately implement the critical function solely from the second input datum B. According to this variant, the auxiliary processing module 18 is then configured to additionally supply the monitoring device 20 with a second intermediate reference datum SR' associated with the second implementation of the critical function.

The benchmark algorithm is different from the reinforcement learning algorithm. Thus, the reference algorithm is not configured to implement reinforcement learning.

The set of implementations of the critical function by the reference algorithm defines an arrival domain of the reference algorithm. Advantageously, the arrival domain is included in the security domain. Thus, the implementation of the critical function by the auxiliary processing module 18 outputs an intermediate reference datum SR always included in the safety domain of the critical function, thus guaranteeing the safety of the aircraft.

The reference algorithm is in particular an algorithm that does not implement an automatic learning method. In particular, the reference algorithm is a deterministic algorithm consisting, for example, of a succession of conditional instructions (“if, then” type) and/or based on the implementation of a physical model consisting of a set of equations describing the operation of the system, the equations being in particular established by a person skilled in the art from physical laws, possibly simplified, or even signal processing or filtering methods. The signal processing or filtering methods are for example methods implementing digital filters, Kalman filters, known per se.

Those skilled in the art will then understand that the reference algorithm is a deterministic algorithm, that is to say entirely justifiable, which is not the case with the reinforcement learning algorithm which is built as as the tests progress, in particular as it learns.

The critical function is therefore implemented by the auxiliary processing module 18 with the same input data A, B as the first and second processing modules 14, 16 but with a different algorithm.

The auxiliary processing module 18 is further configured to send the intermediate reference data(s) SR, SR' to the monitoring device 20.

The monitoring device 20 comprises a control module 22 and a generation module 24.

In addition, the monitoring device 20 further comprises a reward module 26.

The reward module 26 is configured to calculate a first reward RA associated with the implementation of the critical function by the first processing module 14 and a second reward RB associated with the implementation of the critical function by the second processing module. treatment 16.

In particular, the reward module 26 is configured to calculate the first and second rewards RA, RB according to a predetermined reward rule.

As explained above, the predetermined reward rule defines whether the calculation performed by the first or the second processing module 14, 16 is close to a predetermined objective.

By way of example, the objective is for example a trajectory of the aircraft to be followed and the reward rule is defined by the distance between the actual trajectory followed by the aircraft and the target trajectory. Thus, the closer the distance between the two trajectories, the higher the reward in response, such that the first and second processing modules 14, 16 take into account in a privileged manner the actions leading to the closest possible trajectory. of the target trajectory.

The reward module 26 is further configured to penalize each first reward RA and/or respectively each second reward RB, if the first intermediate datum SA and/or respectively if the second intermediate datum SB does not belong to the security domain.

The reward module 26 is for example configured to multiply each first reward RA and/or respectively each second reward RB by a multiplier coefficient strictly less than 1 if the first intermediate datum SA and/or respectively if the second intermediate datum SB does not belong not to the security domain. The greater the difference between the first and/or second intermediate data SA, SB with respect to the security domain, the smaller the multiplier coefficient in order to penalize the associated first or second reward RA, RB.

As a variant, the reward module 26 is configured to subtract from each first reward RA and/or respectively from each second reward RB, a value proportional to the difference between the first or second reward RA, RB with respect to the security domain.

The first processing module 14 is configured to receive the first reward RA at each iteration, and to modify the first plurality of algorithmic parameters according to the first reward RA received at each iteration in order to maximize the next first reward RA received.

The second processing module 16 is configured to receive the second reward RB at each iteration, and to modify the second plurality of algorithmic parameters according to the second reward RB received at each iteration in order to maximize the next second reward RB received.

The control module 22 is configured to check whether the first and second intermediate data SA, SB belong to the security domain.

In particular, if the first and second intermediate data SA, SB are digital values, the control module 22 is configured to check whether the first and second intermediate data SA, SB belong to the range of values defining the security domain.

If the first and second intermediate data SA, SB are Boolean indicators, the control module 22 is configured to check whether the first and second intermediate data SA, SB are equal to the value associated with the security domain, typically 0 or 1.

The control module 22 is also configured to compare the first and second intermediate data SA, SB with each other in order to detect any inconsistency.

The comparison between the two intermediate data SA, SB determines whether these two intermediate data are mutually consistent or not according to a predetermined consistency rule. The consistency rule defines a criterion on the result of the comparison making it possible to define whether the intermediate data is consistent with each other.

Typically, if the intermediate data are numeric values, the consistency rule defines a calculation of a metric, such as a distance between the data, for example the absolute value of the difference, and a consistency threshold. The intermediate data SA, SB are then consistent with one another if the metric, such as the distance between the values, is less than the consistency threshold.

By way of example, two intermediate data representing aircraft speed instructions having a difference in value of more than 10% are considered to be inconsistent with each other.

The control module 22 is further configured to count the number of times respectively where the first and second intermediate data SA, SB do not comply with the security domain.

The control module 22 is configured to reset respectively the first plurality of algorithmic parameters and the second plurality of algorithmic parameters when said number is greater than a predetermined threshold.

In particular, the reinforcement learning algorithms implemented in the first and second processing modules 14, 16 are trained in a controlled environment of the simulation type, before the operation in real conditions of the system 10. This training allows the establishment of a behavior of the algorithms ensuring operation of the algorithms ensuring performance and security deemed sufficient by a person skilled in the art before the operation of the system 10. The algorithmic parameters of the algorithm obtained after the training are saved in a database of data embedded in the system 10.

Thus, the term "resetting the first plurality of algorithmic parameters and the second plurality of algorithmic parameters, the fact of resetting the reinforcement learning algorithms implemented in the first and second processing modules 14, 16 according to the algorithmic parameters defined during training before operating the system 10.

Advantageously, when the auxiliary processing module 18 is configured to supply two intermediate reference data associated from respectively the first input data A and the second input data B for the implementation of the critical function, the module 22 is configured to compare the first intermediate datum SA with the intermediate reference datum SR associated with the first input datum A, and respectively to compare the second intermediate datum SB with the intermediate reference datum SR' associated with the second input data B.

Thus, in the event of an inconsistency detected between the first and second intermediate data SA, SB, the control module 22 is able to detect whether the failure is due to the input data A, B or else to the implementation of the critical function by the first and second processing modules 14, 16.

Indeed, if the first intermediate datum SA and the intermediate reference datum SR associated with the first input datum A are coherent and if the second intermediate datum SB is not coherent with the intermediate reference datum associated with the second datum input B, then the failure comes from the second processing module 16.

If the first intermediate datum SA and the intermediate reference datum SR associated with the first input datum A are consistent and if the second intermediate datum SB is consistent with the intermediate reference datum associated with the second input datum B, then the failure comes from one of the two upstream electronic devices 12A, 12B.

The control module 22 is, moreover, configured to compare the first and second rewards RA, RB with each other in order to detect any inconsistency.

In particular, the control module 22 calculates a metric, such as a distance between the rewards RA, RB, for example the absolute value of the difference, and a second consistency threshold. The rewards RA, RB are then consistent with one another if the metric, such as the distance between the values, is less than the second consistency threshold.

The generation module 24 is configured to generate the output data C according to the first and second intermediate data SA, SB when the first and second intermediate data SA, SB belong to the security domain and in the absence of detected inconsistency.

The output data C is for example equal to one of the two intermediate data SA, SB. Typically, when the first and second intermediate data SA, SB belong to the security domain and in the absence of detected inconsistency, the output data C is equal to the first intermediate data SA.

As a variant, when the first and second intermediate data SA, SB belong to the security domain and in the absence of detected inconsistency, the output data C is equal to the intermediate data SA, SB associated with the reward RA, RB la higher calculated by the reward module 26.

As a further variant, the generation module 24 is configured to apply a mathematical formula to the first and second intermediate data SA, SB to obtain the output data C. For example, the generation module 24 is configured to perform an average of the intermediate data SA, SB.

If one of the first and second intermediate data SA, SB does not belong to the security domain and/or if an inconsistency is detected, the output data C is then equal to the reference data SR.

The alert module 21 is configured to generate an alert signal according to each inconsistency detected by the control module 22 between the intermediate data SA, SB and/or between the rewards RA, RB.

The alert signal advantageously includes an alert relating to the failure of the upstream electronic devices 12A, 12B or of one of the processing modules 14, 16.

Optionally, the display module 23 is configured to display the alert intended for a user of the electronic system 10. In particular, when the system 10 is an avionic system, the display module 23 is configured to display the alert on a head-down display screen or a head-up display screen in front of the pilot who then takes into account the detected failure.

In the example of the , the electronic system 10 comprises an information processing unit 50 formed for example of a memory 52 and a processor 54 associated with the memory 52. The first processing module 14, the second processing module 16, the module auxiliary processing unit 18 and the monitoring device 20, and as an optional addition the alert module 21 and the display module 23, are each produced in the form of software, or a software brick, executable by the processor 54. The memory 52 is then able to store a first processing software, a second processing software, an auxiliary processing software and a monitoring software, and in addition optional alert software and display software. The processor 54 is then capable of executing each of these software programs.

In a variant not shown, the first processing module 14, the second processing module 16, the auxiliary processing module 18 and the monitoring device 20, and as an optional addition the alert module 21 and the display module 23, are each made in the form of a programmable logic component, such as an FPGA (from the English Field Programmable Gate Array ), or even in the form of a dedicated integrated circuit, such as an ASIC (from the English Application Specific Integrated Circuit ). The first processing module 14, the second processing module 16, the auxiliary processing module 18 and the monitoring device 20 are then preferably each embedded in a respective electronic computer.

When the electronic system 10 is produced in the form of one or more software, that is to say in the form of a computer program, it is also capable of being recorded on a medium, not shown, readable by computer. The computer-readable medium is, for example, a medium capable of storing electronic instructions and of being coupled to a bus of a computer system. By way of example, the readable medium is an optical disc, a magneto-optical disc, a ROM memory, a RAM memory, any type of non-volatile memory (for example EPROM, EEPROM, FLASH, NVRAM), a magnetic card or an optical card. On the readable medium is then stored a computer program comprising software instructions.

The operation of the electronic system 10 according to the invention will now be explained using the representing a flowchart of the method, according to the invention, for implementing a critical function by the system 10.

Subsequently, an example of implementation of the method will be described for an avionic system, and those skilled in the art will understand that this method applies more generally to any electronic system 10.

Initially, the aircraft is, for example, flying towards an airport.

The avionics system 10 then implements a critical avionics function, such as a calculation of a flight control of the aircraft, a calculation of an optimized flight trajectory according to a predetermined criterion, such as the fuel consumption of the aircraft, or else the calculation of a strategy for avoiding an obstacle by the aircraft. During a first calculation step 100, the first processing module 14 calculates the first intermediate datum SA from the first input datum A and the reinforcement learning algorithm, for the implementation of the critical function for the first input data A.

In the example of the calculation of a flight command, the first input datum A is in particular a first datum relating to the location of the aircraft, a first datum relating to the state of an avionic system or a first datum relating to the trajectory to be followed by the aircraft. The first processing module 14 then calculates the first intermediate datum SA, such as a first flight command for the aircraft, in order to get as close as possible to a target trajectory, this from this first input datum A by applying the reinforcement learning algorithm to it. The first intermediate datum SA is thus representative of a first flight command for the aircraft in order to best follow the targeted trajectory.

In the example of the calculation of an optimized trajectory, the first input datum A is in particular a first datum relating to the location of the aircraft, a first datum relating to the state of the aircraft, a first datum relating to the scheduled destination for the aircraft. The first intermediate datum SA is then representative of a first trajectory to be followed.

In the example of the calculation of an obstacle avoidance strategy, the first input datum A is in particular a first location datum of the aircraft, a first datum of location of the obstacle or a first datum relative to the state of an avionics system. The first intermediate datum SA is then representative of a first avoidance strategy to be followed by the aircraft.

During a second calculation step 110, the second processing module 16 calculates the second intermediate datum SB from the second input datum B and the reinforcement learning algorithm, for the implementation of the critical function for the second input data B.

In the example of the calculation of a flight command, the second input datum B is in particular a second location datum of the aircraft, preferably coming from a sensor different from that which provided the first location datum, a second datum relating to the state of said avionic system or a second datum relating to the trajectory to be followed. The second processing module 16 calculates the second intermediate datum SB, such as a second flight command for the aircraft in order to get as close as possible to the target trajectory, this from this second input datum B in it applying reinforcement learning algorithm. The second intermediate datum SB is thus representative of a second flight command for the aircraft in order to best follow the targeted trajectory.

In the example of the calculation of an optimized trajectory, the second input datum B is in particular a second datum relating to the location of the aircraft, a second datum relating to the state of the aircraft, a second datum relating to the scheduled destination for the aircraft. The second intermediate datum SB is then representative of a second trajectory to be followed.

In the example of calculating an obstacle avoidance strategy, the second input datum B is in particular a second location datum of the aircraft, a second location datum of the obstacle or a second datum relative to the state of an avionics system. The second intermediate datum SB is then representative of a second avoidance strategy to be followed by the aircraft.

During a third calculation step 120, the auxiliary processing module 18 calculates the intermediate reference datum SR from the first input datum A and/or from the second input datum B and from the algorithm of reference, for the implementation of the critical function via the reference algorithm for the first input data A and/or the second input data B.

For example, the auxiliary processing module 18 calculates the intermediate reference datum SR, such as a third flight command for the aircraft in order to approach as close as possible to the targeted trajectory, a third trajectory to be followed or a third strategy avoidance, this from the first input datum A and/or from the second input datum B by itself or by applying the reference algorithm to them.

In the example of the calculation of a flight command, the reference algorithm is for example a simplified physical model providing a deterministic formula giving the flight command according to the difference between the trajectory of the aircraft and the trajectory aimed.

In the example of the calculation of an optimized trajectory, the reference algorithm is for example a simplified physical model providing a deterministic formula giving a trajectory with low fuel consumption as a function of the position of the aircraft and of its destination.

In the example of the calculation of an obstacle avoidance strategy, the reference algorithm is for example a simplified physical model providing a deterministic formula giving an avoidance strategy as a function of the position of the aircraft and the position of the obstacle.

Calculation steps 100, 110 and 120 are carried out independently of one another, at the same time or else successively in any order.

The monitoring device 20 receives the first and second intermediate data SA, SB then verifies, during a control step 130, whether the first and second intermediate data SA, SB belong to the security domain and compares the first and second intermediate data SA, SB between them in order to detect a possible inconsistency.

By way of example, if the flight command is a speed instruction, the safety domain is then in particular an associated speed interval. If the flight command is an altitude instruction, the safety domain is then an altitude interval. If the flight command is an engine command, the safety domain is then in particular an interval of engine power. If the flight command is a direction command, the safety domain is for example a change in direction limit of the aircraft, etc.

In addition, two flight commands are for example detected as being inconsistent with each other, if the trajectories associated with these two commands have a spatial difference greater than a predetermined threshold distance.

When the first and second intermediate data SA, SB belong to the security domain and in the absence of detected inconsistency, the monitoring device 20 delivers the output data C according to the first and second intermediate data SA, SB during a generation stage 140.

For example, the output data C is equal to the first flight command or is equal to the average between the first and second flight commands.

If an inconsistency is detected, the monitoring device 20 then delivers the output datum C equal to the reference datum SR during the generation step 140. In addition, the alert module 21 generates an alert signal during of an alert step 150 and, optionally, during a display step 160, the display module 25 displays the alert intended for a user of the electronic system 10, in particular the pilot of the aircraft .

During a reward step 170, the reward module 26 receives the first and second intermediate data SA, SB and calculates a first reward RA associated with the implementation of the critical function by the first processing module 14 and a second reward RB associated with the implementation of the critical function by the second processing module 16.

In the example of calculating a flight command, the closer the distance between the calculated trajectory and the target trajectory, the higher the reward.

In the example of the calculation of an optimized trajectory, the reward is calculated according to the actual measured fuel consumption. The lower the consumption, the higher the reward.

In the example of calculating an obstacle avoidance strategy, the reward is calculated after the avoidance of the obstacle according to the minimum difference in distance between the aircraft and the obstacle. The higher this distance gap, the higher the reward is also.

The reward module 26 sends the first reward RA and respectively the second reward RB to the first and respectively second processing module 14, 16.

At the next iteration of the first calculation step 100, the first processing module 14 receives the first reward RA and modifies the first plurality of algorithmic parameters in order to maximize the next first reward RA received, and thus for example calculate a closer trajectory of the target trajectory.

At the next iteration of the second calculation step 110, the second processing module 16 receives the second reward RB and modifies the second plurality of algorithmic parameters in order to maximize the next second reward RB received, and thus for example calculate a trajectory consuming less fuel or a more effective avoidance strategy.

It can then be seen that the present invention has a certain number of advantages.

Indeed, the implementation of the critical function by the first and second processing modules 14, 16 via the reinforcement learning algorithm makes it possible to improve the performance of the system 10 during its operation. Indeed, such an algorithm makes it possible to adapt to new and unexpected events encountered by the electronic system 10 during its operation.

In addition, the comparison between these two intermediate data SA, SB resulting from two different input data and the verification of the belonging to the security domain makes it possible to detect a possible inconsistency and to provide, if necessary, an output data C equal to the intermediate reference datum SR belonging to the security domain, and thus to guarantee the security of the system 10 even in the event of a detected inconsistency.

Finally, the system 10 according to the invention makes it possible to alert the user, if necessary, of a failure due to the input data or else to the implementation of reinforcement learning algorithms.

Thus, the invention makes it possible to obtain a high-performance electronic system 10 while being sufficiently safe, in particular with regard to safety requirements in the avionics field.

Claims (10)

Electronic system (10) configured to implement a critical function, the electronic system (10) being able to receive first and second input data (A, B) and to deliver output data (C) associated with said critical function, the output data (C) being associated with a predetermined security domain, the security domain being in the form of a range of predefined values,
the electronic system (10) comprising:
- a first processing module (14) configured to calculate a first intermediate datum (SA) from the first input datum (A) and a reinforcement learning algorithm, for the implementation of the function critical for the first input data (A),
- a second processing module (16) configured to calculate a second intermediate datum (SB) from the second input datum (B) and the reinforcement learning algorithm, for the implementation of the function critical for the second input datum (B), the second input datum (B) being different from the first input datum (A),
- an auxiliary processing module (18) configured to calculate an intermediate reference datum (SR) from the first input datum (A) and/or from the second input datum (B) and from an algorithm reference different from the reinforcement learning algorithm, for the implementation of the critical function for the first input datum (A) and/or the second input datum (B) via the reference algorithm ,
- a monitoring device (20) comprising:
+ a control module (22) configured to check whether the first and second intermediate data (SA, SB) belong to the security domain and to compare the first and second intermediate data (SA, SB) with each other in order to detect a possible inconsistency ;
+ a generation module (24) configured to generate the output data (C) according to the first and second intermediate data (SA, SB) if the first and second intermediate data (SA, SB) belong to the security domain and if no inconsistency is detected, the output datum (C) being equal to the intermediate reference datum (SR) otherwise. A system (10) according to claim 1, wherein the set of implementations of the critical function by the reference algorithm defines an arrival domain of the reference algorithm, the arrival domain being included in the security domain. A system (10) according to claim 1 or 2, wherein the monitoring device (20) includes a reward module (26) configured to calculate a first reward (RA) for the performance of the critical function by the first module (14) and a second reward (RB) for the implementation of the critical function by the second processing module (16),
the first processing module (14) being configured to implement the reinforcement learning algorithm based on a first plurality of algorithmic parameters, the first processing mold (14) being configured, at each iteration, to receive the first reward (RA) and to modify the first plurality of algorithmic parameters according to the received first reward (RA) in order to maximize the next received first reward (RA),
the second processing module (14) being configured to implement the reinforcement learning algorithm according to a second plurality of algorithmic parameters, the second processing module (16) being configured, at each iteration, to receive the second reward (RB) and to modify the second plurality of algorithmic parameters according to the received second reward (RB) in order to maximize the next received second reward (RB). System (10) according to claim 3, in which the reward module (26) is configured to penalize each first reward (RA) and/or respectively each second reward (RB) when the first intermediate datum (SA) and/or respectively the second intermediate datum (SB) does not belong to the security domain. System (10) according to Claims 3 or 4, in which, when the first and second intermediate data (SA, SB) belong to the security domain and in the absence of detected inconsistency, the generation module (24) is configured to deliver, as output datum, the intermediate datum associated with the highest reward (RA, RB) calculated by the reward module (26). System (10) according to any one of the preceding claims, in which the control module (22) is further configured to compare the first and second rewards (RA, RB) with each other in order to detect any inconsistency,
and wherein the electronic system (10) further comprises an alert module (21) configured to generate an alert signal as a function of each inconsistency between the intermediate data (SA, SB) and/or between the rewards (RA , RB). System (10) according to any one of claims 3 to 5, in which the control module (22) is further configured to count the number of times the first and second intermediate data (SA, SB) respectively do not respect the security domain and to reset respectively the first plurality of algorithmic parameters and the second plurality of algorithmic parameters when said number is greater than a predetermined threshold. System (10) according to any one of the preceding claims, in which the system (10) is an avionic system suitable for being onboard in a control station of an aircraft, in particular a navigation system of the aircraft. Method for implementing a critical function by an electronic system (10), the electronic system (10) being configured to implement a critical function, the electronic system (10) being capable of receiving first and second data from input (A,B) and delivering an output datum (C) associated with said critical function, the output datum (C) being associated with a predetermined security domain, the security domain being in the form of a range predefined value(s),
the method comprising the following steps:
- calculation (100) of a first intermediate datum (SA) from the first input datum (A) and of a reinforcement learning algorithm, for the implementation of the critical function for the first datum input (A),
- calculation (110) of a second intermediate datum (SB) from the second input datum (B) and the reinforcement learning algorithm, for the implementation of the critical function for the second datum input (B), the second input data (B) being different from the first input data (A),
- calculation (120) of an intermediate reference datum (SR) from the first input datum (A) and/or from the second input datum (B) and from a reference algorithm different from the reinforcement learning algorithm, for the implementation of the critical function for the first input datum (A) and/or the second input datum (B) via the reference algorithm,
- verification (130) whether the first and second intermediate data (SA, SB) belong to the security domain and comparison of the first and second intermediate data (SA, SB) with each other in order to detect any inconsistency;
- generation (140) of the output data (C) as a function of the first and second intermediate data (SA, SB) if the first and second intermediate data (SA, SB) belong to the security domain and if no inconsistency is detected, the output datum (C) being equal to the reference datum (SR) otherwise. Computer program comprising software instructions which, when executed by a computer, implement a method according to the preceding claim.