FR3144113A1 - Method for determining a set of optimal maintenance thresholds for a plurality of mechanical parts - Google Patents

Abstract

Method for determining a set of optimal maintenance thresholds for a plurality of mechanical parts One aspect of the invention relates to a method for determining a set of optimal maintenance thresholds for a plurality of mechanical parts, each mechanical part of the plurality of mechanical parts being divided into a plurality of modules, each module being associated with a type of module, each type of module being associated with at least one maintenance threshold below which a maintenance operation must be carried out on a module of said type of module, the method being implemented using a model of the mechanical part and a range of values for each maintenance threshold, the determination method comprising for a predefined duration: For each set maintenance thresholds of a plurality of sets of maintenance thresholds compatible with the range associated with each threshold, a step of simulating the number of mechanical parts available and the number of maintenance operations carried out over the entire predefined duration so as to obtain a plurality of simulations, each simulation being associated with a set of maintenance thresholds; When, for at least one simulation carried out during the previous step, the number of available parts is above a first target value and/or the number of maintenance operations is below a second target value over the entire predetermined duration, a step of selecting the set of thresholds associated with the simulation considered as the optimal set of maintenance thresholds; When no set of thresholds makes it possible to obtain a number of available mechanical parts above the first target value or a number of maintenance operations below the second target value throughout the period considered, a determination step of a new plurality of sets of thresholds compatible with the range associated with each threshold, the new plurality of sets of thresholds being at least partly different from the plurality of sets of thresholds already used, the previous steps being repeated with this new plurality sets of thresholds.

Description

Method for determining a set of optimal maintenance thresholds for a plurality of mechanical parts TECHNICAL FIELD OF THE INVENTION

The technical field of the invention is that of the maintenance of a plurality of mechanical parts.

The present invention relates to a method for determining an optimal set of maintenance thresholds.

TECHNOLOGICAL BACKGROUND OF THE INVENTION

Managing a plurality of mechanical parts, for example a fleet of turbomachines, can be relatively complex when one wishes to ensure a target number of mechanical parts available. Beyond the logistical aspects inherent to the management itself, technical aspects come into play, such as the threshold for triggering maintenance operations on all or part of the mechanical parts in question.

In the state of the art, the management of this level of availability is done through several means independent of each other. The target number of mechanical parts available can, for example, be evaluated by taking into account, theoretically, maintenance plans possibly supplemented by feedback. This number can also be evaluated using models that take into account part of the life cycle of the parts and modules that compose them. These models generally allow isolated choices to be made regarding maintenance rules. For example, a model according to the state of the art may be interested in the stocks of spare modules without however taking into account the operating rules of the workshops in charge of maintenance operations on the mechanical parts considered (in particular, rules of grouping of parts disassembly and reassembly rules). In another example, a model according to the state of the art could take into account the number of dismantlings of the mechanical parts considered (for example the number of removals for a turbomachine) once again without taking into account the operating rules of the workshops . In yet another example, a state-of-the-art model will be able to calculate the need for spare parts without taking into account the random nature of breakdowns or even the number of parts available.

In other words, the solutions of the state of the art do not take all the parameters in their entirety and therefore do not make it possible to determine precisely the value of the maintenance thresholds that should be chosen to obtain a number target available mechanical parts while minimizing the maintenance burden.

The invention offers a solution to the problems mentioned above, by making it possible to determine a set of maintenance thresholds making it possible to ensure a determined number of mechanical parts available while limiting unnecessary maintenance operations.

• Pour chaque jeu de seuils de maintenance d’une pluralité de jeux de seuils de maintenance compatible avec la plage associée à chaque seuil, une étape de simulation, à l’aide du modèle de la pièce mécanique, de la pluralité de pièces mécanique sur la durée prédéfinie de sorte à déterminer le nombre de pièces mécaniques disponibles et le nombre d’opérations de maintenance effectuées sur la durée prédéfinie, chaque simulation étant associée au jeu de seuils de maintenance correspondant ;
• Lorsque, pour au moins une simulation effectuée lors de l’étape précédente, le nombre de pièces disponibles est au-dessus d’une première valeur cible et/ou que le nombre d’opérations de maintenance est en dessous d’une deuxième valeur cible sur toute la durée prédéterminée, une étape de sélection du jeu de seuils associé à la simulation considérée comme jeu de seuils de maintenance optimal ;
• Lorsqu’aucun jeu de seuils ne permet d’obtenir un nombre de pièces mécaniques disponibles au-dessus de la première valeur cible et/ou un nombre d’opérations de maintenance en dessous de la deuxième valeur cible durant toute la durée prédéfinie, une étape de détermination d’une nouvelle pluralité de jeux de seuils compatible avec la plage associée à chaque seuil, la nouvelle pluralité de jeu de seuils étant au moins en partie différente de la pluralité de jeux de seuils déjà utilisée, les étapes précédentes étant répétées avec cette nouvelle pluralité de jeux de seuils.

A first aspect of the invention relates to a method for determining a set of optimal maintenance thresholds for a plurality of mechanical parts, each mechanical part of the plurality of mechanical parts being divided into a plurality of modules, each module being associated to a type of module, each type of module being associated with at least one maintenance threshold below which a maintenance operation must be carried out on a module of said type of module, the method being implemented using a model of the mechanical part and a range of values for each maintenance threshold, the model of the mechanical part taking into account the aging of the mechanical part and the maintenance cycle of the mechanical part, the determination method comprising, for a predefined duration:

• For each set of maintenance thresholds of a plurality of sets of maintenance thresholds compatible with the range associated with each threshold, a step of simulation, using the model of the mechanical part, of the plurality of mechanical parts on the predefined duration so as to determine the number of mechanical parts available and the number of maintenance operations carried out over the predefined duration, each simulation being associated with the corresponding set of maintenance thresholds;
• When, for at least one simulation carried out during the previous step, the number of parts available is above a first target value and/or the number of maintenance operations is below a second target value over the entire predetermined duration, a step of selecting the set of thresholds associated with the simulation considered as the optimal set of maintenance thresholds;
• When no set of thresholds makes it possible to obtain a number of mechanical parts available above the first target value and/or a number of maintenance operations below the second target value throughout the predefined duration, a step determining a new plurality of sets of thresholds compatible with the range associated with each threshold, the new plurality of sets of thresholds being at least partly different from the plurality of sets of thresholds already used, the previous steps being repeated with this new plurality of sets of thresholds.

Thanks to the invention, it is possible to minimize maintenance operations carried out on mechanical parts while guaranteeing a target level of availability of said parts.

In addition to the characteristics which have just been mentioned in the previous paragraph, the process according to the first aspect of the invention may present one or more complementary characteristics among the following, considered individually or in all technically possible combinations.

• un seuil de régénération représentant un seuil au-delà duquel une régénération du module doit être effectuée ; et/ou
• un seuil de visite avancé représentant un seuil au-delà duquel une visite préventive du module doit être effectuée ; et/ou
• un seuil d’alignement représentant un seuil au-delà duquel un alignement des potentiels du module doit être effectué.

In one embodiment, each set of thresholds comprises:

• a regeneration threshold representing a threshold beyond which a regeneration of the module must be carried out; and or
• an advanced visit threshold representing a threshold beyond which a preventive visit of the module must be carried out; and or
• an alignment threshold representing a threshold beyond which an alignment of the module potentials must be carried out.

In one embodiment, the method implements an evolutionary optimization algorithm, the plurality of sets of initial maintenance thresholds being generated randomly, and the step of determining a new plurality of sets of thresholds being carried out by " reproduction”, “crossing” and/or “replacement” of the sets of maintenance thresholds.

In an alternative embodiment, the method implements a reinforcement learning algorithm.

• Chaque instant correspond à une période de la durée prédéfinie de sorte que où est le nombre de périodes de la pluralité de périodes ;
• Un état du système correspond à l’état de la pluralité de pièces mécaniques à l’instant défini par un vecteur avec le nombre de pièce mécanique disponibles, la première valeur cible et le nombre d’opérations de maintenance pour la période considérée ;
• une action est la modification des seuils de maintenance dans la limite des plages fixés ;
• une valeur représentant la récompense à la période , cette valeur étant négative lorsque la première valeur cible n’est pas atteinte à la fin de la période considérée et positive lorsque la première valeur cible est atteinte à la fin de la période considérée.

In one embodiment, the predefined duration is divided into a plurality of periods, and the learning algorithm is implemented using a Markovian decision process defined as follows:

• Every moment corresponds to a period of the predefined duration so that Or is the number of periods of the plurality of periods;
• A state of the system corresponds to the state of the plurality of mechanical parts at the moment defined by a vector with the number of mechanical parts available, the first target value and the number of maintenance operations for the period considered ;
• an action is the modification of the maintenance thresholds within the limits of the fixed ranges;
• a value representing the reward at the period , this value being negative when the first target value is not reached at the end of the period considered and positive when the first target value is reached at the end of the period considered.

In addition, the new plurality of sets of thresholds is determined as a function of the reward(s) obtained during the previous simulations.

In one embodiment, the value of the reward is given by the following relation:

Or is the maximum number of maintenance operations.

In one embodiment, the model of the mechanical part is modeled using an object language, a first object class “Park” being associated with the plurality of mechanical parts, a second object class “Part » being associated with each mechanical part and a third object class “Module” being associated with each module of each part, a “Part” object comprising an object of class “Module” for each of the modules associated with it.

A second aspect of the invention relates to a data processing device comprising means for implementing the method according to the first aspect of the invention.

A third aspect of the invention relates to a computer program comprising instructions which, when the program is executed by a computer, lead it to implement a method according to the first aspect of the invention.

A fourth aspect of the invention relates to a computer-readable data carrier on which the computer program according to the third aspect of the invention is recorded.

The invention and its various applications will be better understood on reading the following description and examining the accompanying figures.

BRIEF DESCRIPTION OF THE FIGURES

The figures are presented for information purposes only and in no way limit the invention.

There shows a representation of a mechanical part comprising twelve modules.

There shows a maintenance flow in the form of a plurality of states, each transition from one state to another state being associated with a transition.

There shows a representation of the state of each module in the form of a table, this representation being of type DataFrame.

There shows an object representation of the maintenance cycle.

THE , And show the performance gain associated with the use of an object-oriented approach in the simulation of the maintenance cycle.

THE And show an implementation of a method according to the invention using an evolutionary algorithm.

THE And show the evolution of the number of mechanical parts available for two agents trained on a small number of iterations on the and on a significant number of iterations on the .

THE has show the evolution of the number of mechanical parts available depending on the expression of the reward used.

DETAILED DESCRIPTION

The figures are presented for information purposes only and in no way limit the invention. In the remainder of the description, the invention will be illustrated using an example in which the mechanical part is an engine, for example an aircraft turbomachine. However, this is only an example and the invention can be implemented for any type of mechanical part.

A first aspect of the invention relates to a method for determining the maintenance thresholds of a plurality of mechanical parts PM, subsequently a plurality of aircraft engines, each mechanical part PM of the plurality of mechanical parts being divided into a plurality of modules, each module being associated with a type of module (thus a turbomachine comprises blades, each blade can be assimilated to a module of the same type – of course, the fineness of cutting depends on the desired degree of precision of the model and of the desired objective). The mechanical parts of the plurality of mechanical parts are similar parts (they are composed of a majority of identical modules) or identical parts (all of the modules which compose them are identical).

Furthermore, each type of module is associated with at least one maintenance threshold below which a maintenance operation must be carried out on a module of the module type considered. This threshold is identical for all modules of the same type regardless of the mechanical part PM in which said module is present.

The method according to the invention can be implemented by a calculation means (for example a processor or an ASIC card) coupled to a memory (for example a hard disk and/or a RAM type random access memory), said memory being configured to store the data and instructions necessary for the implementation of the method according to the invention by the calculation means.

In one embodiment, the method according to the invention takes into account the regeneration threshold (called the operating limit potential tail in the field and abbreviated by QDP-LF) and/or the advanced visit threshold (called the operating limit tail). normal visit potential in the domain and abbreviated by QDP-VN) and/or the alignment threshold (called potential alignment in the domain). Of course, the method according to the invention may take into account other maintenance thresholds. In order to implement the method according to the invention, a range of values is associated with each threshold, the determination method then determining a value included in the range in question for each threshold considered.

In the field of aeronautics, maintenance thresholds may notably concern the number of flight hours, the number of takeoff/landing cycles or even be linked to a level of wear measured on the module in question. Generally speaking, these thresholds are examined when the mechanical part PM suffers a breakdown, for example because one (or more) of the modules has failed. The faulty module(s) are repaired or replaced and the remaining modules are examined if maintenance thresholds have been crossed (in the field, we speak of checking the potential, for example the flight potential in aeronautics). For example, it is possible to regenerate the potential of a module preventively when the regeneration threshold is reached or crossed or even to carry out a preventive visit when the advanced visit threshold is reached or crossed. In other words, when a maintenance threshold is reached or crossed (based on an objective quantity linked to the operation of the module in question), one or more measurements or inspections are carried out (level of wear, presence of cracks, etc. .) in order to ensure the proper functioning of the module in question.

By modifying these thresholds, it is possible to influence the availability rate of the mechanical parts PM of the plurality of mechanical parts PM. Indeed, thresholds that are too low can lead to an increase in the risk of breakdown and therefore a reduction in the availability of mechanical parts. In the same way, thresholds that are too high can lead to unnecessary maintenance operations with all the disadvantages that this entails. The invention makes it possible to determine a set of maintenance thresholds making it possible to avoid these two pitfalls. Thus, thanks to the invention, it is possible to minimize the maintenance operations carried out on the PM mechanical parts while guaranteeing a target level of availability of said PM mechanical parts.

For this, the method according to the invention firstly comprises, for each set of maintenance thresholds of a plurality of sets of maintenance thresholds compatible with the range associated with each threshold, a step of simulating the number of mechanical parts available and the number of maintenance operations carried out so as to obtain a plurality of simulations, each simulation being associated with a set of maintenance thresholds. Thus, during this step, for each of the sets of maintenance thresholds, a simulation is carried out in order to determine in particular the number of mechanical parts available and the number of maintenance operations carried out over the period considered. Of course, other indicators can also be taken into account during these simulations. The results thus obtained can then be compared to a target objective in terms of number of mechanical parts available and/or number of maintenance operations carried out.

The method according to the invention also comprises when, for at least one simulation carried out during the previous step, the number of mechanical parts available is above a first target value and/or the number of maintenance operations is below a second target value over the entire predefined duration, a step of selecting the set of thresholds associated with the simulation considered as the set of optimal maintenance thresholds. If several simulations meet these criteria, then the simulation allowing the results closest to the target values is retained. In one embodiment, the predefined duration is divided into periods, each period is associated with a first threshold value and/or a second threshold value which may be different for each period, and compliance with the set threshold values is examined period by period .

The method also comprises, when no set of maintenance thresholds makes it possible to obtain a number of mechanical parts available above the first target value and/or a number of maintenance operations below the second target value during the entire predefined duration, a step of determining a new plurality of sets of maintenance thresholds compatible with the range associated with each threshold, at least partly different from the plurality of sets of thresholds already used, the previous steps being repeated with this new plurality of sets of thresholds. Thus, the steps of the process are repeated until a set of thresholds is obtained making it possible to obtain the desired target value(s).

Mechanical part model

For this, the method according to the invention is implemented using a model of the mechanical part PM taking into account in particular the aging and failures of the mechanical part PM as well as the maintenance cycle of this mechanical part PM.

• les chaînes de montage et démontage, notamment les temps moyens nécessaire à chaque opération de maintenance associée à chaque module ;
• la fourniture de nouveaux modules ou la mise hors service de modules ne pouvant pas être réparés ou arrivés en fin de vie ;
• les conditions d’utilisation des pièces mécaniques de la pluralité de pièces mécaniques et donc des modules composant lesdites pièces mécaniques ;
• les événements aléatoires pouvant survenir, par exemple une panne au niveau d’un module ou de plusieurs modules ou la découverte d’un défaut sur une pièce lors du contrôle d’un module.

In one embodiment, the model in question takes into account in particular:

• the assembly and disassembly lines, in particular the average times necessary for each maintenance operation associated with each module;
• the supply of new modules or the decommissioning of modules that cannot be repaired or have reached the end of their life;
• the conditions of use of the mechanical parts of the plurality of mechanical parts and therefore of the modules composing said mechanical parts;
• random events that may occur, for example a failure of a module or several modules or the discovery of a defect on a part during the inspection of a module.

These different aspects are materialized through maintenance parameters which are fixed during the implementation of the process according to the invention and which define the model used during this implementation. Thus, the model in question makes it possible to account for the aging of the mechanical parts of the plurality of mechanical parts and the maintenance cycle (in particular by considering several levels of maintenance depending on the nature of the maintenance operation, etc.).

The model implemented in the present invention is more particularly described in has . There represents a mechanical part PM composed of twelve modules of different types and independent of each other. When a breakdown occurs on at least one of the modules, it is possible to disassemble and replace and/or repair the module in question. Thus, in the maintenance flow, each PM mechanical part goes through a succession of states and transitions at different scales: at the scale of the PM mechanical part and at the scale of each module. As illustrated in , the maintenance flow during the aging of such a mechanical part PM can be represented using a state diagram representing the states (the circles) that the motor or the different modules can take during the maintenance cycle as well as the transitions (the arrows – in the figure, the transitions are numbered from T1 to T10) possible between these different states. More particularly, this diagram can be seen as a directed graph (in the sense of graph theory). Each vertex indicates the location of the PM mechanical part or module in the maintenance chain: this is its state. Each arc (or arrow) indicates a transition from one state to another. On the , dotted transitions represent transitions at the scale of the PM mechanical part, while solid line transitions represent transitions at the module scale.

It is thus possible to follow the maintenance flow of a PM mechanical part, for example an aircraft engine. The aircraft engine is initially in the aircraft state (i.e. in working order) or “aircraft” state represented by the “A/C” circle in the diagram. Then, this motor suffered a breakdown due to an incident on module 7: this module must therefore be removed from the motor. The engine will then pass through different locations (transitions T1 and T2) and then be disassembled at the Removal-Reassembly workshop (transition T3 - “DEP/REM” on the diagram). After disassembling the motor to dismantle module 7, the transitions are made at the scale of each module. In particular, module 7 will go through different workshops to be repaired (transitions T4 and T6). This repair will regenerate the flight potential (mentioned previously) of the module, the latter then being stored (“STOCK MODULES” on the diagram – transitions T5 and T7). Furthermore, a check of the remaining flight potential of all engine modules is carried out. In the chosen example, module 4 has a fairly low theft potential (it will soon reach its operating limit), and module 10 must soon undergo a technical inspection. It is possible to regenerate the flight potential of module 4 in a preventive manner. The threshold below which it is wise to carry out this regeneration defines the notion of operating limit potential tail mentioned previously. In the same way, the remaining time threshold below which it is wise to carry out the visit in advance defines the notion of a normal visit potential tail also mentioned previously. The modules will therefore be transferred to specific workshops, undergo repairs if necessary, then be put back into stock. In addition, depending on the modules present in stock, it is possible to rebuild one or more engines in the workshop (“DEP/REM” on the diagram – T8 transition). From this state, transitions are made again at the engine level. The engine thus reconstituted undergoes a series of tests on the test bench (“BANC” on the diagram – transition T9) before being flown (return to the “A/C” state – transition T10).

In order to be able to follow the position of each PM mechanical part and each module of each PM mechanical part in the maintenance cycle, it is possible to use a DataFrame type table as illustrated in . In this table, each module is represented by a line, the information provided on each line allowing each module to be associated with a PM mechanical part. When a module or motor is moved, the position of the module or modules associated with the motor in question is updated to take the movement into account. Using the previous example, if the engine with serial number “TITI” is aircraft, then all the modules attached to this engine will be in “A/C” location. Then, when the engine breaks down, the “Location” column is modified for all the modules associated with this engine which move to the “TRANSIT” location. All motor or module transitions work on this principle. In this embodiment, the simulation implemented in the method according to the invention therefore comprises a loop of transition function calls which modify the rows and columns of the tables (in the example of the , ten transition functions are necessary, but this is only an example and this number varies depending on the maintenance cycle to be simulated). For each transition function, it is therefore necessary to apply one or more conditional filters to select the lines of interest (e.g., recover all the modules of a given engine) then modify the location (among others).

• Modifier la valeur de l’attribut « localisation » de l’objet « moteur » ;
• Mettre à jour l’attribut localisation de tous les modules de l’objet « moteur » ;
• Mettre à jour une liste recensant tous les moteurs en localisation « A/C » (par exemple dans l’objet « Parc »)

In an alternative embodiment illustrated in , the maintenance cycle model is represented using object language. More particularly, the pool of mechanical parts (i.e. the plurality of mechanical parts) is represented by an object of a first class, for example the Park class. In addition, the object of the Park class includes a plurality (n on the ) of objects of the PieceMecanique class (or of the Engine class in the example of the ) which, in turn, includes a plurality (20 on the ) of objects of the Modules class. In the example of the , it is also possible to consider a plurality of Airplane objects each associated with one or more Engine objects. This architecture makes it possible to avoid using the filter functions mentioned previously. More particularly, in this embodiment, the attributes of a class can only be manipulated via methods associated with said class. This encapsulation allows for example to modify the default behavior of an instruction like “motor.location = 'A/C'” (or motor is the object and location is an attribute associated with this object – this instruction modifies the value of the attribute). By encapsulation, this instruction will pass through a method (sometimes called a “setter”) which will:

• Modify the value of the “location” attribute of the “motor” object;
• Update the location attribute of all modules of the “motor” object;
• Update a list listing all the motors in “A/C” location (for example in the “Park” object)

Thus, functions that need access to all aircraft engines in the fleet will have access to them directly via this list. The adoption of object programming (or OOP for Object Oriented Programming) has the advantage of considerably reducing the calculation time necessary for a simulation as shown in , there and the which illustrate, for the calculation time required to execute a transition with the classic Dataframe (left) and Object (right) approach, and for the And this calculation time for different transitions with the classic Dataframe approach ( ) and the object programming approach proposed by the present invention ( ). It is clear from these figures that the savings in calculation time are significant.

Evolutionary optimization algorithm

In one embodiment, the method according to the invention is carried out using an evolutionary optimization algorithm. In this embodiment, the plurality of sets of initial maintenance thresholds (i.e. used for the first simulations) is generated randomly, each set of maintenance thresholds being called an individual. Furthermore, the step of determining a new plurality of sets of maintenance thresholds is carried out by "reproduction", "crossing" and/or "replacement", these actions constituting one or more mathematical operations carried out on the individuals having obtained the results closest to the target values. The principle of the evolutionary optimization algorithm is known to those skilled in the art and it will therefore not be detailed further in this description.

There and the illustrate the implementation of a method according to the invention with an evolutionary algorithm. More particularly, the represents the evolution of the number of mechanical parts available over time for 50 different simulations (the dotted lines representing the end of the year 2022 and 2023), the gray area representing the difference at two sigmas compared to the average value over all simulations. There represents a set of maintenance thresholds for ten modules depending on the year considered.

However, although the evolutionary optimization algorithm has demonstrated its effectiveness and relevance, it is not without its drawbacks. First of all, during a simulation, it is common for the target value(s) (for example the availability target value) to be not reached. Furthermore, in the evolutionary optimization algorithm, it is necessary to carry out, for each set of maintenance thresholds, complete simulations over the entire duration considered, to select the best candidates. In addition, the duration considered is generally divided into periods (for example of a year when the duration considered is several years), each period being associated with one or more target values, and the number of parameters to be optimized is then proportional. to the number of periods. Indeed, each individual corresponds to a complete set of parameters (i.e. the set of maintenance thresholds year by year). Thus, the number of parameters can increase considerably.

Reinforcement learning algorithm

An original way to overcome the limitations stated above is to use a reinforcement learning algorithm.

As a reminder, reinforcement learning is a machine learning method inspired by human and animal learning. This type of learning is based on the concept of “operant conditioning,” which suggests that individuals can learn new behaviors by observing the consequences of their own actions. Reinforcement Learning consists of programming an artificial intelligence, the agent, which will interact within an environment. Depending on the actions performed, the state of the environment will be modified, and the agent will receive a reward (or “reward” in English – positive or negative). This agent will therefore be rewarded for good actions, and punished for incorrect behavior. By updating the knowledge it has about the environment, the agent will seek to maximize its rewards by choosing the most advantageous actions.

• ensembles finis d’états , l’état du système à l’instanttest désigné par l’état ;
• Une famille d’actions avec l’ensemble des actions possibles à l’instant quand le système est à l’état .

Formally, reinforcement learning is based on a Markovian decision process, as defined in applied mathematics by probability theory and decision theory. It is wise to recall the definition of such a process in order to illustrate the advantages of this approach compared to the evolutionary algorithm introduced previously. In a reinforcement learning algorithm, the system evolves over time under the influence of an agent. Furthermore, time is discretized into different instants from 0 to N so that the system is defined by:

• finite sets of states , the state of the system at time t is denoted by the state ;
• A family of actions with all possible actions at the moment when the system is in state .

• Une probabilité de transition de passer de l’état à l’état en ayant effectué l’action ;
• Une fonction de récompense .

and, for everything :

• A probability of transition to move from state in the state having performed the action ;
• A reward function .

The goal of the reinforcement learning algorithm is to estimate the transition probability . The more correct this estimate is, the more detailed knowledge the agent will have about the actions to perform to maximize its rewards.

• Chaque instant correspond à une période de la durée considérée ; par exemple une année lorsque la durée considérée est de plusieurs années – pour une simulation de dix ans, on a donc ;
• L’état du système correspond à l’état de la pluralité de pièces mécaniques à l’instant . Cet état est un vecteur avec le nombre de pièces mécaniques disponibles, la valeur cible du nombre de pièces mécaniques disponibles et le nombre d’opérations de maintenance pour l’année considérée ;
• L’action est la modification des seuils de maintenance de la pluralité de seuils de maintenances dans la limite des plages fixées ;
• La valeur de la récompense à l’instant sera négative si la valeur cible n’est pas atteinte à la fin de la sous-période , elle sera positive dans le cas contraire, cette valeur étant de préférence de plus en plus élevée à mesure que la valeur obtenue se rapproche de la valeur cible (afin de ne pas induire des seuils trop bas qui entraîneraient des opérations de maintenance intempestives).

In order to be able to apply such an algorithm to the method according to the invention, a Markovian decision process was developed by the inventors, the latter being defined as follows:

• Every moment corresponds to a period of the duration considered; for example a year when the duration considered is several years – for a simulation of ten years, we therefore have ;
• System status corresponds to the state of the plurality of mechanical parts at the moment . This state is a vector with the number of mechanical parts available, the target value of the number of mechanical parts available and the number of maintenance operations for the year in question ;
• The action is the modification of the maintenance thresholds of the plurality of maintenance thresholds within the limits of the fixed ranges;
• The value from the reward to the moment will be negative if the target value is not reached at the end of the sub-period , it will be positive otherwise, this value preferably being increasingly higher as the value obtained approaches the target value (in order not to induce too low thresholds which would lead to untimely maintenance operations).

As already mentioned, this algorithm is an alternative to the evolutionary optimization algorithm. It has the particular advantage of not requiring complete simulations to be carried out over the entire duration considered before determining whether the choice of maintenance thresholds is satisfactory. Indeed, learning takes place period after period (for example, year after year) according to the target value or target values for the period considered (which may be different from one period to another) which allows a significant saving in calculation time. This advantage is particularly illustrated in And . In these figures, the number of PM mechanical parts available as a function of periods is illustrated for an agent trained over 15 iterations at the (i.e. 15 repetitions of the steps of the process according to the invention) and over 2657 iterations at the (i.e. 2657 repetitions of the steps of the process according to the invention). As shown in the , the simulation does not need to be carried out for the entire duration, but can be stopped when it is clear that the target values will not be achieved. As a reminder, when the evolutionary algorithm is used, the simulation must necessarily be carried out over the entire duration, that is to say for all periods of this duration.

The performance of the reinforcement learning algorithm depends on how the reward function is defined, the choice of the learning algorithm, and the choice of hyperparameters (i.e., the parameters of the algorithm itself such as the number of iterations, convergence tolerance, etc.).

In one embodiment, the value of the reward is given by the following relation (or reward function):

Or is the maximum number of maintenance operations. As shown in the , such a reward function makes it possible to respect the target value of the number of mechanical parts available without however deviating too far from it.

More particularly, the term defined the intensity of the punishment (negative value of the reward) depending on the distance between the number of mechanical parts available and the first target value : the agent will therefore look for actions that will bring it closer to the first target value (this encourages, for example, the agent to lower at least part of the threshold values when the number of mechanical parts available is too low or, on the contrary, to increase at least part of the threshold values when the number of mechanical parts available is too high). As shown in the ,in the absence of this term, the agent will tend to reach values well above the first target value. Then, the term is always strictly positive because . If (ie if the first target value is respected), the agent will be positively rewarded (provided that the distance remains reasonable). Finally, the term will “punish” the agent because the first target value is not exceeded, the factor N making it possible to encourage the agent to always reach the first target value to reach the end of each simulation. Indeed, as soon as the agent no longer respects the first target value, the positive sum of which was accumulated in previous years will be offset (negatively) by the punishment .

Furthermore, the And show that simplistic formulations of the reward function do not achieve good performance. For example, the shows that a reward function based solely on availability (+1 if the target value is reached and -1 otherwise) leads to a number of available mechanical parts well above the target value and therefore a number of operations of too much maintenance. In the same way, the shows that a reward function based only on do not make it possible to obtain satisfactory results, the number of mechanical parts available being lower than the target value from the first year.

Preferably, the reinforcement learning algorithm used in the method according to the invention is chosen from an A2C type algorithm (from Advantages Actor Critic), an SAC type algorithm (from Soft Actor Critic). ) or a PPO (Proximal Policy Optimization) type algorithm, these algorithms proving more efficient than other algorithms (e.g. “Model-Based”) in the context of the method according to the invention. More generally, the reinforcement learning algorithm used in the method according to the invention is a “Model-Free” type reinforcement learning algorithm, that is to say that no hypothesis is made about how the agent's environment will change.

Claims (10)

Computer-implemented method for determining a set of optimal maintenance thresholds for a plurality of mechanical parts, each mechanical part (PM) of the plurality of mechanical parts being divided into a plurality of modules, each module being associated with a type of module, each type of module being associated with at least one maintenance threshold below which a maintenance operation must be carried out on a module of said type of module, the method being implemented using a model of the mechanical part (PM) and a range of values for each maintenance threshold, the model of the mechanical part (PM) taking into account the aging of the mechanical part (PM) and the maintenance cycle of the mechanical part (PM) ), the determination method comprising for a predefined duration:

• For each set of maintenance thresholds of a plurality of sets of maintenance thresholds compatible with the range associated with each threshold, a step of simulation, using the model of the mechanical part (PM), of the plurality of parts mechanical over the predefined duration so as to determine the number of mechanical parts available and the number of maintenance operations carried out over the predefined duration, each simulation being associated with the corresponding set of maintenance thresholds;
• When, for at least one simulation carried out during the previous step, the number of available parts is above a first target value and/or the number of maintenance operations is below a second target value over the entire predetermined duration, a step of selecting the set of thresholds associated with the simulation considered as the optimal set of maintenance thresholds;
• When no set of thresholds makes it possible to obtain a number of available mechanical parts above the first target value and/or a number of maintenance operations below the second target value throughout the predefined duration, a step determining a new plurality of sets of thresholds compatible with the range associated with each threshold, the new plurality of sets of thresholds being at least partly different from the plurality of sets of thresholds already used, the previous steps being repeated with this new plurality of sets of thresholds.

Method according to the preceding claim in which each set of thresholds comprises:

• a regeneration threshold representing a threshold beyond which a regeneration of the module must be carried out; and or
• an advanced visit threshold representing a threshold beyond which a preventive visit of the module must be carried out; and or
• an alignment threshold representing a threshold beyond which an alignment of the module potentials must be carried out.

Method according to one of the preceding claims in which the method implements an evolutionary optimization algorithm, the plurality of sets of initial maintenance thresholds being generated randomly, and the step of determining a new plurality of sets of thresholds being carried out by “reproduction”, “crossing” and/or “replacement” of the sets of maintenance thresholds. Method according to one of claims 1 to 2 in which the method implements a reinforcement learning algorithm. Method according to the preceding claim in which the predefined duration is divided into a plurality of periods, and the learning algorithm is implemented using a Markovian decision process defined as follows:

• Every moment corresponds to a period of the predefined duration so that Or is the number of periods of the plurality of periods;
• A state of the system corresponds to the state of the plurality of mechanical parts at the moment defined by a vector with the number of mechanical parts (PM) available, the first target value and the number of maintenance operations for the period considered ;
• an action is the modification of the maintenance thresholds within the limits of the fixed ranges;
• a value representing the reward at the period , this value being negative when the first target value is not reached at the end of the period considered and positive when the first target value is reached at the end of the period considered;

the new plurality of sets of thresholds is determined based on the reward(s) obtained during the previous simulations. Method according to the preceding claim in which the value of the reward is given by the following relation:

Or is the maximum number of maintenance operations. Method according to one of the preceding claims in which the model of the mechanical part (PM) is modeled using an object language, a first object class “Park” being associated with the plurality of mechanical parts, a second object class “Part” being associated with each mechanical part (PM) and a third object class “Module” being associated with each module of each part, a “Part” object comprising an object of class “Module” for each of the modules associated with it. Data processing device comprising means for implementing the method according to one of the preceding claims. Computer program comprising instructions which, when the program is executed by a computer, lead it to implement a method according to one of claims 1 to 7. Computer-readable data carrier on which the computer program according to claim 9 is recorded.