WO2023237205A1

WO2023237205A1 - Computer-implemented method and system for detecting anomalies during the operation of a technical device

Info

Publication number: WO2023237205A1
Application number: PCT/EP2022/065756
Authority: WO
Inventors: Daniel SCHALL
Original assignee: Siemens Ag Österreich
Priority date: 2022-06-09
Filing date: 2022-06-09
Publication date: 2023-12-14

Abstract

Computer-implemented method for processing a set of computing tasks by means of a system, comprising at least one first computing apparatus (ELC1-ELCN) and at least two second computing apparatuses (EAC1-EACN), wherein the following steps are carried out: i) determining a weight function (w11-w13, wN1-wN3) for each computing task from the set of computing tasks between the at least one first computing apparatus (ELC1-ELCN) and the at least two second computing apparatuses (EAC1-EACN), ii) determining the execution capacity with regard to communication, storage and execution properties of the at least two second computing apparatuses (EAC1-EACN), iii) sorting the previously determined weight functions (w11-w13, wN1-wN3), and determining a subset of the sorted weight functions that are within a predetermined range of values of the determined execution capacity, iv) distributing the subset of computing tasks assigned to the subset of sorted weight functions to the at least two second computing apparatuses (EAC1-EACN), executing the subset of computing tasks and deleting the associated weight function, v) increasing the sequence factor for the computing tasks which were not distributed in the preceding step on account of the execution capacity, vi) continuing with step ii) as long as there are weight functions, and otherwise terminating the method.

Description

Computer-implemented method and system for anomaly detection in the operation of a technical device

The invention relates to a computer-implemented method and a computing system for processing a set of computing tasks by a system comprising at least a first computing device and at least two second computing devices, each with a processor and a memory, the at least a first Computing device is connected to the at least two second computing devices by respective communication connections.

The invention also relates to the use of the method for anomaly detection when operating a technical device.

The invention further relates to a computer program, an electronically readable data carrier and a data carrier signal.

Typical artificial intelligence systems (“AI” for short) adapt machine learning models (“ML” for short) only very slowly to new data or environmental changes. The typical ML lifecycle includes three phases: i) data collection and data preparation, ii) model building, and iii) model deployment and inference.

Inferring data from (hypothetical) models is called statistical inference.

However, ML models often cannot adapt quickly to live data or new input data.

Furthermore, not all data in a cloud edge context is available in the cloud. Sending all data to the cloud bypasses the advantage of edge computing.

In a client-server system, a computing device with a processor and memory is located at the edge. Additionally, edge devices often do not have sufficient capabilities to perform full training of ML models.

For example, in modern state-of-the-art cloud edge environments, training and inference are only performed in the cloud. This has the advantage that all data is processed and stored in one place. Clients access this data from a central location. However, real-time interaction with a process that is running on a client is not possible.

Alternatively, in the prior art, training is carried out in the cloud and inference is carried out at the edge. This results in high-performance training in the cloud and real-time process interaction at the edge. However, data is processed or stored scattered across the cloud and the edge, which is disadvantageous because updating models is typically only possible in the cloud.

However, if an ML method is to be executed on a distributed system, additional complexity may arise, namely in that an initial model for anomaly detection must be trained in the cloud and then distributed to various edge devices .

It is often difficult to carry out a suitable distribution of computing tasks, such as ML model updates through retraining, to multiple high-performance computing devices and to achieve an advantageous utilization of the entire computing system at the edge.

In the context of machine learning, an initial model is often created in the cloud, trained and deployed to distributed clients.

A calculation at the edge may be preferred for data protection reasons. An application of this ML model is therefore advantageously carried out at the edge, whereby ML model updates from the operation or application of the ML model are also carried out at the edge for the same reasons, but this requires a very high computing capacity For example, with a highly parallelized computing core such as a GPU (graphic processing unit).

The pure application of an ML model can mostly be carried out on computing devices with simpler complexity and capacity. This also includes, for example, data collection by sensors such as cameras or by a PLC (programmable logic controller) or other sensor means.

It is therefore the object of the invention to provide a solution by means of which an advantageous distribution of computing tasks can be carried out across multiple computing devices.

The object according to the invention is solved by a method of the type mentioned at the beginning, the following steps being carried out: i) determining a weight function for each computing task from the set of computing tasks between the at least one first computing device and the at least two second computing devices, whereby the Weight function has:

• a priority of the calculation task,

• a communication parameter regarding the communication connection,

• a ranking factor for processing the calculation problem, preferably starting at zero,

• a memory parameter for the memory required when executing the calculation task, • an execution parameter for the required computing requirements when executing the computing task, ii) determining the execution capacity with regard to communication, storage and execution properties of the at least two second computing devices, iii) sorting the previously determined weight functions, and Determining a subset of the sorted weight functions, which lie within a predetermined value range of the determined execution capacity, iv) distributing the subset of the computing tasks, which is assigned to the subset of the sorted weight functions, to the at least two second computing devices, executing the subset of computing tasks and Deleting the associated weight function, v) increasing the ordering factor for the calculation tasks that were not distributed in the previous step due to the execution capacity, vi) As long as weight functions are available, continue with step ii), otherwise terminate.

This ensures that the computing tasks can be processed in a particularly efficient manner in the computing system and an edge device, which includes the computing devices, can be designed to be optimally adapted for the corresponding use.

In a further development of the invention it is provided that the weight function is determined in each case by the priority, the communication parameter and the ranking factor each being in a value range between 0 and 1 and the sum of which forms a calculation parameter, and the storage parameters and the execution parameters each lie in a value range between 0 and 1 and the sum of which forms a calculation parameter, and the difference between calculation parameters Parameters and calculation parameters form the weight function.

This ensures that the weight function can be determined in a simple manner and which particularly representative parameters are used to solve the problem according to the invention.

In a further development of the invention it is provided that after step i) it is checked whether the respective weight function is equal to zero, and if so, the associated calculation task is divided into sub-arithmetic tasks and the sub-arithmetic tasks are added to the set of calculation tasks and the current calculation task is removed from the set of calculation tasks and the method is continued with step i).

This ensures that even large computing tasks or their associated weight functions - for example with high computing requirements or high memory requirements - can be processed efficiently using the method and at the same time other computing tasks do not remain unprocessed for too long.

A calculation task is assigned to a weight function and is represented by it, according to which a subset of a weight function also corresponds to a subset of a calculation task. Accordingly, partial calculation tasks can also be assigned partial weight functions.

In a further development of the invention, it is provided that the method is used for anomaly detection when operating a technical device, which is operated by a client-server system with a server and a client with the at least one first computing device and The following procedural steps must also be carried out: a) Acquiring from the operation of the technical device and providing first training data from the client to the server, generating and training a first model in the form of an auto-encoder by the server, and providing the first model in the form of first weights from the server to the client, as well as loading and storing the first model in the at least one first computing device, b) acquiring operating data of the technical device and determining an anomaly parameter with regard to the agreement with the first model by the at least one first computing device, c) Check whether the anomaly characteristic is within a predetermined value range, if so, continue with step b), otherwise continue with step d), d) provide the first model weights of the auto-encoder and save with the anomaly characteristic in the memory by the at least one first computing device, e) Check whether a predetermined number of stored first model weights of the auto-encoder for respective anomaly parameters has been reached, if so, then continue with step b), otherwise continue with step f ) , f ) Transmitting the stored first model weights for respective anomaly parameters to the at least two second computing devices, which are included by the client, and calculating a second model in the form of an auto-encoder with second model weights by determining the inference by one of the at least two second computing devices, g) transmitting the second model weights of the previously calculated second model to the at least one first computing device, and deleting the first model weights transmitted in step f) from the memory of the at least one nen first computing device, and continue with step b).

This provides for model training in the cloud, but the inference is calculated at the edge, and re-training also takes place at the edge.

This makes it possible to carry out high-performance training in the cloud, as well as to allow process interaction in real time, and also to re-train ML models at the edge, for example to improve model accuracy based on new training data obtained from the previous operation of a technical device with the previous ML model.

Although the data is distributed across clients and servers, by applying federated learning, the new ML model can be distributed to other clients without distributing the raw data itself in the distributed client-server system, resulting in improved data security and privacy .

In a further development of the invention it is provided that the anomaly parameter is the reconstruction error of the auto-encoder.

This makes it possible to determine a particularly meaningful anomaly parameter in a particularly simple manner.

In a further development of the invention, it is provided that the size of the memory occupancy of the memory of the at least one first computing device for storing the number of stored first model weights of the auto-encoder predetermined in step e) is proportional to the bandwidth of the communication connection between the at least a first computing device and at least two second computing devices are provided. This ensures that the model is only updated when a predetermined number of reconstruction errors are exceeded.

In a further development of the invention it is provided that the second model weights calculated in step f) are provided by the client to the server, and the client-server system preferably has a further client with a further technical device and the server is set up for this purpose to provide the second model weights to the other client through federated learning.

This means that the updated second model can be easily used by other clients without having to carry out the necessary arithmetic operations themselves. This means that a design with reduced complexity and lower costs is possible for other clients, and in particular principles of federated learning can be applied.

The object according to the invention is achieved by a computing system of the type mentioned at the outset, comprising at least a first computing device and at least two second computing devices, each with a processor and a memory, the at least one first computing device each being connected to the at least two second computing devices by a Communication connection is connected and the computing system is set up to carry out the method according to the invention.

The object according to the invention is solved by a computer program comprising commands which, when executed by a computer, cause it to carry out the method according to the invention.

The object according to the invention is achieved by an electronically readable data carrier with readable control information stored thereon, which at least corresponds to that according to the invention Include computer program and are designed such that they carry out the method according to the invention when using the data carrier in a computing device.

The object according to the invention is achieved by a data carrier signal which transmits the computer program according to the invention.

The invention is explained in more detail below using an exemplary embodiment shown in the accompanying drawings. In the drawings shows:

Fig. 1 an exemplary embodiment of the method according to the invention as a flow chart,

Fig. 2 an exemplary embodiment of the system according to the invention as a block diagram,

Fig. 3 an exemplary embodiment of a use of the method according to the invention as a flow chart,

Fig. 4 an exemplary embodiment of a use of the system according to the invention as a block diagram.

Fig. 1 shows an exemplary embodiment of the method according to the invention as a flow chart, the method being based on one shown in FIG. 2 can run.

The process for processing a number of computing tasks by a system is at least partially computer-implemented.

A computing system includes at least a first computing device ELC1-ELCN and at least two second computing devices EAC1-EACN, each with a processor and a memory.

The at least one first computing device ELC1-ELCN is connected to the at least two second computing devices EAC1-EACN through respective communication connections.

The following procedural steps are carried out: i) determining a weight function wll-wl3, wNl-wN3 for each calculation task from the set of calculation tasks between the at least one first computing device ELC1-ELCN and the at least two second computing devices EAC1-EACN, the weight function having:

• a priority of the calculation task,

• a communication parameter regarding the communication connection,

• a sequencing factor for processing the calculation problem, preferably starting at zero,

• a memory parameter for the memory required when executing the calculation task,

• an execution parameter for the required computing requirements when executing the computing task, ii) determining the execution capacity with regard to communication, storage and execution properties of the at least two second computing devices EAC1-EACN, iii) sorting the previously determined weight functions wll-wl3 , wNl-wN3, and determining a subset of the sorted weight functions, which lie within a predetermined value range of the determined execution capacity, iv) distributing the subset of the computing tasks, which is assigned to the subset of the sorted weight functions, to the at least two second computing devices EAC1-EACN, Executing the subset of calculation tasks and deleting the associated weight function, v) increasing the ordering factor for the calculation tasks that were not distributed in the previous step due to the execution capacity, vi) As long as weight functions are available, continue with step ii), otherwise terminate.

Optionally, the weight function can be determined in each case by the priority, the communication parameter and the ranking factor each being in a value range between 0 and 1 and the sum of which forms a calculation parameter, and the storage parameter and the execution parameter each in one Value range between 0 and 1 and the sum of which forms a calculation parameter, and the difference between the calculation parameter and the calculation parameter forms the weight function.

For example, the priority P (priority factor) for a current computing task, which is selected by a first computing device, can be in the value range 0% ... 100%.

A high value like 100% means that the model needs an urgent update, and a low value like 1% means that an update is not necessary.

The communication parameter E ("communication efficiency factor") can also be in the value range 0% ... 100%, where a high value such as 100% means that there is very good network quality, and a low value like 1% means that the connection has poor transmission quality.

The ranking factor S (English "starvation factor") can also be in the value range 0% ... 100%, whereby the ranking factor is increased with each iteration, for example by 10%. This serves to determine the urgency of processing a calculation task increase gradually so as not to “starve” the calculation task.

The memory parameter U (“utilization factor”) can also be in the value range 0% ... 100%, whereby the memory Parameters can map a memory occupancy of the main memory and/or hard drive storage. The memory may be required, for example, to temporarily or permanently store a data set on a second computing device.

A high value like 100% means the model has a high memory footprint, and a low value like 1% means the model has a low memory footprint.

The execution parameter C (English "computational factor") can also be in the value range 0% ... 100%, whereby the execution parameter can represent a computing capacity. The computing capacity can be required, for example, to carry out a computing task to a second computing device.

A high value such as 100% means that the model has a high computing requirement - for example due to a high number of parallel processing cores, like a GPU - and a low value such as 1% means that the model has a low computing requirement

The calculation parameter is formed by the sum of the priority, the communication parameter and the ordering factor.

The calculation parameter is formed by the sum of the storage parameter and the execution parameter.

If the calculation parameter is subtracted from the calculation parameter, the weight function w _AB = (P + E + 5) - ((/ + C) is obtained, where A is a first computing device and B is a second computing device.

For any calculation task with a weight function of more than 100%, these “outgoing” edges can be sorted, if to illustrate an analogy to a bipartite graph as shown in Fig. 2 is produced.

After re-sorting, all edges that cannot be processed in the current run are removed and the remaining graph is used for further processing.

The calculation tasks that have not yet been processed are then recorded and processed as a new iteration loop as previously carried out until all calculation tasks have been completed.

For example, P=100%, E=80%, S=0% and U=50%, C= 50% can be selected, so that the weight function is w=80%.

In this example, for a computing task this means that it has a high priority, there is good network bandwidth, the computing task is intended to be processed for the first time, the data set is of moderate size and a moderate computing requirement is planned. It is therefore likely that a request to process the computing task will be completed successfully, even if there is currently a high load on a second computing device. It can also be considered whether a model update is requested in order to process the computing task by the second computing device in order to improve the parameters for determining the weight function.

Furthermore, it can optionally be checked after step i) whether the respective weight function is equal to zero, and if so, the associated calculation task is divided into sub-arithmetic tasks and the sub-arithmetic tasks are added to the set of calculation tasks and the current calculation task is removed from the Amount of calculation tasks is removed and the method is continued with step i).

For example, P=80%, E=50%, S=0% and U=80%, C= 0% can be selected, so that the weight function is w=-30%. In this example, this means that a computing task has a high priority, there is a moderate network bandwidth, the computing task is intended to be processed for the first time, the size of the data set is large and a high computing requirement is planned. It is therefore unlikely that a request to process the computing task will be completed successfully if there is currently a high load on a second computing device, as long as the sequencing factor S is increased or a first computing device divides the computing task into smaller sub-arithmetic tasks with smaller data sets.

In other words, based on the presence of a negative weight function, it can be recognized that a calculation task should advantageously be divided into smaller partial calculation tasks in order to achieve efficient processing.

An advantageous use of the method for anomaly detection when operating a technical device is shown in Fig. 3 shown, for which the system of FIG. 4 can be used.

A client-server system CSS for anomaly detection when operating a technical device TD1 comprises a server S and a client Kl with a first computing device 1 and a further, in this example, identical client K2 with a further technical device TD2.

However, other clients can also be used in the system, which have a simpler design and use the model using federated learning.

The first computing device 1 has a processor and a memory.

The client Kl further comprises a second computing device 2, which communicates with the first computing device 1 via a communication cation connection C is connected, the communication connection C having a predefined bandwidth.

The client Kl preferably records test data DT with the aid of the first computing device 1 during a reference operation of the technical device TD1, which represents a permissible, valid operation of the technical device TD1.

The data acquisition can be done, for example, with a sensor means such as a camera and the test data DT can be camera recordings.

With the help of this test data DT, a global model based on machine learning can be created and trained for the technical device TD1 in the server using an auto-encoder.

Subsequently, the global model can be provided to clients K1, K2 for operating the technical device TD1 in the form of weights of the global model.

The process is computer-implemented, meaning one or more steps are carried out by a computer.

The method can be used particularly advantageously for anomaly detection when operating a technical device TD1, which is operated by a client-server system CSS with a server S and a client Kl with the at least one first computing device ELC1-ELCN, and additionally The following procedural steps are carried out: a) recording from the operation of the technical device TD1 and providing first training data DT from the client Kl to the server S, generating and training a first model in the form of an auto-encoder by the server, and Providing the first model in the form of first weights from the server S to the client Kl, as well as loading and saving the first model in the at least one first computing device ELC1-ELCN, b) acquiring operating data of the technical device TD1 and determining an anomaly parameter with regard to the agreement with the first model by the at least one first computing device ELC1-ELCN, c) checking whether the anomaly parameter lies within a predetermined value range , if yes, continue with step b), otherwise continue with step d), d) providing the first model weights of the auto-encoder and storing the anomaly parameter in the memory by the at least one first computing device ELC1-ELCN, e) Check whether a predetermined number of stored first model weights of the auto-encoder has been reached for the respective anomaly parameters, if so, then continue with step b), otherwise continue with step f), f) transmit the stored first model weights. Weights for respective anomaly parameters to the at least two second computing devices EAC1-EACN, which is included by the client Kl and calculating a second model in the form of an auto-encoder with second model weights by determining the inference by one of the at least two second computing devices EAC1-EACN, g) transmitting the second model weights of the previously calculated second model to the at least one first computing device ELC1-ELCN, and deleting the first model weights transmitted in step f) from the memory of the at least one first computing device ELC1-ELCN , and continue with step b),

The size of the memory occupancy of the memory of the at least one first computing device ELC1-ELCN for storing the number of stored first model weights of the auto-encoder predetermined in step e) is proportional to the bandwidth of the communication connection C between the at least a first computing device ELC1-ELCN and at least two second computing devices EAC1-EACN are provided.

The at least two second computing devices EAC1-EACN are preferably graphics processors ("graphics processing unit", or GPU for short), which are processors specialized and optimized for efficient calculation and can be used for artificial intelligence and machine learning. Because GPUs do an extraordinary amount Offering computing power, they can achieve enormous acceleration in computing tasks due to parallel processing, for example.

A graphics processor can be an integrated part of the first computing device, such as an integrated graphics card of an edge computer.

Providing the first model weights of the auto-encoder in step d) is also referred to as “embedding” compressed features or a compressed feature space, which is formed by the model weights.

It is clear that one or more anomaly metrics can be applied throughout the process.

The second model weights calculated in step f) can be provided to the server S by the client Kl.

Server S can now update its global ML model.

If the client-server system CSS has at least one further client K2 with a further technical device TD2, the system can provide the second model weights to the further client K2 through federated learning. List of reference symbols:

C data connection with transmission capacity

"communication line")

CSS client-server system

D, DT data, training data

EAC computing device with processor and memory,

Client with high processing capacity (“edge advanced computing”)

ELC computing device with processor and memory,

Client with low processing capacity (“edge limited computing”)

Kl, Kl client, edge device

S servers

TD1, TD2 technical device, e.g. motor of a production plant wll-wl3, wNl-wN3 weight, weight function

Claims

Patent claims

1 . Computer-implemented method for processing a set of computing tasks by a system, comprising at least a first computing device (ELC1-ELCN) and at least two second computing devices (EAC1-EACN), each with a processor and a memory, the at least one first computing device (ELC1-ELCN) are connected to the at least two second computing devices (EAC1-EACN) by respective communication connections and the following steps are carried out: i) determining a weight function (wl l-wl 3, wNl-wN3) for each Computing task from the set of arithmetic tasks between the at least one first computing device (ELC1-ELCN) and the at least two second computing devices (EAC1-EACN), the weighting function having:

• a priority of the calculation task,

• a communication parameter regarding the communication connection,

• an execution parameter for the required computing requirements when executing the computing task, ii) determining the execution capacity with regard to communication, storage and execution properties of the at least two second computing devices (EAC1-EACN), iii) sorting the previously certain weight functions (wl l-wl 3 , wNl-wN3 ), and specifying a subset of the sorted weight functions which are within a predetermined value range of the determined execution capacity, iv) distributing the subset of computing tasks, which is assigned to the subset of the sorted weight functions, to the at least two second computing devices (EAC1-EACN), executing the subset of computing tasks and deleting the associated weight function, v) Increasing the ordering factor for the calculation tasks that were not distributed in the previous step due to the execution capacity, vi) As long as weight functions are available, continue with step ii), otherwise terminate.

2. Method according to the preceding claim, wherein the weight function is determined in each case by the priority, the communication parameter and the ranking factor each being in a value range between 0 and 1 and the sum of which forms a calculation parameter, and the storage Parameters and the execution parameters each lie in a value range between 0 and 1 and their sum forms a calculation parameter, and the difference between calculation parameters and calculation parameters forms the weighting function.

3. Method according to the preceding claim, wherein after step i) it is checked whether the respective weight function is equal to zero, and if so, the associated calculation task is divided into sub-arithmetic tasks and the sub-arithmetic tasks are added to the set of calculation tasks and the Current calculation task is removed from the set of calculation tasks and the method is continued with step i).

4. Use of the method according to one of the preceding claims for anomaly detection when operating a technical see device (TD1) which is used by a client server

System (CSS) with a server (S) and a client (Kl) with which at least one first computing device (ELC1-ELCN) is operated and the following procedural steps are additionally carried out: a) Recording from the operation of the technical device (TD1) and providing first training data (DT) from the client (Kl) to the server (S), generating and training a first model in the form of an auto-encoder by the server, and providing the first model in the form of first weights from the server (S) to the client (Kl), as well as loading and saving the first model in the at least one first computing device (ELC1-ELCN), b) acquiring operating data of the technical device (TD1) and determining an anomaly parameter with regard to the Agreement with the first model by the at least one first computing device (ELC1-ELCN), c) Check whether the anomaly parameter is within a predetermined value range, if so, continue with step b), otherwise continue with step d), d) Providing the first model weights of the auto-encoder and storing them with the anomaly parameter in the memory by the at least one first computing device (ELC1-ELCN), e) Checking whether a predetermined number of stored first model weights of the auto-encoder encoder for the respective anomaly parameters is reached, if so, then continue with step b), otherwise continue with step f), f) transmit the stored first model weights for the respective anomaly parameters to the at least two second computing devices (EAC1 -EACN), which is included by the client (Kl ) and calculating a second model in the form of an auto-encoder with second model weights by determining the inference by one of the at least two second computing devices (EAC1-EACN), g) transmitting the second model weights of the previously calculated second model to the at least one first computing device (ELC1-ELCN), and deleting those transmitted in step f). first model weights from the memory of the at least one first computing device (ELC1-ELCN), and continuing with step b),

5. Use according to the preceding claim, wherein the anomaly parameter is the reconstruction error of the auto-encoder.

6. Use according to one of claims 4 or 5, wherein the size of the memory occupancy of the memory of the at least one first computing device (ELC1-ELCN) for storing the number of stored first model weights of the auto-encoder predetermined in step e) is proportional to the bandwidth the communication connection (C) is provided between the at least one first computing device (ELC1-ELCN) and the at least two second computing devices (EAC1-EACN).

7. Use according to one of claims 4 to 6, wherein the second model weights calculated in step f) are provided by the client (Kl) to the server (S), and the client-server system preferably includes a further client (K2). a further technical device (TD2) and the server (S) is set up to provide the second model weights to the further client (K2) through federated learning.

8th . Computing system for processing a set of computing tasks, comprising at least a first computing device (ELC1-ELCN) and at least two second computing devices (EAC1-EACN), each with a processor and a memory, the at least one first computing device (ELC1- ELCN) is each connected to the at least two second computing devices (EAC1-EACN) by a communication connection and the computing system is set up to carry out the method according to one of claims 1 to 3.

9. Computer program, comprising commands which are used in their

Execution by a computer causes it to carry out the method according to one of claims 1 to 3.

10. Electronically readable data carrier with readable control information stored thereon, which includes at least the computer program according to the preceding claim and is designed in such a way that when the data carrier is used in a computing device, they carry out the method according to one of claims 1 to 3.

11. Data carrier signal which the computer program according to claim 9 transmits.