WO2024056178A1

WO2024056178A1 - A device and method for network traffic classification

Info

Publication number: WO2024056178A1
Application number: PCT/EP2022/075646
Authority: WO
Inventors: Lixuan YANG; Alessandro Finamore; Fuxing Chen; Dario Rossi
Original assignee: Huawei Technologies Co., Ltd.
Priority date: 2022-09-15
Filing date: 2022-09-15
Publication date: 2024-03-21

Abstract

The present disclosure relates to classification of network traffic. The disclosure provides a device and a method for network traffic classification. The device is configured to receive a labelled training data set as an input, and to obtain a trained classification model, which was trained based on the labelled training data set. Further, the device is configured to determine one or more misclassifications of the labelled training data set by the trained classification model, which violate one or more operational intents. Then, the device is configured to update the trained classification model based on the labelled training data set, the one or more determined misclassifications, and a loss-function. The method functions in the same way. The device and method can correct the one or more determined misclassifications by the updating of the trained classification model, and taking into account the one or more operational intents.

Description

A DEVICE AND METHOD FOR NETWORK TRAFFIC CLASSIFICATION

TECHNICAL FIELD

The present disclosure relates to network traffic analytics, specifically to classification of network traffic. The disclosure provides a device and a method for network traffic classification. The device and method optimize a trained classification model for network traffic classification based on a labelled training data set and one or more operational intents.

BACKGROUND

Network traffic classification is usually performed by a Traffic Classification (TC) engine, which is a component of a traffic monitoring system aiming to identify which application has generated a certain network traffic, for instance, a certain network flow.

Traffic classification technologies applied in TC engines can be split into two classes. A first class is traffic classification based on deep packet inspection (DPI) and relies on very large catalogs of rules that can identify thousands of applications. Although such rules could be generated even from a single traffic flow (or even from a packet), those rules are typically handcrafted. Thus, DPI-based TC engines require a lot of manual effort and domain knowledge. Further, DPI-based TC engines suffer from poor automation, and do not cope well with encrypted traffic, because the rules are based on packet payload content. As the portion of encrypted traffic is steadily growing, DPI-based classification is thus being phased out in favor of alternative technologies.

A second class is traffic classification based on artificial intelligence (Al) and is one such alternative. Al-based classification relies on algorithms to learn patterns in a data-driven fashion. This means that Al-based classification can be highly automated, and can also better cope with encrypted traffic, because it can simply rely on generic packet features (e.g., packet size), without the need to check packet content (e.g., header field values). However, the training of classifiers for such an Al-based traffic classification requires large labeled datasets, which typically can only be obtained for popular applications. Thus, Al-based classification typically targets only tens of applications generating the majority of the traffic. SUMMARY

The present disclosure and its solutions are based the following considerations.

During the training phase of an Al-based classifier, the underlying model is optimized with respect to the training data distribution. However, since Al-based classifiers are not perfect, misclassifications can occur, often with significant impact on the ability to properly operate those models.

As an example, an internal policy of an enterprise network may prevent users to access social network applications during their worktime, and therefore connections to the social networks may be blocked by a firewall/gateway of the enterprise. If now one expected class (e.g., an allowed search engine of a company) is confused with an array of other classes (e.g., with a social network application from the same company), by a trained classifier used to implement the policy (e.g., block social network traffic), some of the allowed flows (e.g., search queries to the search engine) may be blocked as a consequence of the misclassification. Thus, such misclassifications may be disadvantageous.

Further, most conventional classification techniques are not capable of attaining and guaranteeing a 100% accuracy under all circumstances. Therefore, effective traffic classification engines need to align with the operational requirements (e.g., policies) of an enterprise. In practice, such operational requirements may vary significantly from one enterprise to the other, and thus it may not be possible to create in advance all models capable of satisfying all kinds of policy (due to combinatorial explosion of the policy space). Additionally, the operational requirement and policies may be constantly evolving, thus, the models may need to be updated.

Integrating such model updates or changes is not easily reflected in a re-training cycle. For instance, for each change, a whole new model may need to be created, starting from the beginning. Such a procedure is time and resource consuming, in particular, in case of frequent changes.

In view of the above, this disclosure aims for an improved network traffic classification device and method. An objective is to enable the correction of misclassifications of a model. Thereby, operational requirements should be taken into account. These and other objectives are achieved by this disclosure as described in the independent claims. Advantageous implementations are described in the dependent claims.

A first aspect of this disclosure provides a device for network traffic classification, the device being configured to: receive a labelled training data set as an input; obtain a trained classification model, which was trained based on the labelled training data set; determine one or more misclassifications of the labelled training data set by the trained classification model, which violate one or more operational intents; and update the trained classification model based on the labelled training data set, the one or more determined misclassifications, and a lossfunction.

The device of the first aspect may thus be able to optimize the trained classification model for network traffic classification, based on the labelled training data set and the one or more operational intents. The one or more operational intents may reflect operational requirements as described above. These operational intents may set boundary conditions or constraints for the updating of the trained classification model, and may allow the trained classification model to classify network traffic more accurately, with less misclassifications, while taking into account the constraints set by the operational intents. The operational intents may thus be considered as rules for the traffic classification.

The device of the first aspect enables correction of misclassifications of the trained classification model. In this way improved network traffic classification device can be achieved.

In an implementation form of the first aspect, the device is configured to correct the one or more determined misclassification by updating the trained classification model.

The updated or corrected trained classification model is suited for a more accurate network traffic classification, with less misclassifications.

In an implementation form of the first aspect, the device is configured to generate a dataset including one or more traffic samples corresponding to the one or more determined misclassifications, and to update the trained classification model based on the dataset. In an implementation form of the first aspect, the one or more operational intents are user- specified, have different intent granularities, and comprise at least one of a pair wise operational intent, indicating two similar but different traffic classes; a class intent, indicating an intended classification result for a specific class; a sample wise intent, indicating an intended classification result for one or more traffic samples; a complex intent, resulting by any combination of the above; wherein the operational intent specifies the pair and/or class and/or sample or combination thereof, for which a classification result is expected to be correct.

The trained classification model can accordingly be updated by the device, to reduce misclassifications, while taking into account at least one of the user-specified operational intents.

In an implementation form of the first aspect, the trained classification model comprises a trained neural network (NN).

In an implementation form of the first aspect, updating the trained NN comprises updating weights of the trained NN based on the loss function, in order to correct the one or more misclassifications to comply with the one or more operational user-specified intents.

In an implementation form of the first aspect, the loss function is designed to cause an unlearning by the trained classification model of the one or more misclassifications, and to retain the trained classification model of one or more correct classifications.

In this way, the misclassifications can be reduced or removed efficiently.

In an implementation form of the first aspect, the loss function combines a forgetting loss for causing the unlearning, and a classic learning loss to retain classification performance for the other classes.

In an implementation form of the first aspect, the learning loss comprises at least one of a cross entropy loss, a mean square error, a focal loss, and a weighted loss; and/or the forgetting loss comprises at least one of an inverse of the cross entropy loss and a hessian weighted invers of the cross entropy loss. These implementation forms achieve good results in correcting misclassifications.

In an implementation form of the first aspect, the device is configured to obtain the trained classification model by running a training process on the labelled training data set, or to receive the trained classification model as an input.

The training process and the overall process for obtaining the trained classification model can work in a conventional manner. The focus of this disclosure is on updating and correcting such a previously trained classification model.

A second aspect of this disclosure provides a method for network traffic classification, the method comprising of receiving a labelled training data set as an input; obtaining a trained classification model, which was trained based on the labelled training data set; determining one or more misclassifications of the labelled training data set by the trained classification model, which violate one or more operational intents; and updating the trained classification model based on the labelled training data set, the one or more determined misclassifications, and a loss-function.

In an implementation form of the second aspect, the method comprises correcting the one or more determined misclassification by updating the trained classification model.

In an implementation form of the second aspect, the method comprises generating a dataset including one or more traffic samples corresponding to the one or more determined misclassifications, and to update the trained classification model based on the dataset.

In an implementation form of the second aspect, the one or more operational intents are user- specified, have different intent granularities, and comprise at least one of a pair wise operational intent, indicating two similar but different traffic classes; a class intent, indicating an intended classification result for a specific class; a sample wise intent, indicating an intended classification result for one or more traffic samples; a complex intent, resulting by any combination of the above; wherein the operational intent specifies the pair and/or class and/or sample or combination thereof, for which a classification result is expected to be correct. In an implementation form of the second aspect, the trained classification model comprises a trained NN.

In an implementation form of the second aspect, updating the trained NN comprises updating weights of the trained NN based on the loss function, in order to correct the one or more misclassifications to comply with the one or more operational user-specified intents.

In an implementation form of the second aspect, the loss function is designed to cause an unlearning by the trained classification model of the one or more misclassifications, and to retain the trained classification model of one or more correct classifications

In an implementation form of the second aspect, the loss function combines a forgetting loss for causing the unlearning, and a classic learning loss to retain classification performance for the other classes.

In an implementation form of the second aspect, the learning loss comprises at least one of a cross entropy loss, a mean square error, a focal loss, and a weighted loss; and/or the forgetting loss comprises at least one of an inverse of the cross entropy loss and a hessian weighted invers of the cross entropy loss.

In an implementation form of the second aspect, the method comprises obtaining the trained classification model by running a training process on the labelled training data set, or to receive the trained classification model as an input.

The method of the second aspect and its implementation forms achieve the same advantages as described above for the device of the first aspect and its respective implementation forms.

A third aspect of this disclosure provides a computer program comprising instructions which, when the program is executed by a computer, cause the computer to perform the method according to the second aspect or any of its implementation forms.

A fourth aspect of this disclosure provides a non-transitory storage medium storing executable program code which, when executed by a processor, causes the method according to the second aspect or any of its implementation forms to be performed. In summary of the above-described aspects and implementation forms, this disclosure provides an iterative way to perform post-training adaptations of the trained classification model, in order to integrate operational requirements - reflected by the operational intents - not known when training of the model is/was performed. This disclosure provides a novel device and novel method for updating the trained classification model and addressing misclassification and/or mislabeling according to the operational intents. The disclosure, for example, proposes leveraging machine unlearning methods to achieve a faster and/or more accurate correction of the trained classification model. This may mean, in other words, that the correction of the trained classification model can be driven by the operational requirements and an update based on machine unlearning techniques.

It has to be noted that all devices, elements, units and means described in the present application could be implemented in the software or hardware elements or any kind of combination thereof. All steps which are performed by the various entities described in the present application as well as the functionalities described to be performed by the various entities are intended to mean that the respective entity is adapted to or configured to perform the respective steps and functionalities. Even if, in the following description of specific embodiments, a specific functionality or step to be performed by external entities is not reflected in the description of a specific detailed element of that entity which performs that specific step or functionality, it should be clear for a skilled person that these methods and functionalities can be implemented in respective software or hardware elements, or any kind of combination thereof.

BRIEF DESCRIPTION OF DRAWINGS

The above described aspects and implementation forms will be explained in the following description of specific embodiments in relation to the enclosed drawings, in which

FIG. 1 shows a device for network traffic classification according to this disclosure.

FIG. 2 illustrates a procedure performed by a device for network traffic classification according to this disclosure.

FIG. 3 illustrates advantages of the present disclosure.

FIG. 4 shows a method for network traffic classification according to this disclosure. DETAILED DESCRIPTION OF EMBODIMENTS

FIG. 1 shows a device 100 for network traffic classification according to this disclosure. The device 100 may be implemented in a network node or device, or in user equipment (UE), or the like. The device 100 may be a TC engine, and may be part of a network traffic classification system.

The device 100 is configured to receive a labelled training data set 101 as an input. The labelled training data set 101 may be sent to the device 100 by another entity, for example, another device in the same network as the device 100, or may be input manually or by configuration into the device 100. The labelled training data set 101 may be preinstalled into the device 100, before the device 100 is operated.

Further, the device 100 is configured to obtain a trained classification model 102, which was trained based on the labelled training data set 101. The trained classification model may be a NN or DNN. The training of the trained classification model 102 may be performed by the device 100. That is, the device 100 may be configured to obtain the trained classification model 102 by running a training process on the labelled training data set 101. Alternatively, some other device or entity may first train the trained classification model 102, which is then sent to or configured at the device 100. In this case, the device 100 may be configured to receive the trained classification model 102 as an input (as exemplarily illustrated in FIG. 1).

The device 100 is further configured to determine one or more misclassifications 103 of the labelled training data set 101 by the trained classification model 102, wherein these one or more misclassifications violate one or more operational intents. That is, operational requirements taken into account may lead to a certain expected classification, while the trained classification model 102 provided a different classification. That is, a misclassification 103 in view of these operational requirements, which are reflected by the one or more operational intents. The operational intents may also be identical to the operational requirements. The one or more operational intents may be user-specified. For example, the one or more operational intents may also have different intent granularities, or different operational intents may have different priority. An operational intent may specify a pair of traffic classes, and/or a specific class, and/or a specific traffic sample, for which a classification result is expected to be correct. The device 100 is further configured to update the trained classification model 102 based on the labelled training data set 101, based on the one or more determined misclassifications 103, and based on a loss-function 104. In particular, the device 100 is configured to correct the one or more determined misclassifications 103 by updating the trained classification model 102. For example, the device 100 may update a trained NN - being the trained classification model 102, for example - by updating the weights of the trained NN based on the loss function 104, in order to correct the one or more misclassifications 103. In particular, correcting the one or more misclassifications 103 so as to comply with the one or more operational intents.

The device 100 may comprise a processor or processing circuitry (not shown) configured to perform, conduct or initiate the various operations of the device 100 described herein. The processing circuitry may comprise hardware and/or the processing circuitry may be controlled by software. The hardware may comprise analog circuitry or digital circuitry, or both analog and digital circuitry. The digital circuitry may comprise components such as applicationspecific integrated circuits (ASICs), field-programmable arrays (FPGAs), digital signal processors (DSPs), or multi-purpose processors. The device 100 may further comprise memory circuitry, which stores one or more instruction(s) that can be executed by the processor or by the processing circuitry, in particular under control of the software. For instance, the memory circuitry may comprise a non-transitory storage medium storing executable software code which, when executed by the processor or the processing circuitry, causes the various operations of the device 100 to be performed. In one embodiment, the processing circuitry comprises one or more processors and a non-transitory memory connected to the one or more processors. The non-transitory memory may carry executable program code which, when executed by the one or more processors, causes the device 100 to perform, conduct or initiate the operations or methods described herein.

For instance, one or more processors or processing circuitry of the device 100 may be configured to take the labelled training data set 101 as an input, and to perform the steps of obtaining (e.g., receiving or training) the trained classification model 102, determining the one or more misclassifications 103 and updating the trained classification model 102 based on the labelled training data set 101, the one or more determined misclassifications 103, and the lossfunction 104. FIG. 2 illustrates a procedure performed by a device 100 for network traffic classification according to this disclosure, wherein the device 100 of FIG. 2 builds on the device 100 shown in FIG. 1. Accordingly, also the device 100 of FIG. 2 is configured to receive the labelled training data set 101 as an input, and to obtain the trained classification model 102, for example, by also receiving it as an input as illustrated. The procedure performed by the device 100 may include/combine two steps as illustrated and described below.

A first step may be a step of checking the trained classification model 102, in order to localize the misclassifications 103 that violate the operational intent(s) 201. The device 100 may be configured to generate a misclassification dataset 203 (referred to as D_mis in FIG. 2), which may include one or more traffic samples corresponding to the one or more determined misclassifications 103. This dataset 203 may be passed on to the next step.

A second step may be a step of misclassification correction, in order to correct the misclassifications 103 determined before, by updating the trained classification model 102 with a special loss function 104.

For instance, it may be assumed that the labelled training dataset 101 comprises traffic samples (xi, yi) wherein x; corresponds to some input traffic features and y i = { 1 ... N} is the associated label. For instance, in a traffic classification scenario, x; may be the packet length for the first P packets of a flow, and yi may be the related application selected as one among N applications. If the labelled training dataset 101 is available, traffic samples from the training data set 101 that violate the operational intents 201 set can be collected as boundary conditions or constraints. Those traffic samples may be used to construct misclassification dataset 203.

The device 100 may thereby allow prioritizing certain model updates of the trained classification model 102, by specifying the one or more operational intents 201 of the misclassification correction, at multiple levels. For instance, pair wise and abstract intents can be specified a priori (either prior to the first training, or after), while sample-wise correction intents may further allow to correct problems detected (automatically, or by human intervention) at later stages (e.g., test time and subsequent deployment).

An example of a pair wise operational intent 201 may be “Do not confuse Class A with Class B”. A pair wise operational intent may indicate two similar but different traffic classes. This case is the simplest. In this case, the operational intent may concern two classes A and B (e.g., a web search vs. a social network), and more particularly the set of samples of A that are misclassified as class B. These traffic samples may be added to the misclassification dataset 203, as they constitute misclassification errors that need to be fixed. In particular, such errors may appear in a “confusion matrix”, in which the position (A, B) of the matrix may have nonzero elements. An operational intent 201 may be to correct the model 102 so that the model 102 labels these samples as A, and brings them in the diagonal position (A, A) of the confusion matrix.

An example of an abstract operational intent 201 may be “Do not block a web search”. The description may be mapped to the classes used, e.g., based on knowledge about the traffic or the model 102 (e.g., an operator knows which are the classes recognized by the classification device 100 that match “web search”). After mapping, the misclassification dataset 203 may include all traffic samples for the classes corresponding to web search (e.g., class W) that are misclassified for applications that are blocked in the target environment (e.g., classes Bl, B2, .. Bn). This may translate into correcting several pair wise intents (W, Bl), (W, Bn) as previously introduced.

An example of a sample wise operational intent may be “Do not misclassify these samples” . A sample wise intent may indicate an intended classification result for one or more traffic samples. One can collect hard examples when the model 102 is already deployed (e.g., using novelty discovery and/or open set recognition methods). Hard examples are traffic samples that in the current model’s 102 prediction have low confidence. It is also possible to involve a human in the loop tasked to check the quality of model’s prediction (e.g., as in case of misclassifications that lead to troubleshooting tickets as they violate the operational intent) to construct the misclassification dataset 203. The annotator (human or automated) can also associate a priority to each misclassification 103. The misclassification 103 can be corrected accordingly with associated priority.

The model updating with misclassification correction can be performed as follows, for the example of the trained classification model being a NN, in particular, a traditional learning cycle of a NN may work at follows. Given an input Xi, the NN network is trained by updating the weights according to the loss function 104. The loss function 104 may measure the distance between the network output yi_pred (prediction) with an expected output yigt (ground truth). As illustrated FIG. 3, by using a typical learning loss 104 (e.g., a cross entropy), the weights may be updated to have smaller output for the other classes, except for the ground truth. In turn, the output may pushed to have larger values. Consequently, the model 102 is trained to produce a desired output.

In the context of this disclosure, to correct errors, the model 102 needs to “forget” about those and to “replace” them with new knowledge. The purpose of machine unlearning is to let the model 102 “forget” the contribution of data. By using a typical forgetting loss (e.g., inverse of cross entropy), the correct output may be pushed to be smaller while the other outputs are not changed. In this way the model 102 becomes more “ambiguous” about the output, thus “forgets” the desired output.

For correcting the misclassifications 103, the loss function 104 may be defined as the combination of the forgetting loss and the learning loss (e.g., a traditional cross entropy). In other first, the forgetting loss (Lf) aims to forget the misclassification 103 given by the uncorrected classification model 102 and takes (xj, yimis) as input. The learning loss (Lc) reinforces the correct classification and takes (xj, yigt) as input. Therein, xj, yimis, yigt denote, respectively, input, misclassified label, and ground truth label. As illustrated in the example of FIG. 3, yimis is 0, yigt is 3.

Including a memory 202 during the model 102 update, as shown in FIG. 2, may reduce the catastrophic forgetting effect. During the correction phase, the accuracy of other classes may be maintained. The memory 202 can be chosen by following mechanisms: random sampling, herding, etc.

FIG. 3 also illustrates advantages of this disclosure. In the example of FIG. 3, there is a classifier (implemented by the trained classification model 102 of the device 100) with four classes, and for a sample considered in the example, the classification reported is class 0. However, suppose that it is discovered that the actual class of this sample is class 3, then an aim is to change class 0 to class 3 for this specific sample. Intuitively, to perform this change, one needs to reduce the confidence of class 0 and to promote the one for class 3, while maintaining the confidence of the other classes unchanged. This is exactly the effect to combine the forgetting loss (Lf) with the learning loss (Lc). Specifically, the combination of the two loss function allows to have the following desired effects: (1) Push back the misclassified output (correct the wrong predict); (2) Push forward the right output (inforce the correct label); (3) Zero out the impact for others (without impacting the output of others).

FIG. 4 shows a method according to this disclosure. The method 400 is for classifying network traffic. The method 400 may be performed by the device 100.

The method 400 comprises a step 401 of receiving a labelled training data set 101 as an input. Further, the method 400 comprises a step 402 of obtaining a trained classification model 102, which was trained based on the labelled training data set 101. Then, the method 400 comprises a step 403 of determining one or more misclassifications 103 of the labelled training data set 101 by the trained classification model 102, which violate one or more operational intents 201. Further, the method 400 comprises a step 404 of updating the trained classification model 102 based on the labelled training data set 101, the one or more determined misclassifications 103, and a loss-function 104.

The loss function 104 in this disclosure may combine a forgetting loss, for causing an unlearning by the trained classification model 102 of the one or more misclassifications 103, with a classic learning loss, for retaining classification performance for the other classes. The learning loss can be chosen from the following list: cross entropy loss, mean squared error, focal loss, and weighted loss. The unlearning loss can be chosen from the following list: inverse of cross entropy loss, and hessian weighted inverse of cross entropy loss. Considering the combination of the loss function, it may be a sum, average, or weighted average. That is, the device 100 or method 400 may combine the forgetting loss and the learning loss by summing, averaging, or building a weighted average of, the forgetting loss and the learning loss.

The present disclosure has been described in conjunction with various embodiments as examples as well as implementations. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed matter, from the studies of the drawings, this disclosure and the independent claims. In the claims as well as in the description the word “comprising” does not exclude other elements or steps and the indefinite article “a” or “an” does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in the mutual different dependent claims does not indicate that a combination of these measures cannot be used in an advantageous implementation.

Claims

1. A device (100) for network traffic classification, the device (100) being configured to: receive a labelled training data set (101) as an input; obtain a trained classification model (102), which was trained based on the labelled training data set (101); determine one or more misclassifications (103) of the labelled training data set (101) by the trained classification model (102), which violate one or more operational intents (201); and update the trained classification model (102) based on the labelled training data set (101), the one or more determined misclassifications (103), and a loss-function (104).

2. The device (100) according to claim 1, configured to correct the one or more determined misclassifications (103) by updating the trained classification model (102).

3. The device (100) according to claim 1 or 2, configured to generate a dataset including one or more traffic samples corresponding to the one or more determined misclassifications (103), and to update the trained classification model (102) based on the dataset.

4. The device (100) according to one of the claims 1 to 3, wherein the one or more operational intents (201) are user-specified, have different intent granularities, and comprise at least one of:

• a pair wise operational intent, indicating two similar but different traffic classes;

• a class intent, indicating an intended classification result for a specific class;

• a sample wise intent, indicating an intended classification result for one or more traffic samples;

• a complex intent, resulting by any combination of the above wherein the operational intent (201) specifies the pair and/or class and/or sample or combination thereof, for which a classification result is expected to be correct.

5. The device (100) according to one of the claims 1 to 4, wherein the trained classification model (102) comprises a trained neural network, NN.

6. The device (100) according to claim 5, wherein updating the trained NN comprises updating weights of the trained NN based on the loss function (104), in order to correct the one or more misclassifications (103) to comply with the one or more operational user-specified intents (201).

7. The device (100) according to one of the claims 1 to 6, wherein the loss function (104) is designed to cause an unlearning by the trained classification model (102) of the one or more misclassifications (103), and to retain the trained classification model (!02) of one or more correct classifications.

8. The device (100) according to claim 7, wherein the loss function (104) combines a forgetting loss for causing the unlearning, and a classic learning loss to retain classification performance for the other classes.

9. The device (100) according to claim 8, wherein: the learning loss comprises at least one of a cross entropy loss, a mean square error, a focal loss, and a weighted loss; and/or the forgetting loss comprises at least one of an inverse of the cross entropy loss and a hessian weighted invers of the cross entropy loss.

10. The device (100) according to one of the claim 1 to 9, configured to obtain the trained classification model (102) by running a training process on the labelled training data set (101), or to receive the trained classification model (102) as an input.

11. A method (400) for network traffic classification, the method (400) comprising: receiving (401) a labelled training data set (101) as an input; obtaining (402) a trained classification model (102), which was trained based on the labelled training data set (101); determining (403) one or more misclassifications (103) of the labelled training data set (101) by the trained classification model (102), which violate one or more operational intents (201); and updating (404) the trained classification model (102) based on the labelled training data set (101), the one or more determined misclassifications (103), and a loss-function (104).

12. A computer program comprising instructions which, when the program is executed by a computer, cause the computer to perform the method (400) according to claim 11.