WO2023245620A1

WO2023245620A1 - Handover failure cause classification

Info

Publication number: WO2023245620A1
Application number: PCT/CN2022/101119
Authority: WO
Inventors: Zhenhua HE; Jinghao WANG; Min Liu; Huaxiong XU
Original assignee: Telefonaktiebolaget Lm Ericsson (Publ)
Priority date: 2022-06-24
Filing date: 2022-06-24
Publication date: 2023-12-28

Abstract

A method (1200) for facilitating a telecommunication network (10) in reducing its HO failures comprises: determining (S1210) an HO failure graph for a first cell (205-1) at least based on PM data associated with the first cell (205-1); determining (S1220) a failure category for the HO failure graph by using a trained GNN model (700); and triggering (S1230) one or more remedy actions corresponding to the determined failure category to reduce HO failures associated with the first cell (205-1).

Description

HANDOVER FAILURE CAUSE CLASSIFICATION

Technical Field

The present disclosure is related to the field of telecommunication, and in particular, to a method and an electronic device for handover (HO) failure cause classification.

Background

With the development of the electronic and telecommunication technologies, mobile devices, such as mobile phones, smart phones, laptops, tablets, vehicle mounted devices, become an important part of our daily lives. To support a numerous number of mobile devices, a Radio Access Network (RAN) , such as a fourth generation (4G) Long Term Evolution (LTE) RAN or a fifth generation (5G) New Radio (NR) RAN, will be required.

With the handover procedure, a user equipment (UE) may travel across different cells served by different base stations (e.g., eNBs, gNBs) without loss of connectivity, thus enabling a seamless data communication for voice calls, video calls, gaming, etc. However, it is still possible for a handover procedure to fail due to many different causes, such as, insufficient capacity of a target cell, degraded radio links, Physical Cell Identity (PCI) confusion, etc. Typically, different measures shall be taken to address HO failures with different failure causes, respectively.

Summary

Therefore, a method for classifying HO failure causes is needed. With some embodiments of the present disclosure, a method and an electronic device for HO failure cause classification are provided.

According to a first aspect of the present disclosure, a method for facilitating a telecommunication network in reducing its HO failures is provided. The method comprises: determining an HO failure graph for a first cell at least based on performance measurement (PM) data associated with the first cell; determining a failure category for the HO failure graph by using a trained Graph Neural Network (GNN) model. Further, in some embodiments, one or more remedy actions corresponding to the determined failure category may be triggered to reduce HO failures associated with the first cell.

In some embodiments, the step of determining the HO failure graph for the first cell comprises: determining one or more neighbor cells for the first cell at least based on the PM data; and determining the HO failure graph such that the one or more neighbor cells and the first cell correspond to vertices of the HO failure graph, respectively, and an edge is present between any two vertices in the HO failure graph only when cells corresponding to the two vertices have a neighbor relation. In some embodiments, the PM data is PM data during an abnormal Recording Output Period (ROP) with the maximum HO outgoing preparation attempts.

In some embodiments, the step of determining the HO failure graph for the first cell comprises: removing, from the HO failure graph, at least one edge between at least one pair of vertices, wherein for each of the at least one pair of vertices, a number of outgoing preparation attempts between corresponding cells is less than a threshold. In some embodiments, one or more features are associated with each vertex in the HO failure graph. In some embodiments, the one or more features comprise at least one of: a Preparation Success Rate as Originator (PSRO) ; a Preparation Success Rate as Terminator (PSRT) ; an Execution Success Rate as Originator (ESRO) ; an Execution Success Rate as Terminator (ESRT) ; a Preparation Reject Rate as Originator (PRRO) ; and an Access Successful Rate (ASE) .

In some embodiments, at least one of following is true:

-

and

-

In some embodiments, the trained GNN model comprises at least one of: one or more graph convolution layers; one or more sort pooling layers; one or more 1D convolution layers and max pooling layers; and one or more fully connected layers. In some embodiments, the determined failure category for the HO failure graph is one of multiple failure categories that has the highest probability that is determined by the trained GNN model based on the HO failure graph. In some embodiments, before the step of determining the HO failure graph for the first cell, the method further comprises: determining whether there is HO failure anomaly for one or more cells comprising the first cell at least based on the PM data. In some embodiments, for at least one of the one or more cells, the step of determining whether there is HO failure anomaly comprises at least one of: determining whether there is an abnormal ROP associated with one or more cells or not; determining which of the one or more cells has a number of abnormal ROPs greater than a threshold during a period of time, as first candidate cells; sorting the first candidate cells by their average deviations of anomaly time during a period of time in ascending order, and determining one or more of the first candidate cells with top ranks, as second candidate cells; and sorting the second candidate cells by their average busy period connected users during a period of time in descending order, and determining one or more of the second candidate cells with top ranks, as a list of HO failure cells.

In some embodiments, the step of determining whether there is an abnormal ROP associated with one or more cells or not comprises at least one of: determining whether an outgoing HO success rate (OHSR) is lower than a threshold or not; and determining whether a number of outgoing preparation attempts (OPAs) is higher than a threshold or not.

In some embodiments, at least one of following is true:

-

and

- OPA = Sum of Outgoing Preparation Attempts.

In some embodiments, an average deviation of anomaly time for a cell during the period of time is determined as an average of time differences between all anomaly times associated with the cell during the period of time and the median of all anomaly times associated with the cell during the period of time. In some embodiments, an average busy period connected users during a period of time is determined as a number of average connected users in a busy period during the period of time. In some embodiments, a peak ROP in a day associated with a cell is an ROP during which the cell serves the greatest number of average connected users in the day, wherein a busy period associated with a cell is an ROP that has the greatest number of days, in each of which the ROP is the peek ROP, during the period of time. In some embodiments, at least one cell in the list of HO failure cells is determined as the first cell. In some embodiments, the PM data comprises at least one of: a count of preparation success; a count of preparation attempts; a count of execution success; a count of preparation rejects; a count of random access success; a count of random access attempts; a count of connection setup success; a count of connection setup attempts; a number of average connected users. In some embodiments, the method is performed at an Operation, Administration, & Maintenance (OAM) platform.

In some embodiments, the determined failure category is at least one of: Preparation Denied; Preparation Failure due to no network response; Execution failure due to UE missing; and Execution failure due to target cell access failure.

According to a second aspect of the present disclosure, an electronic device is provided. The electronic device comprises: a processor; a memory storing instructions which, when executed by the processor, cause the processor to perform any of the methods of the first aspect.

According to a third aspect of the present disclosure, an electronic device is provided. The electronic device comprises: a first determining module configured to determine an HO failure graph for a first cell at least based on PM data associated with the first cell; a second determining module configured to determine a failure category for the HO failure graph by using a trained GNN model; and a triggering module configured to trigger one or more remedy actions corresponding to the determined failure category to reduce HO failures associated with the first cell. In some embodiments, the electronic device further comprises one or more module configured to perform any of the methods of the first aspect.

According to a fourth aspect of the present disclosure, a computer program comprising instructions is provided. The instructions, when executed by at least one processor, cause the at least one processor to carry out the method of the first aspect.

According to a fifth aspect of the present disclosure, a carrier containing the computer program of the fourth aspect is provided. The carrier is one of an electronic signal, optical signal, radio signal, or computer readable storage medium.

With some embodiments of the present disclosure, a cause isolation method based on performance measurements in the Root Cause Analysis (RCA) of handover performance degradation is provided. The cause of handover performance degradation can be narrowed down to one of a limited number of categories and the speed to solve the problem is improved as less diagnosis procedure is needed after classification.

Further, the cause isolation method of some embodiments of the present disclosure may outperform the rule-based methods. Traditional rules need decide key performance indicator (KPI) thresholds to attribute the problem to the root cause of handover failures. However, there are more than 40 possible causes, and the rules are very complicated. By contrast, with the cause isolation method of some embodiments of the present disclosure, the training of the model may only need human operators to label each problem to 4 kinds of category in the training set, which is much easier to be obtained.

Furthermore, the GNN model of some embodiments of the present disclosure may outperform other classification models such as decision trees in generalization capability. The advantage of the GNN model is that the input to the model is irrelevant to the scale of the graph. The GNN model could accept a graph with two connected cells or a network with tens of connected cells. Graphs with different sizes and structures may be supported by the classification model of some embodiments of the present disclosure.

Brief Description of the Drawings

Fig. 1 is a diagram illustrating exemplary handover failures for which failure causes can be classified according to an embodiment of the present disclosure.

Fig. 2A is a diagram illustrating an exemplary cell network generated according to an embodiment of the present disclosure.

Fig. 2B is a diagram illustrating a simplified version of the exemplary cell network shown in Fig. 2A.

Fig. 3 is a diagram illustrating an exemplary GNN according to an embodiment of the present disclosure.

Fig. 4 is a diagram illustrating exemplary services for classifying HO failure causes according to an embodiment of the present disclosure.

Fig. 5 is a diagram illustrating an exemplary handover failure anomaly detection module according to an embodiment of the present disclosure.

Fig. 6 is a diagram illustrating an exemplary handover failure graph construction module according to an embodiment of the present disclosure.

Fig. 7 is a diagram illustrating an exemplary handover failure graph classification module according to an embodiment of the present disclosure.

Fig. 8 is a diagram illustrating an exemplary handover failure graph classification model training service according to an embodiment of the present disclosure.

Fig. 9 is a diagram illustrating test results of classifications of handover failure causes according to an embodiment of the present disclosure.

Fig. 10 is a diagram illustrating an exemplary overall procedure for handover failure cause classification according to an embodiment of the present disclosure.

Fig. 11 is a diagram illustrating an exemplary overall procedure for training a model for handover failure cause classification according to an embodiment of the present disclosure.

Fig. 12 is a flow chart illustrating an exemplary method for facilitating a telecommunication network in reducing its HO failures according to an embodiment of the present disclosure.

Fig. 13 schematically shows an embodiment of an arrangement which may be used in an electronic device for facilitating a telecommunication network in reducing its HO failures according to an embodiment of the present disclosure.

Fig. 14 is a block diagram of an exemplary electronic device for facilitating a telecommunication network in reducing its HO failures according to an embodiment of the present disclosure.

Detailed Description

Hereinafter, the present disclosure is described with reference to embodiments shown in the attached drawings. However, it is to be understood that those descriptions are just provided for illustrative purpose, rather than limiting the present disclosure. Further, in the following, descriptions of known structures and techniques are omitted so as not to unnecessarily obscure the concept of the present disclosure.

Those skilled in the art will appreciate that the term "exemplary" is used herein to mean "illustrative, " or "serving as an example, " and is not intended to imply that a particular embodiment is preferred over another or that a particular feature is essential. Likewise, the terms "first" , "second" , "third" , "fourth, " and similar terms, are used simply to distinguish one particular instance of an item or feature from another, and do not indicate a particular order or arrangement, unless the context clearly indicates otherwise. Further, the term "step, " as used herein, is meant to be synonymous with "operation" or "action. " Any description herein of a sequence of steps does not imply that these operations must be carried out in a particular order, or even that these operations are carried out in any order at all, unless the context or the details of the described operation clearly indicates otherwise.

Conditional language used herein, such as "can, " "might, " "may, " "e.g., " and the like, unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey that certain embodiments include, while other embodiments do not include, certain features, elements and/or states. Thus, such conditional language is not generally intended to imply that features, elements and/or states are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without author input or prompting, whether these features, elements and/or states are included or are to be performed in any particular embodiment. Also, the term "or" is used in its inclusive sense (and not in its exclusive sense) so that when used, for example, to connect a list of elements, the term "or" means one, some, or all of the elements in the list. Further, the term "each, " as used herein, in addition to having its ordinary meaning, can mean any subset of a set of elements to which the term "each" is applied.

The term "based on" is to be read as "based at least in part on. " The term "one embodiment" and "an embodiment" are to be read as "at least one embodiment. " The term "another embodiment" is to be read as "at least one other embodiment. " Other definitions, explicit and implicit, may be included below. In addition, language such as the phrase "at least one of X, Y and Z, " unless specifically stated otherwise, is to be understood with the context as used in general to convey that an item, term, etc. may be either X, Y, or Z, or a combination thereof.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limitation of example embodiments. As used herein, the singular forms "a" , "an" , and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" , "comprising" , "has" , "having" , "includes" and/or "including" , when used herein, specify the presence of stated features, elements, and/or components etc., but do not preclude the presence or addition of one or more other features, elements, components and/or combinations thereof. It will be also understood that the terms "connect (s) , " "connecting" , "connected" , etc. when used herein, just mean that there is an electrical or communicative connection between two elements and they can be connected either directly or indirectly, unless explicitly stated to the contrary.

Of course, the present disclosure may be carried out in other specific ways than those set forth herein without departing from the scope and essential characteristics of the disclosure. One or more of the specific processes discussed below may be carried out in any electronic device comprising one or more appropriately configured processing circuits, which may in some embodiments be embodied in one or more application-specific integrated circuits (ASICs) . In some embodiments, these processing circuits may comprise one or more microprocessors, microcontrollers, and/or digital signal processors programmed with appropriate software and/or firmware to carry out one or more of the operations described above, or variants thereof. In some embodiments, these processing circuits may comprise customized hardware to carry out one or more of the functions described above. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive.

Although multiple embodiments of the present disclosure will be illustrated in the accompanying Drawings and described in the following Detailed Description, it should be understood that the disclosure is not limited to the disclosed embodiments, but instead is also capable of numerous rearrangements, modifications, and substitutions without departing from the present disclosure that as will be set forth and defined within the claims.

Further, although the following description of some embodiments of the present disclosure is given in the context of 4G LTE (Long Term Evolution) , the present disclosure is not limited thereto. In fact, as long as handover failure cause classification is involved, the inventive concept of the present disclosure may be applicable to any appropriate communication architecture, for example, to Global System for Mobile Communications (GSM) /General Packet Radio Service (GPRS) , Enhanced Data Rates for GSM Evolution (EDGE) , Code Division Multiple Access (CDMA) , Wideband CDMA (WCDMA) , Time Division -Synchronous CDMA (TD-SCDMA) , CDMA2000, Worldwide Interoperability for Microwave Access (WiMAX) , Wireless Fidelity (Wi-Fi) , LTE-Advance (LTE-A) , or 5G New Radio (NR) , etc. Therefore, one skilled in the arts could readily understand that the terms used herein may also refer to their equivalents in any other infrastructure. For example, the term "User Equipment" or "UE" used herein may refer to a terminal device, a mobile device, a mobile terminal, a mobile station, a user device, a user terminal, a wireless device, a wireless terminal, or any other equivalents. For another example, the term "network node" used herein may refer to a network function, a network element, a RAN node, an OAM node, a testing network function, a transmission reception point (TRP) , a base station, a base transceiver station, an access point, a hot spot, a NodeB, an Evolved NodeB (eNB) , a gNB, or any other equivalents. Further, the term "electronic device" used herein may refer to any of above listed devices.

As mentioned above, the HO procedure is one of the most critical functions in a cellular network and plays a key role in maintaining seamless connectivity of UEs across multiple cells. Specifically, in an LTE system, the entire handover procedure is controlled by the network (e.g., eNB and Mobility Management Entity (MME) ) as shown in Fig. 1.

As shown in Fig. 1, an eNB 105-1 may listen to the quality of its radio environment and send measurement control to a UE 100, notifying the UE 100 that the UE 100 should send measurement reports when the specific conditions are satisfied. While the UE 100 moves (e.g., when the UE 100 moves from a source cell served by the source eNB 105-1 to a target cell served by a target eNB 105-2) and handover triggering conditions are satisfied, a handover procedure may start.

The procedure may include two main phases: a preparation phase (indicated by ①) and an execution phase (indicated by ②) . In the preparation phase, the target eNB 105-2 may synchronize with the source eNB 105-1 through MME 110 via S1 interfaces or synchronizes with the source eNB 105-1 directly via X2 interface for resource preparation. By the end of the preparation phase, a handover command may be delivered to the UE 100 and the execution phase starts. Upon reception of the command, the UE 100 may be disconnected from the source cell and handed over to the target cell according to the command.

As shown in Fig. 1, a handover failure may occur in the preparation phase and/or the execution phase due to many different causes, as indicated by the black "X" marks.

Root Cause Analysis (RCA) is a technique to identify the origin of such problems. RCA can be broken down to a series of steps to find the primary cause of the problem, so that one can determine what happened and why it happened. The RCA of handover failure is a very complex task in RAN because it is a complicated procedure and it involves a network of cells.

There are a lot of reasons that cause handover process fail. In a published patent application, WO 2017/025773 A1, more than forty possible causes for handover failures are listed. The reasons could be overload of the target cell, admission control by the target cell, transport link failure between the eNBs 105 and the MME 110, missed definition of neighbors, PCI confusion, radio link failure caused by too early/late handover or wrong handover, weak coverage, interference, etc.

Due to above reasons, RCA of handover performance degradation for a base station is a time consuming work, while the investigation scope is required to be quickly narrowed down to figure out and solve the root cause problem to improve the handover performance.

Further, some approaches are based on UE logs or network traces which record the signaling messages between UEs and base stations. These methods focus on the handover call flow for each UE and report failures once the abnormal execution of the call flow is detected.

Further, some approaches are based on performance measurements. There are different performance measurements in the preparation and execution phases, and they attribute the degradation problem to the handover failure cause when the handover degradation is detected.

However, the approaches based on UE logs or network traces do not fit to operators operating a large service network. One reason is that analysing the call flow for each UE is a labour intensive task. Another reason is that we are looking for the major cause contributor of the handover performance degradation problem, and outlier UE failures from majority failures with common reasons are taken as noises.

The cause analysis based on performance measurements are usually based on rules or decision trees. Operator needs to devise rules with thresholds to determine the major cause contributor.

Further, deep learning methods are applied in industry to learn the handover parameters in base stations to improve handover performance. For example, a method is proposed to use a deep learning method to learn the possible handover failure causes from performance measurements, but it only learns from data for a single base station, instead of a cluster of base stations.

However, for handover scenarios, clusters containing neighbour cells are important for cause analysis. Making rules against a varied number of cells is a challenge to the cause analyser. It is hard for a common deep learning method to handle an unseen neighbour cluster with different sizes and structures.

Attributing the handover performance degradation problem to major causes of the list is a multi-class classification problem. In this phase, the RCA procedure generally use finite state machines to do the attribution. However, similar to the decision tree, the threshold setting in these methods is not intelligent and is difficult to evolve.

Since there are many categories into which the causes are classified, it is difficult to build an intelligent model as it shall require much data to learn and it is easy to be underfitting. Therefore, an accurate and intelligent cause analysis model is needed, so that the system could take correct and timely remedy actions to fix the problem and improve the handover performance.

In some embodiments of the present disclosure, a handover failure cause classification solution based on performance measurements is proposed. According to the understanding of the handover procedure, in some embodiments, 4 handover degradation categories covering different causes may be defined. However, the present disclosure is not limited thereto. In some other embodiments, more than 4 categories, less than 4 categories, and/or 4 different categories than those described below may be provided.

In some embodiments, firstly, the solution may monitor the handover performance of the radio network and report the handover degradation cells ranked by importance. Then, it may use a Graph Neural Network (GNN) model to perform the graph classification task. In some embodiments, it may classify the graph of the handover degradation cluster to one of the degradation categories. In some embodiments, the accuracy of the classification could reach 90%according to the testing result. In some embodiments, the input graph of the classification model may be constructed by using neighbour relations of the degradation cell and building selected features for cells from performance measurement counters.

In some embodiments, after classification, the scope of the causes may be narrowed down, and corresponding handover failure diagnosis procedures with remedy actions may be applied at eNBs to fix the problem accordingly.

In general, several procedures are provided as building blocks to achieve the above goals.

(1) Handover Failure Graph Model Training

It is novel to transform the cause isolation of handover failure to a graph classification problem. Graph Neural Network may be used to do graph classification on Handover Failure Graphs. In some embodiments, a classification model may be learned to predict the probability of belonging to one of the 4 categories as follows.

1. Preparation denied

a) Definition: The target cell denied the handover preparation request

b) The root cause could be overload of target cell, missing license, admission control, etc.

2. Preparation failure due to no network response

a) Definition: No response received from network to the handover reparation request

b) The root cause could be transport link problem, missed MME configuration in source cell, missed definition of neighbours in source cell, core network issue, etc.

3. Execution failure due to UE missing

a) Definition: UE quits handover before attempting to access target cell during handover execution

b) The root cause could be too late handover, wrong handover due to PCI confusion, weak coverage or interference around source cell, etc.

4. Execution failure due to target cell access failure

a) Definition: UE meets problem in accessing target cell during handover execution

b) The root cause could be too early handover, wrong handover due to weak coverage or interference around target cell, target eNB internal problem, etc.

2) Handover Failure Graph Construction

This procedure may transform the handover failure issue to a graph of cells with features. The feature selection is critical to the classification performance. The pruning of graph is also important as the less active neighbour relations are noises to the classification problem.

3) Handover Failure Anomaly Detection

In this procedure, steps are devised to rank the handover failures to make the failures with severe impact outstanding.

With some embodiments of the present disclosure, a cause isolation method based on performance measurements in the RCA of handover performance degradation may be provided. The cause of handover performance degradation may be narrowed down to one of 4 categories and the speed to solve the problem may be improved as less diagnosis procedure is needed after classification.

Further, the cause isolation method of some embodiments of the present disclosure may outperform the rule-based methods. Traditional rules need decide KPI thresholds to attribute the problem to the root cause of handover failures. However, there are more than 40 possible causes, and the rules are very complicated. By contrast, with the cause isolation method of some embodiments of the present disclosure, the training of the model may only need human operators to label each problem to 4 kinds of category in the training set, which is much easier to be obtained.

As will be seen below, cross validation is used as the evaluation method and accuracy is used as the evaluation metric. In some embodiments, the accuracy may be calculated as the number of correct predictions divided by number of predictions. The method of some embodiments of the present disclosure may achieve high test accuracy with a mean value of 90%and a standard deviation of 6%. With this result, the solution may firstly attribute the handover degradation problem to one category, and then perform cause diagnosis inside the scope of the category. In this way, a significant reduction of time may be achieved in solving the handover degradation problem.

Fig. 4 is a diagram illustrating exemplary services for isolating HO failure causes according to an embodiment of the present disclosure. As shown in Fig. 4, the proposed solution may be deployed in an OAM platform 450 of a Radio Access Network as services. As shown in Fig. 4, there are two services, that is, "Handover Failure Remedy Service" 460 and "Handover Failure Graph Model Training Service" 470. However, the present disclosure is not limited thereto. In some other embodiments, one or both of the services may be deployed in other locations, separately or together. In some other embodiments, a single service may be distributed across multiple locations. For example, multiple modules in the handover failure remedy service 460 may be deployed at and executed by multiple physical devices in a coordinated manner.

Referring back to Fig. 1, at the OAM platform 450, performance measurements (PM) counters 401 may be collected for each eNB 405. In some embodiments, the PM counters 401 may have a resolution of 15-minute ROP or another ROP. For example, the OAM platform 450 may collect the PM counters 401 from eNBs 405 for every 15 minutes. For another example, the eNB 405 may store its PM counters 401 as log files, each having a 15-minute ROP, and the handover failure remedy service 460 may retrieve these log files once a day.

In some embodiments, the PM counters 401 could be pegged on cell relation between neighbour cells or pegged on cell. In some embodiments, the PM counters 401 may include at least one of:

a) Cell Relation PM counters, comprising at least one of:

- Count of Preparation Success;

- Count of Preparation Attempts;

- Count of Execution Success;

- Count of Preparation Rejects.

b) Cell PM counters, comprising at least one of:

- Count of Random-Access Success;

- Count of Random-Access Attempts;

- Count of Connection Setup Success;

- Count of Connection Setup Attempts;

- Number of Average Connected Users.

In some embodiments, the PM counters 401 may be collected and stored in the storage of the OAM platform 450.

In some embodiments, four handover degradation categories may be defined as:

1. Preparation denied

a) Definition: The target cell denied the handover preparation request,

2. Preparation failure due to no network response

a) Definition: No response received from network to the handover reparation request,

3. Execution failure due to UE missing

a) Definition: UE quits handover before attempting to access target cell during handover execution,

4. Execution failure due to target cell access failure

a) Definition: UE meets problem in accessing target cell during handover execution,

However, the present disclosure is not limited thereto. In some other embodiments, any number of categories can be defined, as long as it is appropriate for the GNN model.

In some embodiments, the main function of "Handover Failure Remedy Service" 460 may be provided as follows.

Firstly, it may use the module "Handover Failure Anomaly Detection" 461 to detect the Handover Failure Cell (s) and output the failure cell list, for example, based on the PM counters 401 collected from the eNBs 405.

Then, the "Handover Failure Graph Construction" module 463 may perform graph construction for each failure cell and output the graph with features to the "Handover Failure Graph Classification" module 465 for cause classification.

The "Handover Failure Graph Classification" module 465 may use the classification model to classify the Handover Failure graph to one of the four categories.

The "Handover Failure Remedy Action Executor" module 467 may correlate each specific handover remedy action to at least one failure category and it may apply or trigger the remedy actions 402 corresponding to the predicted category on eNBs.

In some embodiments, the main function of "Handover Failure Graph Model Training Service" 470 may be to train the Graph Neural Network to output the new classification model with better classification accuracy.

Fig. 5 is a diagram illustrating an exemplary handover failure anomaly detection module 461 according to an embodiment of the present disclosure. As shown in Fig. 5, the PM counters 401 collected from eNBs 405 may be input to the handover failure anomaly detection module 461.

In some embodiments, as the handover related PM counters are pegged on cell relation between neighbour cells, for anomaly detection, the counters from neighbour relations may be aggregated and then some handover KPIs may be calculated as below.

1) Cell Outgoing Handover Success Rate (OHSR)

2) Cell Outgoing Preparation Attempts (OPA)

OPA = Sum of Outgoing Preparation Attempts

In some embodiments, following anomaly rules may be used to decide whether an ROP is abnormal for a cell or not at block 510:

a. Cell Outgoing Handover Success Rate ＜ 50%; and/or

b. Cell Outgoing Preparation Attempts ＞ 100.

In some embodiments, at block 520, the number of abnormal ROPs in latest 30 days may be used to filter the cells. In some embodiments, the cells satisfying below rule may be filtered out:

c. Number of abnormal ROPs in latest 30 days ＞ 60.

In some embodiments, at block 530, an average deviation of anomaly time defined as below may be calculated:

d. Average deviation of anomaly time = average time difference from the anomaly time to the median of anomaly times.

At block 530, the failure cells may be sorted in ascending order by this average deviation of anomaly time firstly. The cells with lower values may be put forward. A lower value means that the anomalies are concentrated and the failure cell experienced abruptly emerging handover failures.

At block 540, PM counter (Number of Average Connected Users) in busy hour may be used to sort the cells in descending order. The cells with more users are put forward.

Finally, a ranked list of Handover Failure cells 501 may be output for further analysis.

In some embodiments, the thresholds (e.g., 50%, 100, 60, or the like) and/or numbers (e.g., latest 30 days, top 100, top 50, or the like) used in this module may be parameters configurable for different network operators. In some other embodiments, these thresholds and/or numbers may be configured with different values.

Fig. 6 is a diagram illustrating an exemplary handover failure graph construction module 463 according to an embodiment of the present disclosure. As shown in Fig. 6, the PM counters 401 collected from eNBs 405 may be input to the handover failure graph construction module 463. In some embodiments, only the PM counters 401 for the cells in the list 501 that is output from the handover failure anomaly detection module 461 may be input to the handover failure graph construction module 463. In other words, only the PM counters 401 for the cells in the list 501 are processed by the handover failure graph construction module 463.

In some embodiments, as the Handover PM counters 401 are pegged based on directed neighbour relations, the neighbours of a cell may be extracted from PM counters 401. As soon as a failure cell is received, its abnormal ROP with maximum handover outgoing preparation attempts may be selected out and all neighbours of the failure cell at this ROP may be extracted accordingly.

Now a set of cells containing the failure cells and its neighbours may be obtained at block 610. The relations from PM data between any of two cells may be checked and an edge for each pair may be drawn if they have a neighbour relation to construct a complete neighbour graph. For example, Fig. 2A shows an exemplary complete neighbour graph 20 generated for a failure cell 205-1.

At block 620, the graph may be pruned to cut the relations with outgoing preparation attempts less than 50 (or any other appropriate number) to de-noise the unimportant relations. For example, Fig. 2B shows an exemplary pruned graph 20′ for the failure cell 205-1. For the pruned graph, an edge table may be obtained. An exemplary edge table generated for the pruned graph 20′ shown in Fig. 2B is provided below:

From	To
Cell 1 (or its ID)	Cell 2 (or its ID)
Cell 1 (or its ID)	Cell 3 (or its ID)
Cell 1 (or its ID)	Cell 4 (or its ID)
Cell 3 (or its ID)	Cell 4 (or its ID)

In some embodiments, The content in the Edge Table may be cell names, or cell IDs, or anything that can distinguish one cell from another.

At block 630, features for each unique cell in the edge table may be calculated. In some embodiments, there are totally 6 features that can be used to describe a cell. They are:

1) PSRO (Cell Preparation Success Rate as Originator)

2) PSRT (Cell Preparation Success Rate as Terminator)

3) ESRO (Cell Execution Success Rate as Originator)

4) ESRT (Cell Execution Success Rate as Terminator)

5) PRRO (Cell Preparation Reject Rate as Originator)

6) ASE (Cell Access Successful Rate)

In some embodiments, one or more of the features may be used for generating a graph for the failure cell 205-1.

As shown in Fig. 6, a list of handover failure graph with corresponding edge tables and cell feature tables may be output from the handover failure graph construction module 463.

Fig. 7 is a diagram illustrating an exemplary handover failure graph classification module 465 according to an embodiment of the present disclosure. As shown in Fig. 7, the handover failure graph classification module 465 may comprise a "Handover Failure Classification Model" 700 that may accept the Handover Failure Graph 601 as input and output the predicted category 701.

In some embodiments, the classification model 700 may be a Graph Neural Network (GNN) Model. It may contain 4 components. They are:

1) Graph Convolution Layers 710:

- Component to consolidate neighbour feature and perform neural unit computing.

2) Sort Pooling Layers 720:

- Component to sort nodes and select a fixed number of nodes.

3) 1D Convolution Layers and Max Pooling Layers 730:

- Component to do convolution computing to extract hidden features.

4) Fully Connected Layer 740:

- Classification using hidden features.

However, the present disclosure is not limited thereto. In some other embodiments, the classification model 700 may comprise more layers, less layers, and/or different layers.

In some embodiments, the model output may be the probability that the input graph 601 belongs to each of the categories. In some embodiments, the predicted category 701 may be the one with the highest probability. An exemplary Handover Failure Classification model 300 is shown in Fig. 3.

As shown in Fig. 3, the Handover Failure Classification model 300 may comprise one or more layers, comprising at least one of: one or more graph convolution networks (GCNs) or graph convolution layers (GCLs) , a sort pooling layer, one or more 1D convolution layers and max pooling layers, and a fully connected layer. A graph (e.g., the

graph

20 or 20′ shown in Fig. 2A or Fig. 2B, respectively) may be input to the Handover Failure Classification model 300, and probabilities indicating how likely the input graph will be classified into corresponding categories, respectively, may be output by the model 300.

In some embodiments, the graph convolution layers may aggregate node information in local neighborhoods in the graph to extract local substructure information. To extract multi-scale substructure features, multiple graph convolution layers may be stacked. In some embodiments, after multiple graph convolution layers, a layer to concatenate the output may be added to form a concatenated output. In the concatenated output, each row can be regarded as a "feature descriptor" of a vertex, encoding its multi-scale local substructure information.

In some embodiments, after that, a sort pooling layer may be provided. The main function of the sort pooling layer may be to sort the feature descriptors, each of which represents a vertex, in a consistent order before feeding them into traditional 1-D convolutional and dense (or fully connected) layers. In some embodiments, after sort pooling, a tensor with each row representing a vertex and each column representing a feature channel may be obtained. Further, in some embodiments, a 1-D convolutional layer may be added, in order to sequentially apply filters on vertices′ feature descriptors. In some embodiments, after that, several Max Pooling layers and 1-D convolutional layers may be added in order to learn local patterns on the node sequence. Finally, a fully-connected layer followed by a softmax layer may be provided.

Fig. 8 is a diagram illustrating an exemplary handover failure graph classification model training service 470 according to an embodiment of the present disclosure. As shown in Fig. 8, a GNN model under training 800 may comprise at least one of one or more graph convolution layers 810, a sort pooling layer 820, one or more 1D convolution layers and max pooling layers 830, and a fully connected layer 840. These layers are substantially same to those shown in Fig. 3 and Fig. 7, and therefore a detailed description thereof is omitted for simplicity.

In some embodiments, the Handover Failure Graph Classification Model 800 may be a supervised learning model. In such a case, a handover failure graph 601 and its corresponding category label 801 are needed to train the model 800 as shown in Fig. 8. However, the present disclosure is not limited thereto. In some other embodiments, the model 800 may be an unsupervised learning model, and therefore no label for training is needed.

In some embodiments, the loss function 850 may calculate a difference between the prediction generated by the model 800 and the labels 801 during the GNN model training, and the difference may be considered as loss to be fed back to the GNN model 800 to update the parameters in the layers automatically.

In some embodiments, cross validation is used as the evaluation method and accuracy is used as the evaluation metric. The accuracy may be calculated as the number of correct predictions divided by the number of predictions. In such a case, the solution of some embodiments of the present disclosure may achieve a high test accuracy.

To be specific, in some embodiments, 110 handover failure graphs were built and labelled based on real world network PM data. The dataset was randomly split into a train set and a test set according to a ratio of 80 to 20, which means 88 graphs for training and 22 graphs for testing. Further, the random split was repeated for 20 times. In other words, training and evaluation were repeated for 20 times with the random split sets. The test accuracy is 90%± 6%as shown in (a) of Fig. 9. Further, a corresponding candlestick graph is shown in (b) of Fig. 9, which indicates that the best and worst accuracies are 96.59%and 84.09%, respectively and that the median accuracy is 90.34%and the high 25%percentile is 93.18%. In other words, the accuracy of the trained model outperforms the existing methods.

Fig. 10 is a diagram illustrating an exemplary overall procedure for handover failure cause classification according to an embodiment of the present disclosure. The procedure may be performed by the Handover Failure Remedy Service 460 shown in Fig. 4. In the Handover Failure Remedy Service 460, the eNB′s PM counters 401 may be periodically collected at ROPs (e.g., 15 mins) and stored in the PM data storage 1001.

As shown in Fig. 10, the procedure may comprise following steps. However, some of the steps may be performed in a different order than that shown in Fig. 10, and some of the steps may be omitted and additional steps may be added into the procedure.

At step S1010, the GNN model may be loaded into the by the handover failure graph classification module 465 for initialization.

At step S1020, required PM data for all cells may be pulled or otherwise obtained at every ROP by the handover failure anomaly detection module 461 to detect cells with handover degradation (or anomaly) .

At step S1030, a ranked list of cells with handover degradation cells may be output from the handover failure anomaly detection module 461 to the handover failure graph construction module 463.

At step S1040, required PM data (e.g., PM data for the cells indicated by the handover failure anomaly detection module 461) may be pulled or otherwise obtained by the handover failure graph construction module 463 for handover failure graph construction including graph pruning, feature engineering, or the like.

At step S1050, the generated handover failure graph may be stored, for example, in a handover failure graph storage 1003.

At step S1060, the handover failure graph may be classified by the handover failure graph classification module 465 to one of 4 (or any other appropriate number) defined failure categories.

At step S1070, the failure category may be mapped to a list of remedy actions, for example, by the handover failure remedy action executor 467.

At step S1080, the remedy actions on the eNB (s) may be performed or at least triggered by the handover failure remedy action executor 467.

With the embodiment described above, a cause isolation method based on performance measurements in the RCA of handover performance degradation is provided. The cause of handover performance degradation can be narrowed down to one of a limited number (e.g., 4) of categories and the speed to solve the problem is improved as less diagnosis procedure is needed after classification.

Fig. 11 is a diagram illustrating an exemplary overall procedure for training a handover failure graph model according to an embodiment of the present disclosure. The procedure may be performed by the Handover Failure Graph Model Training Service 470 shown in Fig. 4. The Graph classification model (e.g., the model 800 shown in Fig. 8 or the model 800 shown in Fig. 8) may be retrained periodically to improve the model.

As shown in Fig. 11, the procedure may comprise following steps. However, some of the steps may be performed in a different order than that shown in Fig. 11, and some of the steps may be omitted and additional steps may be added into the procedure.

At step S1110, a domain expert (or any other source, which could be human being or artificial intelligence (AI) ) may assign graph labels to the handover failure graph in the handover failure graph storage 1003. In some embodiments, the graph label may be one of 4 (or any other appropriate number) defined failure categories.

At step S1120, the handover failure graph and labels may be used to retrain the model, and measure the test accuracy.

At step S1130, the new GNN model may be stored into the model storage 1005, for subsequent use.

With the procedure described above, the classification model can be retrained based on the latest PM data, and therefore the performance of the model can be improved or at least maintained.

Fig. 12 is a flow chart of an exemplary method 1200 for facilitating a telecommunication network in reducing its HO failures according to an embodiment of the present disclosure. The method 1200 may be performed at an electronic device (e.g., the OAM platform 450) . The method 1200 may comprise steps S1210, S1220, and S1230. However, the present disclosure is not limited thereto. In some other embodiments, the method 1200 may comprise more steps, less steps, different steps, or any combination thereof. Further the steps of the method 1200 may be performed in a different order than that described herein when multiple steps are involved. Further, in some embodiments, a step in the method 1200 may be split into multiple sub-steps and performed by different entities, and/or multiple steps in the method 1200 may be combined into a single step.

The method 1200 may begin at step S1210 where an HO failure graph for a first cell may be determined at least based on PM data associated with the first cell.

At step S1220, a failure category for the HO failure graph may be determined by using a trained GNN model.

At an optional step S1230, one or more remedy actions corresponding to the determined failure category may be triggered to reduce HO failures associated with the first cell. In some embodiments, this step S1230 may be omitted.

In some embodiments, the step of determining the HO failure graph for the first cell may comprise: determining one or more neighbor cells for the first cell at least based on the PM data; and determining the HO failure graph such that the one or more neighbor cells and the first cell correspond to vertices of the HO failure graph, respectively, and an edge is present between any two vertices in the HO failure graph only when cells corresponding to the two vertices have a neighbor relation. In some embodiments, the PM data may be PM data during an abnormal ROP with the maximum HO outgoing preparation attempts.

In some embodiments, the step of determining the HO failure graph for the first cell may comprise: removing, from the HO failure graph, at least one edge between at least one pair of vertices, wherein for each of the at least one pair of vertices, a number of outgoing preparation attempts between corresponding cells may be less than a threshold. In some embodiments, one or more features may be associated with each vertex in the HO failure graph. In some embodiments, the one or more features may comprise at least one of: a Preparation Success Rate as Originator (PSRO) ; a Preparation Success Rate as Terminator (PSRT) ; an Execution Success Rate as Originator (ESRO) ; an Execution Success Rate as Terminator (ESRT) ; a Preparation Reject Rate as Originator (PRRO) ; and an Access Successful Rate (ASE) .

In some embodiments, at least one of following may be true:

-

and

-

In some embodiments, the trained GNN model may comprise at least one of: one or more graph convolution layers; one or more sort pooling layers; one or more 1D convolution layers and max pooling layers; and one or more fully connected layers. In some embodiments, the determined failure category for the HO failure graph may be one of multiple failure categories that has the highest probability that is determined by the trained GNN model based on the HO failure graph. In some embodiments, before the step of determining the HO failure graph for the first cell, the method 1200 may further comprise: determining whether there is HO failure anomaly for one or more cells comprising the first cell at least based on the PM data. In some embodiments, for at least one of the one or more cells, the step of determining whether there is HO failure anomaly may comprise at least one of: determining whether there is an abnormal ROP associated with one or more cells or not; determining which of the one or more cells has a number of abnormal ROPs greater than a threshold during a period of time, as first candidate cells; sorting the first candidate cells by their average deviations of anomaly time during a period of time in ascending order, and determining one or more of the first candidate cells with top ranks, as second candidate cells; and sorting the second candidate cells by their average busy period connected users during a period of time in descending order, and determining one or more of the second candidate cells with top ranks, as a list of HO failure cells.

In some embodiments, the step of determining whether there is an abnormal ROP associated with one or more cells or not may comprise at least one of: determining whether an outgoing HO success rate (OHSR) is lower than a threshold or not; and determining whether a number of outgoing preparation attempts (OPAs) is higher than a threshold or not.

In some embodiments, at least one of following may be true:

-

and

- OPA = Sum of Outgoing Preparation Attempts.

In some embodiments, an average deviation of anomaly time for a cell during the period of time may be determined as an average of time differences between all anomaly times associated with the cell during the period of time and the median of all anomaly times associated with the cell during the period of time. In some embodiments, an average busy period connected users during a period of time may be determined as a number of average connected users in a busy period during the period of time. In some embodiments, a peak ROP in a day associated with a cell may be an ROP during which the cell serves the greatest number of average connected users in the day, wherein a busy period associated with a cell may be an ROP that has the greatest number of days, in each of which the ROP is the peek ROP, during the period of time. In some embodiments, at least one cell in the list of HO failure cells may be determined as the first cell. In some embodiments, the PM data may comprise at least one of: a count of preparation success; a count of preparation attempts; a count of execution success; a count of preparation rejects; a count of random access success; a count of random access attempts; a count of connection setup success; a count of connection setup attempts; a number of average connected users. In some embodiments, the method 1200 may be performed at an OAM platform.

In some embodiments, the determined failure category may be at least one of: Preparation Denied; Preparation Failure due to no network response; Execution failure due to UE missing; and Execution failure due to target cell access failure.

Fig. 13 schematically shows an embodiment of an arrangement 1300 which may be used in an electronic device (e.g., the OAM platform 450) according to an embodiment of the present disclosure. Comprised in the arrangement 1300 are a processing unit 1306, e.g., with a Digital Signal Processor (DSP) or a Central Processing Unit (CPU) . The processing unit 1306 may be a single unit or a plurality of units to perform different actions of procedures described herein. The arrangement 1300 may also comprise an input unit 1302 for receiving signals from other entities, and an output unit 1304 for providing signal (s) to other entities. The input unit 1302 and the output unit 1304 may be arranged as an integrated entity or as separate entities.

Furthermore, the arrangement 1300 may comprise at least one computer program product 1308 in the form of a non-volatile or volatile memory, e.g., an Electrically Erasable Programmable Read-Only Memory (EEPROM) , a flash memory and/or a hard drive. The computer program product 1308 comprises a computer program 1310, which comprises code/computer readable instructions, which when executed by the processing unit 1306 in the arrangement 1300 causes the arrangement 1300 and/or the electronic device in which it is comprised to perform the actions, e.g., of the procedure described earlier in conjunction with Fig. 3 through Fig. 8 and Fig. 10 through Fig. 12 or any other variant.

The computer program 1310 may be configured as a computer program code structured in

computer program modules

1310A, 1310B, and 1310C. Hence, in an exemplifying embodiment when the arrangement 1300 is used in an electronic device for facilitating a telecommunication network in reducing its HO failures, the code in the computer program of the arrangement 1300 includes: a module 1310A configured to determine an HO failure graph for a first cell at least based on PM data associated with the first cell; a module 1310B configured to determine a failure category for the HO failure graph by using a trained GNN model; and an optional module 1310C configured to trigger one or more remedy actions corresponding to the determined failure category to reduce HO failures associated with the first cell.

The computer program modules could essentially perform the actions of the flow illustrated in Fig. 3 through Fig. 8 and Fig. 10 through Fig. 12, to emulate the electronic device. In other words, when the different computer program modules are executed in the processing unit 1306, they may correspond to different modules in the electronic device.

Although the code means in the embodiments disclosed above in conjunction with Fig. 13 are implemented as computer program modules which when executed in the processing unit causes the arrangement to perform the actions described above in conjunction with the figures mentioned above, at least one of the code means may in alternative embodiments be implemented at least partly as hardware circuits.

The processor may be a single CPU (Central processing unit) , but could also comprise two or more processing units. For example, the processor may include general purpose microprocessors; instruction set processors and/or related chips sets and/or special purpose microprocessors such as Application Specific Integrated Circuit (ASICs) . The processor may also comprise board memory for caching purposes. The computer program may be carried by a computer program product connected to the processor. The computer program product may comprise a computer readable medium on which the computer program is stored. For example, the computer program product may be a flash memory, a Random-access memory (RAM) , a Read-Only Memory (ROM) , or an EEPROM, and the computer program modules described above could in alternative embodiments be distributed on different computer program products in the form of memories within the electronic device.

Correspondingly to the method 1200 as described above, an electronic device for facilitating a telecommunication network in reducing its HO failures is provided. Fig. 14 is a block diagram of an exemplary electronic device 1400 according to an embodiment of the present disclosure. The electronic device 1400 may be, e.g., the OAM platform 450 in some embodiments.

The electronic device 1400 may be configured to perform the method 1200 as described above in connection with Fig. 12. As shown in Fig. 14, the electronic device 1400 may comprise a first determining module 1410 configured to determine an HO failure graph for a first cell at least based on PM data associated with the first cell; a second determining module 1420 configured to determine a failure category for the HO failure graph by using a trained GNN model; and a triggering module 1430 configured to trigger one or more remedy actions corresponding to the determined failure category to reduce HO failures associated with the first cell.

The

above modules

1410, 1420, and/or 1430 may be implemented as a pure hardware solution or as a combination of software and hardware, e.g., by one or more of: a processor or a micro-processor and adequate software and memory for storing of the software, a Programmable Logic Device (PLD) or other electronic component (s) or processing circuitry configured to perform the actions described above, and illustrated, e.g., in Fig. 12. Further, the electronic device 1400 may comprise one or more further modules, each of which may perform any of the steps of the method 1200 described with reference to Fig. 12.

The present disclosure is described above with reference to the embodiments thereof. However, those embodiments are provided just for illustrative purpose, rather than limiting the present disclosure. The scope of the disclosure is defined by the attached claims as well as equivalents thereof. Those skilled in the art can make various alternations and modifications without departing from the scope of the disclosure, which all fall into the scope of the disclosure.

Abbreviation Explanation

eNB Evolved Node B

GNN Graph Neural Network

HO Handover

MME Mobility Management Entity

OAM Operation Administration Maintenance

PCI Physical Cell Identity

PM Performance Measurement

ROP Recording Output Period

RRC Radio Resource Control

UE User Equipment

Claims

A method (1200) for facilitating a telecommunication network (10) in reducing its handover (HO) failures, the method (1200) comprising:

determining (S1210) an HO failure graph for a first cell (205-1) at least based on performance measurement (PM) data associated with the first cell (205-1) ;

determining (S1220) a failure category for the HO failure graph by using a trained Graph Neural Network (GNN) model (700) ; and

triggering (S1230) one or more remedy actions corresponding to the determined failure category to reduce HO failures associated with the first cell.
The method (1200) of claim 1, wherein the step of determining (S1210) the HO failure graph for the first cell comprises:

determining one or more neighbor cells for the first cell at least based on the PM data; and

determining the HO failure graph such that the one or more neighbor cells and the first cell correspond to vertices of the HO failure graph, respectively, and an edge is present between any two vertices in the HO failure graph only when cells corresponding to the two vertices have a neighbor relation.
The method (1200) of claim 1 or 2, wherein the PM data is PM data during an abnormal Recording Output Period (ROP) with the maximum HO outgoing preparation attempts.
The method (1200) of any of claims 1 to 3, wherein the step of determining (S1210) the HO failure graph for the first cell comprises:

removing, from the HO failure graph, at least one edge between at least one pair of vertices,

wherein for each of the at least one pair of vertices, a number of outgoing preparation attempts between corresponding cells is less than a threshold.
The method (1200) of any of claims 1 to 4, wherein one or more features are associated with each vertex in the HO failure graph.
The method (1200) of claim 5, wherein the one or more features comprise at least one of:

- a Preparation Success Rate as Originator (PSRO) ;

- a Preparation Success Rate as Terminator (PSRT) ;

- an Execution Success Rate as Originator (ESRO) ;

- an Execution Success Rate as Terminator (ESRT) ;

- a Preparation Reject Rate as Originator (PRRO) ; and

- an Access Successful Rate (ASE) .
The method (1200) of claim 6, wherein at least one of following is true:

-

-

-

-

-
and

-
The method (1200) of any of claims 1 to 7, wherein the trained GNN model (700) comprises at least one of:

- one or more graph convolution layers;

- one or more sort pooling layers;

- one or more 1D convolution layers and max pooling layers; and

- one or more fully connected layers.
The method (1200) of any of claims 1 to 8, wherein the determined failure category for the HO failure graph is one of multiple failure categories that has the highest probability that is determined by the trained GNN model (700) based on the HO failure graph.
The method (1200) of any of claims 1 to 9, wherein before the step of determining (S1210) the HO failure graph for the first cell, the method (1200) further comprises:

determining whether there is HO failure anomaly for one or more cells comprising the first cell at least based on the PM data.
The method (1200) of claim 10, wherein for at least one of the one or more cells, the step of determining whether there is HO failure anomaly comprises at least one of:

determining whether there is an abnormal ROP associated with one or more cells or not;

determining which of the one or more cells has a number of abnormal ROPs greater than a threshold during a period of time, as first candidate cells;

sorting the first candidate cells by their average deviations of anomaly time during a period of time in ascending order, and determining one or more of the first candidate cells with top ranks, as second candidate cells; and

sorting the second candidate cells by their average busy period connected users during a period of time in descending order, and determining one or more of the second candidate cells with top ranks, as a list of HO failure cells.
The method (1200) of claim 11, wherein the step of determining whether there is an abnormal ROP associated with one or more cells or not comprises at least one of:

determining whether an outgoing HO success rate (OHSR) is lower than a threshold or not; and

determining whether a number of outgoing preparation attempts (OPAs) is higher than a threshold or not.
The method (1200) of claim 12, wherein at least one of following is true:

-
and

- OPA = Sum of Outgoing Preparation Attempts.
The method (1200) of any of claims 11 to 13, wherein an average deviation of anomaly time for a cell during the period of time is determined as an average of time differences between all anomaly times associated with the cell during the period of time and the median of all anomaly times associated with the cell during the period of time.
The method (1200) of any of claims 11 to 14, wherein an average busy period connected users during a period of time is determined as a number of average connected users in a busy period during the period of time.
The method (1200) of claim 15, wherein a peak ROP in a day associated with a cell is an ROP during which the cell serves the greatest number of average connected users in the day,

wherein a busy period associated with a cell is an ROP that has the greatest number of days, in each of which the ROP is the peek ROP, during the period of time.
The method (1200) of any of claims 11 to 16, wherein at least one cell in the list of HO failure cells is determined as the first cell.
The method (1200) of any of claims 1 to 17, wherein the PM data comprises at least one of:

- a count of preparation success;

- a count of preparation attempts;

- a count of execution success;

- a count of preparation rejects;

- a count of random access success;

- a count of random access attempts;

- a count of connection setup success;

- a count of connection setup attempts;

- a number of average connected users.
The method (1200) of any of claims 1 to 18, wherein the method (1200) is performed at an Operation, Administration, &Maintenance (OAM) platform (450) .
The method (1200) of any of claims 1 to 19, wherein the determined failure category is at least one of:

- Preparation Denied;

- Preparation Failure due to no network response;

- Execution failure due to UE missing; and

- Execution failure due to target cell access failure.
An electronic device (450, 1300, 1400) , comprising:

a processor (1306) ;

a memory (1308) storing instructions which, when executed by the processor (1306) , cause the processor (1306) to perform the method (1200) of any of claims 1 to 20.
A computer program (1310) comprising instructions which, when executed by at least one processor (1306) , cause the at least one processor (1306) to carry out the method (1200) of any of claims 1 to 20.
A carrier (1308) containing the computer program (1310) of claim 22, wherein the carrier (1308) is one of an electronic signal, optical signal, radio signal, or computer readable storage medium.