WO2023241320A1

WO2023241320A1 - Method and device for determining indicators for network optimization function of autonomous system

Info

Publication number: WO2023241320A1
Application number: PCT/CN2023/095790
Authority: WO
Inventors: 许瑞岳; 石小丽
Original assignee: 华为技术有限公司
Priority date: 2022-06-16
Filing date: 2023-05-23
Publication date: 2023-12-21
Also published as: CN117295080A

Abstract

The present application provides a method and device for determining indicators for the network optimization function of an autonomous system. The method comprises: acquiring first data of an autonomous system, the first data comprising network performance statistical data and/or network optimization process data; determining a first indicator of the autonomous system according to the first data, wherein the first indicator is used for representing the performance of the network optimization function of the autonomous system, and the performance of the network optimization function at least comprises the optimization status of the network performance of the autonomous system and/or the optimization efficiency of the network optimization function; and determining, according to the first indicator, the performance of the network optimization function of the autonomous system, or sending the first indicator to a second device. In this way, the indicator for representing the optimization effect of the network optimization function is used to accurately evaluate the influence of the network optimization function on the network performance of the autonomous system, thus enabling the reasonable deployment of various autonomous systems in which different autonomous technologies are introduced in the network optimization function.

Description

Indicator determination method and device for network optimization function of autonomous system

This application requires the priority of the Chinese patent application submitted to the State Intellectual Property Office on June 16, 2022, with the application number 202210689135.8 and the application title "Method and device for determining indicators of network optimization function of autonomous systems", and its entire content has been approved This reference is incorporated into this application.

Technical field

The present application relates to the field of communication technology, and in particular to a method and device for determining indicators of the network optimization function of an autonomous system.

Background technique

In the life cycle of a telecommunications network, network optimization is a key step committed to ensuring network performance and improving user experience.

At present, in the function of network optimization, autonomous technologies are often introduced, such as artificial intelligence (AI), machine learning (ML), big data analysis, etc., to find suitable network parameter combinations (including network links ( Multi-standard, multi-band and device diversity) and thousands of network parameters) can reduce manual operations, improve operation and maintenance efficiency, and solve network operation and maintenance problems.

In summary, different telecommunications networks introduce different autonomous technologies, which can improve the efficiency of network optimization.

Therefore, how to deploy telecommunications networks that introduce autonomous technology into the network optimization function is an urgent problem that needs to be solved.

Contents of the invention

This application provides a method and device for determining indicators of the network optimization function of an autonomous system, which can fully evaluate the optimization effect of the network optimization function with the help of indicators, and rationally deploy various autonomous systems that introduce different autonomous technologies into the network optimization function.

In the first aspect, this application provides a method for determining indicators of the network optimization function of an autonomous system, including:

Obtain the first data of the autonomous system. The first data includes: network performance statistics and/or network optimization process data;

According to the first data, the first indicator of the autonomous system is determined. The first indicator is used to characterize the performance of the network optimization function of the autonomous system. The performance of the network optimization function at least includes: the optimization of the network performance of the autonomous system and/or the network optimization function. optimization efficiency;

According to the first indicator, determine the performance of the network optimization function of the autonomous system;

Alternatively, send the first indicator to the second device.

Through the method provided in the first aspect, the first device obtains the first data of the autonomous system. The first data includes: network performance statistical data and/or network optimization process data, thereby comprehensively covering the execution of the autonomous system through the first data of the autonomous system. Data before and after the network optimization function is executed. The first device can determine the first indicator of the autonomous system based on the first data. The first indicator is used to characterize the performance of the network optimization function of the autonomous system. The performance of the network optimization function at least includes: the optimization of the network performance of the autonomous system and/ or network optimization function Optimize efficiency. The first device can determine the performance of the network optimization function of the autonomous system based on the first indicator, so that the first device can estimate the impact of the network optimization function on the autonomous system, or the first device sends the first indicator to the second device , so that the second device determines the performance of the network optimization function of the autonomous system based on the first indicator, so that the second device can estimate the impact of the network optimization function on the autonomous system.

Therefore, the first device or the second device can use the indicator used to characterize the performance of the network optimization function of the autonomous system, that is, the above-mentioned first indicator of the autonomous system, as an evaluation dimension of the autonomy capability of the network optimization function of the autonomous system, Able to quantitatively evaluate the impact/optimization effect of the network optimization function of the autonomous system on the network performance of the autonomous system (such as from multiple perspectives of quality performance and/or efficiency performance), avoiding the need for network optimization of telecommunications networks in related technologies The autonomy capability of functions only represents the limitations of the degree of automation and helps to rationally deploy autonomous systems that introduce different autonomy technologies into network optimization functions.

In a possible design, the optimization of the network performance of the autonomous system includes: the network performance after the autonomous system performs the network optimization function, the improvement compared with the network performance before the autonomous system performs the network optimization function, and the performance of the autonomous system Specific situation of network performance after network optimization function.

In one possible design, the method also includes:

Determine a second indicator of the autonomous system based on the first indicator, and the second indicator is used to indicate whether the performance of the network optimization function of the autonomous system meets the preset conditions;

According to the first indicator, the performance of the network optimization function of the autonomous system is determined, including:

Based on the first indicator and the second indicator, the performance of the network optimization function of the autonomous system is determined.

In one possible design, second indicators of autonomous systems are determined, including:

Determine the minimum value or weighted value of the first indicator based on the preset indicator benchmark;

Determine the minimum value or weighted value as the second indicator.

In one possible design, the method also includes:

Send the second indicator to the second device.

In a possible design, the network performance statistics include at least one of: coverage performance data, capacity performance data, rate performance data, delay performance data, poor quality data, or poor experience data. The poor quality data includes user experience data. The data used to represent whether the network performance of the autonomous system meets the preset performance quality standards, the experience difference data includes data used to represent whether the network performance of the autonomous system meets the preset user experience standards;

The network optimization process data includes: the start time of the network optimization function, the end time of the network optimization function, the scope of the network optimization function, the number of cells or grids or users that need to perform the network optimization function, and the cells or grids or users where the root cause is located. The number, or at least one of the number of cells or grids generated after performing the network optimization function, or the number of users.

In a possible design, the first indicator includes: quality performance indicators and/or efficiency performance indicators. The quality performance indicators are used to characterize the optimization of network performance of the autonomous system, and the efficiency performance indicators are used to characterize the autonomous system. The optimization efficiency of the network optimization function.

In one possible design, the quality performance index is determined based on network performance statistics; the efficiency performance index is determined based on network optimization process data.

In a possible design, the quality performance indicators include at least one of: poor quality optimization rate, poor experience optimization rate, poor quality performance, or poor experience performance. The poor quality optimization rate is used to characterize the network optimization performed by the autonomous system. The network performance after the function, the quality improvement compared with the network performance before the autonomous system performs the network optimization function, The experience difference optimization rate is used to characterize the network performance after the autonomous system performs the network optimization function, and the user experience improvement compared with the network performance before the autonomous system performs the network optimization function. The poor quality performance is used to characterize the network performance after the autonomous system performs the network optimization function. Whether the network performance reaches the quality standard, and poor experience performance is used to characterize whether the network performance of the autonomous system reaches the user experience standard after executing the network optimization function;

Efficiency performance indicators include: at least one of the optimization duration of the network optimization function, the optimization scope of the network optimization function, the root cause positioning ratio of the network optimization function, or the optimization ratio of the cells or grids or the number of users generated after the network optimization function is executed. One item.

In a possible design, the poor quality optimization rate includes: at least one of weak coverage optimization rate, high load optimization rate, low rate optimization rate, or low delay optimization rate;

The optimization rate of poor experience includes: at least one of the optimization rate of low-rate users or the optimization rate of low-latency users;

Poor performance includes: at least one of: coverage performance after executing the network optimization function, capacity performance after executing the network optimization function, rate performance after executing the network optimization function, or delay performance after executing the network optimization function;

Poor performance experience includes: the proportion of low-speed users after executing the network optimization function, the proportion of high-latency users after executing the network optimization function, the proportion of users with standard speed after executing the network optimization function, or the proportion of users with high-latency after executing the network optimization function. At least one of the proportion of users reaching the target.

In the second aspect, this application provides a method for determining indicators of the network optimization function of an autonomous system, including:

The first indicator of the autonomous system is received from the first device. The first indicator is used to characterize the performance of the network optimization function of the autonomous system. The performance of the network optimization function at least includes: the optimization of the network performance of the autonomous system and/or the performance of the network optimization function. Optimize efficiency;

Based on the first indicator, the performance of the network optimization function of the autonomous system is determined.

In one possible design, the method also includes:

Determine the minimum value or weighted value as the second indicator.

In one possible design, the method also includes:

Obtaining the second indicator from the first device;

In a possible design, the network performance statistics include at least one of: coverage performance data, capacity performance data, rate performance data, delay performance data, poor quality data, or poor experience data. The poor quality data includes user experience data. Data used to characterize whether the network performance of autonomous systems meets preset performance quality standards, poor experience data Includes data used to characterize whether the network performance of the autonomous system meets preset user experience standards;

In a possible design, the quality performance indicators include at least one of: poor quality optimization rate, poor experience optimization rate, poor quality performance, or poor experience performance. The poor quality optimization rate is used to characterize the network optimization performed by the autonomous system. The network performance after the function is compared with the network performance before the autonomous system performs the network optimization function. The experience difference optimization rate is used to characterize the network performance after the autonomous system performs the network optimization function. It is compared with the network performance before the autonomous system performs the network optimization function. The user experience improvement compared with the network performance. Poor quality performance is used to characterize whether the network performance of the autonomous system reaches the quality standard after executing the network optimization function. Poor experience performance is used to characterize whether the network performance of the autonomous system reaches the quality standard after executing the network optimization function. User experience standards;

In the third aspect, this application provides a method for determining indicators of the network optimization function of an autonomous system, which is applied to a system for determining indicators of the network optimization function of an autonomous system. The system includes: a first device and a second device; the method includes:

The first device acquires first data of the autonomous system, where the first data includes: network performance statistical data and/or network optimization process data;

The first device determines the first indicator of the autonomous system based on the first data. The first indicator is used to characterize the performance of the network optimization function of the autonomous system. The performance of the network optimization function at least includes: the optimization of the network performance of the autonomous system and/or Optimization efficiency of network optimization function;

The first device sends the first indicator to the second device;

The second device determines the performance of the network optimization function of the autonomous system based on the first indicator.

In one possible design, the method also includes:

The first device determines the second indicator of the autonomous system based on the first indicator, and the second indicator is used to indicate whether the performance of the network optimization function of the autonomous system meets the preset conditions;

The first device sends the second indicator to the second device;

The second device determines the performance of the network optimization function of the autonomous system based on the first indicator, including:

The second device determines the performance of the network optimization function of the autonomous system based on the first indicator and the second indicator.

In one possible design, the method also includes:

The second device determines the second indicator of the autonomous system based on the first indicator. The second indicator is used to indicate whether the performance of the network optimization function of the autonomous system meets the preset conditions;

In one possible design, the second indicator of the autonomous system is determined based on the first indicator, including:

Determine the minimum value or weighted value as the second indicator.

In one possible design, quality class performance indicators are determined based on network performance statistics;

Efficiency performance indicators are determined based on network optimization process data.

Efficiency performance indicators include: the optimization time of the network optimization function, the optimization scope of the network optimization function, the root cause positioning ratio of the network optimization function, or the number of cells or grids or users generated after executing the network optimization function. At least one of the optimization ratios.

The beneficial effects of the method provided in any of the above aspects and possible designs of this aspect can be referred to the beneficial effects brought by the above-mentioned first aspect and each possible implementation of the first aspect, and will not be described again here.

In the fourth aspect, the present application provides an indicator determination device for the network optimization function of an autonomous system, applied to a first device. The device includes: a method for executing the first aspect and any possible design method of the first aspect. module.

In a fifth aspect, this application provides an indicator determination device for the network optimization function of an autonomous system, which is applied to a second device. The device includes: a method for executing the second aspect and any possible design method of the second aspect. module.

In a sixth aspect, this application provides an indicator determination system for the network optimization function of an autonomous system, including: an autonomous system, and indicators for realizing the network optimization function of the autonomous system in any of the possible designs of the fourth aspect and the fourth aspect. Determining device; or, an autonomous system, an indicator determining device for realizing the network optimization function of the autonomous system in any one of the fourth aspect and any possible design of the fourth aspect, and any one of the fifth aspect and any possible design of the fifth aspect. An indicator determination device for a network optimization function of an autonomous system.

In a seventh aspect, this application provides an indicator determination system for the network optimization function of an autonomous system, including: implementing the third aspect and a first device in any possible design of the third aspect; and implementing the third aspect and the third aspect. second device in either possible design.

In an eighth aspect, the present application provides a communication device, including: a memory and a processor; the memory is used to store program instructions; the processor is used to call the program instructions in the memory so that the communication device executes the first aspect and any possibility of the first aspect. The method in the design; and/or, perform the second aspect and any possible design method in the second aspect.

In a ninth aspect, the application provides a chip, including: an interface circuit and a logic circuit. The interface circuit is used to receive signals from other chips other than the chip and transmit them to the logic circuit, or to send signals from the logic circuit to the chip. Other chips and logic circuits are used to implement the first aspect and any possible design method of the first aspect; and/or, implement the second aspect and any possible design method of the second aspect.

In a tenth aspect, the present application provides a computer-readable storage medium on which a computer program is stored. The computer program is caused by a processor to implement the method in the first aspect and any possible design of the first aspect when executed by the communication device; And/or, implement the second aspect and any possible design method of the second aspect.

In an eleventh aspect, the present application provides a computer program product, including: execution instructions, the execution instructions are stored in a readable storage medium, and at least one processor of the communication device can read the execution instructions from the readable storage medium, At least one processor executes instructions so that the communication device implements the first aspect and the method in any possible design of the first aspect; and/or implements the second aspect and the method in any possible design of the second aspect.

Description of the drawings

Figure 1 is a schematic architectural diagram of an autonomous system provided by an embodiment of the present application;

Figure 2 is a schematic structural diagram of a first device provided by an embodiment of the present application;

Figure 3 is a signaling interaction diagram of a method for determining indicators of the network optimization function of an autonomous system provided by an embodiment of the present application;

Figure 4 is a signaling interaction diagram of a method for determining indicators of the network optimization function of an autonomous system provided by an embodiment of the present application;

Figure 5 is a schematic structural diagram of an indicator determination device for the network optimization function of an autonomous system provided by an embodiment of the present application;

Figure 6 is a schematic structural diagram of an indicator determination device for the network optimization function of an autonomous system provided by an embodiment of the present application;

Figure 7 is a schematic diagram of the hardware structure of a communication device provided by an embodiment of the present application;

FIG. 8 is a schematic diagram of the hardware structure of a communication device provided by an embodiment of the present application.

Detailed ways

In this application, "at least one" refers to one or more, and "plurality" refers to two or more. "And/or" describes the association of associated objects, indicating that there can be three relationships, for example, A and/or B, which can mean: A exists alone, A and B exist simultaneously, and B exists alone, where A, B can be singular or plural. The character "/" generally indicates that the related objects are in an "or" relationship. "At least one of the following" or similar expressions thereof refers to any combination of these items, including any combination of a single item (items) or a plurality of items (items). For example, at least one of a alone, b alone, or c alone can mean: a alone, b alone, c alone, a combination of a and b, a combination of a and c, a combination of b and c, or a combination of a, b and c, where a, b, c can be single or multiple. In addition, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. The terms "center", "longitudinal", "horizontal", "upper", "lower", "left", "right", "front", "back", etc. indicate the orientation or positional relationship based on that shown in the drawings. The orientation or positional relationship is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present application.

Different telecommunications networks introduce different autonomous technologies to improve the efficiency of network optimization. Based on this, operators need to analyze the autonomy technology introduced by the network optimization function of different telecom networks, and evaluate the autonomy capabilities (also called autonomous network levels) of the network optimization function of different telecom networks, which can reduce Operating expenses (OPEX).

In related technologies, for various telecommunications networks, the degree of human and network participation in each task link involved in the network optimization function is usually used to evaluate the autonomy capabilities of the network optimization function of different telecommunications networks. .

For example, for any telecommunications network, it is assumed that all tasks involved in the network optimization function include: performance anomaly identification, root cause analysis of performance problems, and optimization solution generation.

Then, based on the degree of manual and network participation in the above three task links (including manual participation, manual and network participation, and network participation), the autonomy capability of the network optimization function of the telecommunications network can be evaluated. Can It can be seen that the stronger the automation degree of the network optimization function of the telecommunications network, the higher the automation level of the telecommunications network.

Based on the above description, it can be seen that the network optimization function of the telecommunications network only represents the degree of automation of the telecommunications network. It is impossible to evaluate the effect and value brought by the degree of automation to the telecommunications network. It may even happen that the higher the automation level of the telecommunications network, the higher the automation level of the telecommunications network. The network performance of the network is getting worse.

For example, when the network optimization function is used to optimize coverage performance, the automation level of telecommunications network 1 is higher than that of telecommunications network 2, but it is possible that the coverage performance of telecommunications network 2 is better than that of telecommunications network 2. Coverage performance.

It can be seen that compared with telecommunications network 1, telecommunications network 2 improves the degree of automation at the expense of reducing network performance.

For another example, when the network optimization function is used to optimize rate performance, the automation level of telecommunications network 1 is equal to the automation level of telecommunications network 2, but it is possible that the rate performance of telecommunications network 2 is improved by 20%. The rate performance of Telecom Network 2 has been improved by 5%.

It can be seen that although the automation levels of telecommunications network 1 and telecommunications network 2 are the same, the network performance of telecommunications network 1 and telecommunications network 2 is different.

In summary, the autonomous capability of the network optimization function cannot reflect the impact of the network optimization function on the telecommunications network.

In order to solve the above problems, this application provides an indicator determination method, device, system, chip, computer-readable storage medium and computer program product for the network optimization function of an autonomous system, which can introduce indicators used to characterize the performance of the network optimization function, To measure the impact of introducing autonomous technology into the network optimization function of autonomous systems, it solves the limitation in related technologies that the autonomy capability of the network optimization function of telecommunications networks only represents the degree of automation.

First, some terms used in this application are explained below to facilitate understanding by those skilled in the art.

1. Autonomous system: It can be understood as a telecommunications system that introduces autonomous technology into the network optimization function. The telecommunications system (including telecommunications network and its operation management system) uses autonomous capabilities with as little human intervention as possible. Realize self-management and control (telecommunication system (including management system and network) with autonomy capabilities which is able to be governed by itself with minimal to no human intervention). In addition, autonomous systems can also be called autonomous networks.

2. Autonomous network Level: Used to characterize the level of autonomy of an autonomous system (escribes the level of autonomous capabilities in the autonomous network).

3. Network optimization function: In order to improve network performance or communication service experience, the process of monitoring and analyzing network performance and other related information, and taking performance optimization measures such as adjusting network resources and parameter configurations. The end-to-end workflow of the network optimization function includes the following task processes, which can be automatically performed by the telecommunications system:

Monitoring rules and optimization strategy generation: Generate network monitoring rules (including monitoring areas, monitoring objects, etc.) and optimization strategies (such as optimization means priority and combination strategies, etc.) based on network intentions.

Network assurance intent assessment: After the optimization plan is executed, the execution results are verified and confirmed, and the satisfaction of the intent is evaluated.

Data collection: Collect performance data, configuration data, environmental data and mobile communication system external data according to monitoring rules Internal data (such as Internet business data), etc.

Performance anomaly identification: Process the obtained real-time performance data (such as data correlation) to find performance anomalies, such as finding that it affects the user experience or that resource usage is unreasonable.

Performance degradation prediction: Based on the obtained real-time performance data and historical performance data, predict network performance and other development trends, and predict network performance/resource utilization and other development trends in a certain period of time, which helps to discover potential anomalies that affect network performance in advance.

Delimitation of performance problems: Conduct delimitation analysis of discovered performance anomalies and/or potential anomalies, and identify performance problem categories (such as weak coverage, low rate, high load, etc.).

Root cause analysis of performance problems: Conduct in-depth analysis of discovered performance anomalies and potential anomalies, identify the specific causes of performance problems, and support the generation of optimization solutions.

Optimization plan generation: Based on the performance root cause analysis results, optimization algorithms and artificial intelligence technology are used to provide several alternative plans for performance optimization.

Optimization plan evaluation and determination: Comprehensive evaluation of alternative plans (such as whether it affects customer experience, whether the adjustment plan meets the optimization goals, whether the adjustment cost is acceptable, etc.) and determines the plan that needs to be implemented.

Optimization plan execution: Execute tuning actions according to the optimized plan after decision-making, such as issuing network parameter configuration actions.

4. Indicator, also called key effective indicator (KEI): used to evaluate the effect and value of autonomy capabilities.

5. Grid: The grid structure is a spatial data structure that divides the earth's surface into a grid array of uniform sizes and closely adjacent each other. Each grid is a pixel or pixel consisting of rows and columns. Definition, and contains a code representing the attribute type or magnitude of the pixel, or simply a pointer to its attribute record. Therefore, the grid structure is a data organization that represents the distribution of spatial objects or phenomena in a regular array. Each data in the organization represents the non-geometric attribute characteristics of the objects or phenomena.

6. Cell: A wireless coverage area identified by a base station identification code or a global cell identification code. When using an omnidirectional antenna structure, the cell is the base station area.

Please refer to Figure 1, which shows a schematic architectural diagram of an autonomous system provided by an embodiment of the present application.

As shown in Figure 1, the autonomous system of this application can be divided into three situations: single-domain autonomous system, cross-domain autonomous system, and business autonomous system.

A single-domain autonomous system includes network elements and domain management functional units.

The cross-domain autonomous system includes: network elements, domain management functional units, and cross-domain management functional units.

The business autonomous system includes: network elements, domain management functional units, cross-domain management functional units, and business operation units.

Among them, network element: is an entity that provides network services, including core network elements, access network elements, etc. For example, core network elements may include, but are not limited to, access and mobility management function (AMF) entities, session management function (SMF) entities, policy control function (PCF) entities. Entities, network data analysis function (NWDAF) entities, network repository function (NRF), gateways, etc. Access network elements may include but are not limited to: various types of base stations (such as next-generation base stations (generation node B, gNB), evolved base stations (evolved Node B, eNB), centralized control units (central unit control panel (CUCP), centralized unit (CU), distributed unit (DU), centralized user panel (CUUP), etc.

Among them, domain management functional unit: It can also be called subnetwork management function (subnetwork management function) or network element management functional unit (network element/function management function). The domain management functional unit provides one or more of the following functions or management Services: life cycle management of subnetworks or network elements, deployment of subnetworks or network elements, fault management of subnetworks or network elements, performance management of subnetworks or network elements, guarantee of subnetworks or network elements, subnetwork or network Optimal management of elements, intention translation of subnetworks or network elements, etc. The subnetwork here includes one or more network elements. Alternatively, the subnetwork here may also include one or more subnetworks, that is, one or more subnetworks form a subnetwork with a larger coverage area. Alternatively, the subnetwork here may also include one or more network slice subnetworks. Subnetworks include one of the following description methods:

A network in a certain technical domain, such as wireless access network, core network, transmission network, etc.;

A network of a certain standard, such as a global system for mobile communications (GSM) network, a long term evolution (LTE) network, a fifth generation mobile communication technology (5th Generation Mobile Communication Technology, 5G) network, etc. ;

The network provided by a certain equipment vendor, such as the network provided by equipment vendor X, etc.;

A network in a certain geographical area, such as the network of factory A, the network of prefecture-level city B, etc.

Among them, the cross-domain management functional unit: It can also be called the network management function. The cross-domain management functional unit provides one or more of the following functions or management services: network life cycle management, network deployment, Network fault management, network performance management, network configuration management, network guarantee, network optimization function, translation of network intent from communication service provider (intent-CSP), communication service user's Translation of network intent (intent from communication service consumer, intent-CSC), etc. The network here can include one or more network elements, subnetworks or network slices. For example, the cross-domain management functional unit may be a network slice management function (NSMF), a management data analytical function (MDAF), or a cross-domain self-organization network function. SON-function), or cross-domain intent management functional unit.

It should be noted that in some deployment scenarios, the cross-domain management functional unit can also provide one or more of the following management functions or management services: subnetwork life cycle management, subnetwork deployment, subnetwork fault management, Performance management of subnetworks, configuration management of subnetworks, guarantee of subnetworks, optimization functions of subnetworks, translation of subnetwork intentions of communication service providers, translation of subnetwork intentions of communication service users, etc. Among them, a subnetwork can be composed of multiple small subnetworks or multiple network slice subnetworks.

Among them, the business operation unit: It can also be called the communication service management function unit (communication service management function), which can provide functions such as billing, settlement, accounting, customer service, business, network monitoring, communication service life cycle management, and business intent translation. and management services. The business operation unit includes the operator's operation system or the vertical industry's operation system (vertical operational technology system).

In addition, under the service-oriented management architecture, focusing on the management service producer (MnS Producer) and the management service consumer (MnS Consumer), it should be understood:

When the management service is the management service provided by the above-mentioned business operation unit, the business operation unit is the management service provider, and other business operator units can be management service consumers;

When the management service is the management service provided by the above-mentioned cross-domain management functional unit, the cross-domain management functional unit is the management service provider and the business operation unit is the management service consumer;

When the management service is the management service provided by the above-mentioned domain management functional unit, the domain management functional unit is the management service provider, and the cross-domain management functional unit or business operation unit is the management service consumer;

When the management service is the management service provided by the above-mentioned network element, the network element is the management service provider, and the domain management functional unit or cross-domain management functional unit or business operation unit is the management service consumer.

In addition, this application does not limit the number of network elements, domain management functional units, cross-domain management functional units, and business operation units. For ease of explanation, in Figure 1, the single-domain autonomous system uses two network elements and one domain management functional unit as an example. The cross-domain autonomous system uses four network elements, two domain management functional units, and one cross-domain management function. Taking the unit as an example, the business autonomous system uses four network elements, two domain management functional units, one cross-domain management functional unit, and one business operation unit as an example.

In summary, this application can use the first device to evaluate the autonomous system in any of the above situations.

Among them, the first device can be deployed within the autonomous system or outside the autonomous system, that is, both are independent of the autonomous system. This application does not limit this.

Please refer to Figure 2. Figure 2 shows a schematic structural diagram of a first device provided by an embodiment of the present application. As shown in Figure 2, the first device of the present application may include: a monitoring module, an execution module, and a logical interface.

Among them, the monitoring module is used to obtain the autonomy capability evaluation results of the autonomous system. The execution module is used to obtain data for evaluating the autonomous system and evaluate the autonomous system. The logical interface is used to realize the communication connection between the monitoring module and the execution module.

Among them, the monitoring module or execution module can be deployed within the autonomous system or outside the autonomous system, and this application does not limit this. For example, the monitoring module and execution module can adopt the following deployment scenarios:

Deployment scenario 1: The monitoring module and execution module are deployed outside the autonomous system.

Deployment scenario 2: The monitoring module is deployed outside the autonomous system, and the execution module is deployed within the autonomous system.

Deployment scenario 3: The monitoring module and execution module are deployed in the autonomous system.

In addition, under the service-oriented management architecture, the monitoring module is the management service consumer, and the execution module is the management service provider. The logical interface is the management service.

In addition, the first device can also be communicatively connected with the autonomous system. Optionally, the execution module can also be communicatively connected with the autonomous system.

For ease of explanation, in Figure 2, the first device is deployed outside the autonomous system, and the execution module is connected to the autonomous system for communication as an example.

In addition, the first device can also be communicatively connected with the second device. Optionally, the execution module can also be communicatively connected with the second device.

The first device and the second device may be the same device or different devices.

The first device and the second device may be devices of the same operator, or may be devices of different operators.

For example, the first device may be an operator's evaluation system, and the second device may be an operator's management system.

For ease of explanation, in FIG. 2 , the first device and the second device are different devices, and the communication connection between the execution module and the second device is taken as an example for illustration.

Based on the foregoing description, the following embodiments of this application will take the first device or the second device in the embodiment of Figure 2 to evaluate the autonomous system with the structure shown in Figure 1 as an example. Combined with the drawings and application scenarios, the autonomous system provided by this application will be evaluated. The method of determining the indicators of the network optimization function will be elaborated in detail.

Please refer to Figure 3. Figure 3 shows a signaling interaction diagram of a method for determining indicators of a network optimization function of an autonomous system provided by an embodiment of the present application.

As shown in Figure 3, the indicator determination method of the network optimization function of the autonomous system of this application may include:

S101. The first device obtains first data of the autonomous system. The first data includes: network performance statistical data and/or network optimization process data.

Through the communication connection with the autonomous system, the first device can obtain the first data of the autonomous system in various ways.

For example, the first device may send a request to the autonomous system, the request being used to obtain first data of the autonomous system. The autonomous system can then send the first data of the autonomous system to the first device. Therefore, the first device can obtain the first data of the autonomous system.

For another example, the first device may obtain the first data of the autonomous system from the autonomous system.

In addition, the first device can obtain the first data of the autonomous system through automatic triggering, such as periodicity. Alternatively, the first device may obtain the first data of the autonomous system through manual triggering, such as an end user or operation and maintenance personnel issuing an instruction to the first device.

In addition, the first device can obtain the relevant data of the autonomous system before the autonomous system performs the network optimization function, and obtain the relevant data of the autonomous system after the autonomous system performs the network optimization function, so that the first device of the autonomous system can be obtained based on the aforementioned relevant data. Data can also be obtained after the autonomous system performs network optimization to obtain the first data of the autonomous system.

Among them, the first data is used to evaluate the performance of the network optimization function of the autonomous system. For example, it may involve relevant data before the autonomous system performs the network optimization function, and relevant data after the autonomous system performs the network optimization function, or may involve the autonomous system performing the network optimization function. Relevant data after optimizing functions.

In addition, the network optimization function of the autonomous system mentioned in this application can be understood as: the autonomous system has the ability to perform the network optimization function, that is, the network optimization function is used to optimize the network performance of the autonomous system. After the autonomous system performs the network optimization function, Optimization of network performance of the aforementioned autonomous systems can be achieved. Among them, this application does not limit the number and types of network optimization functions that can optimize the network performance of autonomous systems.

For example, if the network optimization function is used to optimize coverage performance, then after the autonomous system executes the network optimization function, the coverage performance of the autonomous system can be optimized. For example, if the network optimization function is used to optimize rate performance, then after the autonomous system executes the network optimization function, the rate performance of the autonomous system can be optimized.

The first data may include: network performance statistical data and/or network optimization process data.

Among them, network performance statistics mainly focus on changes in network performance before and after the autonomous system executes the network optimization function, and are used to evaluate the optimization of the network performance of the autonomous system by the network optimization function.

It should be noted that network performance statistics can also be called network performance statistics, network performance-related statistics, network performance-related data, etc.

In some embodiments, the network performance statistical data may include: at least one of coverage performance data, capacity performance data, rate performance data, delay performance data, quality difference data, or experience difference data, where the quality difference data includes Data indicating whether the network performance of autonomous systems meets preset performance quality standards, poor experience data Includes data used to characterize whether the autonomous system's network performance meets preset user experience standards.

Coverage performance data includes coverage performance used to characterize autonomous systems. For example, the coverage performance data may include: the number of cells with weak coverage, the number of grids with weak coverage, the number of cells with standard coverage, and the number of grids with standard coverage.

Among them, the weak coverage cell is obtained based on the RSRP value and/or SINR value of the cell. Among them, the RSRP value refers to the reference signal received power (reference signal received power). The SINR value refers to the signal to interference plus noise ratio. For example, if the RSRP value of cell 1 is less than a certain set value (cell weak coverage RSRP threshold), then cell 1 is a weak coverage cell; on the contrary, if the RSRP value of cell 2 is greater than a certain set value (cell coverage meets the standard RSRP threshold), then cell 1 is a weak coverage cell. 2 is a community with standard coverage.

Among them, the weak coverage raster is obtained based on the RSRP value and/or SINR value of the raster. For example, if the SINR value of grid 1 is less than a certain set value (grid weak coverage SINR threshold), then grid 1 is a weak coverage grid; on the contrary, the condition for determining the coverage standard grid is that the RSRP of grid 2 is greater than a certain If the value is set (the grid coverage standard RSRP threshold), grid 2 is the coverage standard grid.

Capacity performance data includes capacity performance used to characterize autonomous systems. For example, the capacity performance data may include: the number of high-load cells, the number of high-load grids, the number of load-compliant cells, and the number of load-compliant grids.

Among them, the high-load cell uses the physical resource block (PRB, that is, the resources of 12 consecutive subcarriers in the frequency domain) utilization of the cell and/or the radio resource control (radio resource controller, RRC, that is, the control of radio resources). The number of connections is obtained by controlling the allocation and sending relevant signaling). For example, if the PRB value of cell 1 is greater than a certain set value (the cell's high-load PRB threshold), then cell 1 is a high-load cell; conversely, if the PRB value of cell 2 is less than a certain set value (the cell's load reaches the PRB threshold), then Community 2 is a community with load standards.

Among them, the high-load grid is obtained based on the PRB value and/or the number of RRC connections of the grid. For example, if the number of RRC connections of grid 1 is greater than a certain set value (the threshold of the number of RRC connections of a grid with high load), then grid 1 is a high-load grid; conversely, the PRB value of grid 2 is less than a certain set value ( If the grid load reaches the standard PRB threshold), then cell 2 is a grid with a load that reaches the standard.

Rate performance data includes rate performance used to characterize autonomous systems. For example, the rate performance data may include: the number of low-rate cells, the number of low-rate grids, the number of low-rate users (user equipment, UE), the number of cells that meet the rate standard, the number of grids that meet the rate standard, and the number of users that meet the rate standard.

Among them, the low-rate cell is obtained based on the average rate (throughput) of the cell. For example, if the average rate of cell 1 is less than a certain set value (cell low rate throughput threshold), then cell 1 is a low rate cell; on the contrary, if the average rate of cell 2 is greater than a certain set value (cell rate compliance threshold), then Community 2 is a community where the rate reaches the standard.

Among them, the low-rate raster is obtained based on the average rate (throughput) of the raster. For example, if the average rate of grid 1 is less than the set value (grid low rate throughput threshold), then grid 1 is a low-rate grid; on the contrary, if the average rate of grid 2 is greater than a certain set value (grid rate meets the standard threshold), then grid 2 is the rate standard grid.

Among them, low-rate users are obtained based on user rate (UE throughput). For example, if the local user rate is less than the set value (low user rate UE throughput threshold), it is determined to be a low-rate user. On the contrary, for example, if the average rate of grid 2 is greater than a certain set value (grid rate compliance threshold), grid 2 is a rate compliance grid.

Latency performance data includes latency performance used to characterize autonomous systems. For example, the latency performance data may include: the number of high-latency cells, the number of high-latency grids, the number of high-latency users, the number of latency-compliant cells, the number of latency-compliant grids, and the number of latency-compliant users.

Among them, the high-latency cell is obtained based on the average latency of the cell. For example, if the average delay of cell 1 is greater than a certain set value (cell high delay threshold), then cell 1 is a high delay cell; on the contrary, if the average delay of cell 2 is less than or equal to a certain set value (the cell delay meets the standard threshold), then cell 2 is the cell that meets the delay standard.

Among them, the high-latency raster is obtained based on the average delay of the raster. For example, the average delay of grid 1 is greater than the set value (grid high delay threshold), then grid 1 is a high-latency grid; conversely, the average delay of grid 2 is less than or equal to a certain set value (grid high delay threshold). grid delay meets the standard threshold), grid 2 is the delay standard grid.

Among them, high-latency users are obtained based on user latency. For example, if user 1's delay is higher than the set value (high user delay threshold), it is determined that user 1 is a high-latency user. On the contrary, if the average delay of user 2 is less than or equal to a certain set value (user delay compliance threshold), then user 2 is a delay compliance user.

Poor quality data includes data used to characterize whether the network performance of an autonomous system meets preset performance quality standards. The aforementioned preset performance quality standards can be set according to network optimization performance and the specific conditions of the autonomous system, as well as the actual needs of users. For example, the poor quality data may include: the number of problem cells, the number of problem rasters, the number of quality-compliant cells, and the number of quality-compliant rasters.

Among them, the problem cell is a cell with one of weak coverage problems, high load problems, low rate problems and high delay problems. On the contrary, a cell that meets the quality standard is a cell that does not have weak coverage problems, high load problems, low rate problems, and high delay problems.

Among them, the problem grid is a grid with one of weak coverage problems, high load problems, low rate problems, and high delay problems. On the contrary, a quality grid is a grid that does not have weak coverage problems, high load problems, low rate problems, and high latency problems.

Poor experience data includes data used to characterize whether the network performance of the autonomous system meets the preset user experience standards. The aforementioned preset user experience standards can be set according to network optimization performance and the specific conditions of the autonomous system, as well as the actual needs of users. For example, poor experience data may include: the number of users with poor experience and the number of users with satisfactory experience.

Among them, users with poor experience (also known as problem users) are users who have one of low-rate problems and high-latency problems. On the contrary, users with satisfactory experience are users who do not have low speed problems and high latency problems.

Among them, the network optimization process data mainly focuses on data related to the network optimization function of the autonomous system, and is used to evaluate the optimization status of the network optimization function of the autonomous system.

It should be noted that the network optimization process data can also be called process data of the network optimization function, data related to the network optimization function, process data of the network optimization function, etc.

In addition, the optimization mentioned in this application refers to the execution of the network optimization function, before optimization is before the network optimization function is executed, and after optimization is after the network optimization function is executed.

In some embodiments, the network optimization process data may include: the start time of the network optimization function, the end time of the network optimization function, the scope of the network optimization function, the number of cells or grids or users that need to perform the network optimization function, root cause location at least one of the cells or grids or the number of users, or the cells or grids or the number of users generated after performing the network optimization function.

Among them, the scope of the network optimization function can be the number of cells, the number of grids, the number of users, or the area of the region.

Among them, the number of cells or grids or users that need to perform network optimization functions: the number of cells/grids/users to be optimized, that is, the number of problem cells or problem grids or the number of users with poor experience, and the number of users identified by the autonomous system that need to be executed The number of cells or grids or users for the network optimization function.

Among them, the number of cells, the number of grids, or the number of users for root cause locating: the number of cells, the number of grids, or the number of users for which the autonomous system performs root cause locating on problem cells, problem grids, or users with poor experience.

Among them, the number of cells or grids or users generated after executing the network optimization function: the number of cells or grids or the number of users generated after the autonomous system executes the network optimization function on problematic cells or grids or users with poor experience.

It should be noted that, for network performance statistics and network optimization process data, the autonomous system can use the collected raw data and/or the data calculated based on the collected raw data as the first data of the autonomous system. Sent to the first device.

Taking the RSRP value of the cell as an example, the autonomous system may send the RSRP value of the cell to the first device as the first data of the autonomous system. And/or, the autonomous system may calculate the number of weak coverage cells according to the RSRP value of the cell, and send the number of weak coverage cells to the first device as the first data of the autonomous system.

With reference to Figure 2, in the first device, the execution module can obtain the first data of the autonomous system.

Wherein, when the execution module needs to evaluate the network optimization function of the autonomous system, it can send a request to the autonomous system, and the request is used to instruct the autonomous system what data needs to be reported.

Among them, the execution module can take into account the subjective wishes of the end user/operation and maintenance user and one or more network performance that the autonomous system needs to pay attention to, to determine what data the autonomous system needs to report.

Therefore, the autonomous system can send the first data of the corresponding autonomous system to the execution module according to the request. Executing the module can obtain the first data of the autonomous system.

S102. The first device determines the first indicator of the autonomous system based on the first data. The first indicator is used to characterize the performance of the network optimization function of the autonomous system. The performance of the network optimization function at least includes: the optimization of the network performance of the autonomous system and / Or the optimization efficiency of network optimization functions.

Based on the description of S101, the first device can determine the first indicator of the autonomous system based on the first data of the autonomous system. For ease of explanation, the first indicator of the autonomous system can be expressed as KEI_Real.

Among them, the first indicator of the autonomous system is used to characterize the performance of the network optimization function of the autonomous system. The performance of the network optimization function at least includes: the optimization of the network performance of the autonomous system and/or the optimization efficiency of the network optimization function of the autonomous system.

The optimization of the network performance of the autonomous system is used to describe the network performance effect brought by the network optimization function to the autonomous system. The optimization efficiency of the network optimization function of the autonomous system is used to describe the efficiency results brought by the network optimization function to the autonomous system.

In some embodiments, the optimization of the network performance of the autonomous system may include: the network performance after the autonomous system performs the network optimization function, the improvement compared with the network performance before the autonomous system performs the network optimization function, and the network performance of the autonomous system. Details of network performance after optimizing features.

Optionally, the first indicator of the autonomous system may include: quality performance indicators and/or efficiency performance indicators.

Among them, quality performance indicators are determined based on network performance statistics. Quality performance indicators are used to characterize the optimization of network performance of autonomous systems.

In some embodiments, the quality performance indicators may include at least one of: poor quality optimization rate, poor experience optimization rate, poor quality performance, or poor experience performance.

For example, if quality performance indicators can be divided into: indicators related to performance optimization rate and indicators related to performance optimization results, then the optimization rate of poor quality and the optimization rate of poor experience are performance optimization rate related indicators, and the optimization rate of poor quality, And experiencing poor performance are indicators related to performance optimization results.

Among them, the quality difference optimization rate is used to characterize the quality improvement of the network performance after the autonomous system performs the network optimization function, compared with the network performance before the autonomous system performs the network optimization function.

For example, the quality difference optimization rate may be a quality difference cell optimization rate or a quality difference raster optimization rate.

The optimization rate of poor-quality cells can be expressed as: (number of poor-quality cells before optimization - number of poor-quality cells after optimization)/number of poor-quality cells before optimization * 100%.

The optimization rate of poor-quality rasters can be expressed as: (number of poor-quality rasters before optimization - number of poor-quality rasters after optimization)/number of poor-quality rasters before optimization * 100%.

In some embodiments, the poor quality optimization rate may include at least one of a weak coverage optimization rate, a high load optimization rate, a low rate optimization rate, or a low delay optimization rate.

For example, the weak coverage optimization rate may be a weak coverage cell optimization rate or a weak coverage grid optimization rate.

The optimization rate of weak coverage cells can be expressed as: (number of weak coverage cells before optimization - number of weak coverage cells after optimization) / number of weak coverage cells before optimization * 100%.

The optimization rate of weakly covered rasters can be expressed as: (number of weakly covered rasters before optimization - number of weakly covered rasters after optimization)/number of weakly covered rasters before optimization * 100%.

For example, the high load optimization rate may be a high load cell optimization rate, or a high load grid optimization rate.

The optimization rate of high-load cells can be expressed as: (number of high-load cells before optimization - number of high-load cells after optimization)/number of high-load cells before optimization * 100%.

The optimization rate of high-load grids can be expressed as: (number of high-load grids before optimization - number of high-load grids after optimization)/number of high-load grids before optimization * 100%.

For example, the low rate optimization rate may be a high load cell optimization rate, or a high load grid optimization rate.

The optimization rate of low-rate cells can be expressed as: (number of low-rate cells before optimization - number of low-rate cells after optimization)/number of low-rate cells before optimization * 100%.

The low-rate raster optimization rate can be expressed as: (number of low-rate rasters before optimization - number of low-rate rasters after optimization)/number of low-rate rasters before optimization * 100%.

For example, the low-latency optimization rate may be a low-latency cell optimization rate or a low-latency grid optimization rate.

The low-latency cell optimization rate can be expressed as: (number of low-latency cells before optimization - number of low-latency cells after optimization) / number of low-latency cells before optimization * 100%.

The low-latency grid optimization rate can be expressed as: (number of low-latency grids before optimization - number of low-latency grids after optimization) / number of low-latency grids before optimization * 100%.

Among them, the experience difference optimization rate is used to characterize the network performance after the autonomous system performs the network optimization function, and the user experience improvement compared with the network performance before the autonomous system performs the network optimization function.

For example, the optimization rate of poor experience can be expressed as: (number of users with poor experience before optimization - number of users with poor experience after optimization) / number of users with poor experience before optimization * 100%.

In some embodiments, the poor experience optimization rate may include at least one of: a low-rate user optimization rate or a low-latency user optimization rate.

The optimization rate of low-rate users can be expressed as: (number of low-rate users before optimization - number of low-rate users after optimization)/number of low-rate users before optimization * 100%.

The optimization rate of low-latency users can be expressed as: (number of low-latency users before optimization - number of low-latency users after optimization) /Number of low-latency users before optimization*100%.

Among them, poor quality performance is used to characterize whether the network performance of the autonomous system after executing the network optimization function reaches the quality standard.

For example, poor quality performance can be the proportion of cells with poor quality after optimization, the proportion of grids with poor quality after optimization, the proportion of cells with standard quality after optimization, or the proportion of grids with standard quality after optimization.

After optimization, the proportion of poor-quality cells can be expressed as: (number of poor-quality cells)/(number of all cells)*100%.

The proportion of poor-quality rasters after optimization can be expressed as: (number of poor-quality rasters)/(number of all rasters)*100%.

After optimization, the proportion of cells that meet quality standards can be expressed as: (number of cells that meet quality standards)/(number of all cells)*100%.

After optimization, the proportion of grids that meet quality standards can be expressed as: (number of grids that meet quality standards)/(number of all grids)*100%.

In some embodiments, poor performance may include: coverage performance after executing the network optimization function, capacity performance after executing the network optimization function, rate performance after executing the network optimization function, or delay performance after executing the network optimization function. at least one of.

For example, the coverage performance after executing the network optimization function can be the proportion of cells with weak coverage after optimization, the proportion of grids with weak coverage after optimization, the proportion of cells with standard coverage after optimization, or the proportion of cells with standard coverage after optimization.

After optimization, the proportion of weak coverage cells can be expressed as: (number of weak coverage cells)/(number of all cells)*100%.

The proportion of weakly covered rasters after optimization can be expressed as: (number of weakly covered rasters)/(number of all rasters)*100%.

After optimization, the proportion of cells that meet the coverage standard can be expressed as: (number of cells that meet the coverage standard)/(number of all cells)*100%.

After optimization, the proportion of grids that meet the coverage standard can be expressed as: (number of grids that meet the coverage standard)/(number of all grids)*100%.

For example, the capacity performance after executing the network optimization function can be the proportion of high-load cells after optimization, the proportion of high-load grids after optimization, the proportion of cells with standard load after optimization, and the proportion of grids with standard load after optimization.

After optimization, the proportion of high-load cells can be expressed as: (number of high-load cells)/(number of all cells)*100%.

After optimization, the proportion of high-load grids can be expressed as: (number of high-load grids)/(number of all grids)*100%.

After optimization, the proportion of cells that meet the load standard can be expressed as: (number of cells that meet the load standard)/(number of all cells)*100%.

After optimization, the proportion of load-compliant grids can be expressed as: (number of load-compliant grids)/(number of all grids)*100%.

For example, the rate performance after executing the network optimization function can be the proportion of low-rate cells after optimization, the proportion of low-rate grids after optimization, the proportion of cells that meet the standard rate after optimization, or the proportion of grids that meet the standard rate after optimization.

After optimization, the proportion of low-rate cells can be expressed as: (number of low-rate cells)/(number of all cells)*100%.

The proportion of low-rate grids after optimization can be expressed as: (number of low-rate grids)/(number of all grids)*100%.

After optimization, the proportion of cells that meet the rate standard can be expressed as: (number of cells that meet the rate standard)/(number of all cells)*100%.

After optimization, the proportion of grids that meet the rate standard can be expressed as: (number of grids that meet the rate standard)/(number of all grids)*100%.

For example, the latency performance after executing the network optimization function can be the proportion of cells with high latency after optimization, the proportion of cells with high latency after optimization, The proportion of delayed grids, the proportion of cells that meet the delay standard after optimization, or the proportion of grids that meet the delay standard after optimization.

After optimization, the proportion of high-latency cells can be expressed as: (number of high-latency cells)/(number of all cells)*100%.

After optimization, the proportion of high-latency grids can be expressed as: (number of high-latency grids)/(number of all grids)*100%.

After optimization, the proportion of cells that meet the latency standard can be expressed as: (number of cells that meet the latency standard)/(number of all cells)*100%.

After optimization, the proportion of grids that meet the delay standard can be expressed as: (number of grids that meet the delay standard)/(number of all grids)*100%.

Among them, poor experience performance is used to characterize whether the network performance of the autonomous system after executing the network optimization function meets the user experience standard.

For example, poor experience performance can be the proportion of users with poor experience after optimization, and/or the proportion of users with standard experience after optimization.

After optimization, the proportion of users with poor experience can be expressed as: (number of users with poor experience)/(number of all users)*100%.

After optimization, the proportion of users whose experience meets the standard can be expressed as: (number of users whose experience meets the standard)/(number of all users)*100%.

In some embodiments, experiencing poor performance may include: the proportion of low-rate users after executing the network optimization function, the proportion of high-latency users after executing the network optimization function, the proportion of users meeting the standard rate after executing the network optimization function, or At least one of the proportion of users whose latency meets the standard after executing the network optimization function.

The proportion of low-rate users after the network optimization function is executed, that is, the proportion of low-rate users after optimization, can be expressed as: (number of low-rate users)/(number of all users)*100%.

The proportion of high-latency users after the network optimization function is executed, that is, the proportion of high-latency users after optimization, can be expressed as: (number of high-latency users)/(number of all users)*100%.

The proportion of users who meet the speed standard after executing the network optimization function, that is, the proportion of users who meet the speed standard after optimization, can be expressed as: (number of users whose speed meets the standard)/(number of all users)*100%.

The proportion of users who meet the delay standard after executing the network optimization function, that is, the proportion of users who meet the delay standard after optimization, can be expressed as: (number of users who meet the delay standard)/(number of all users)*100%.

Among them, efficiency performance indicators are determined based on network optimization process data. Efficiency performance indicators are used to characterize the optimization efficiency of the network optimization function of the autonomous system.

In some embodiments, efficiency performance indicators may include: the optimization duration of the network optimization function, the optimization scope of the network optimization function, the root cause positioning ratio of the network optimization function, or the cells or grids or users generated after executing the network optimization function. At least one of the optimized proportions of the quantity.

Among them, the optimization duration of the network optimization function can be obtained based on the optimization start time and optimization end time.

Among them, the optimization scope of the network optimization function can be the number of cells, the number of grids, the number of users, or the area of the region.

Among them, the root cause positioning ratio of the network optimization function may include: the root cause positioning ratio of the problem community, the root cause positioning ratio of the problem grid, or the root cause positioning ratio of users with poor experience.

The root cause locating ratio of problem cells can be expressed as: the number of root cause locating cells/the number of problem cells*100%.

The root cause positioning ratio of problem rasters can be expressed as: the number of root cause positioning rasters/the number of problem rasters*100%.

The ratio of root cause positioning for users with poor experience can be expressed as: number of root cause positioning users/number of problem users*100%.

Among them, the optimization ratio of cells or grids or the number of users generated after executing the network optimization function may include: the optimization ratio of problem cells, the optimization ratio of problem grids, or the optimization ratio of users with poor experience.

The optimization ratio of problem cells is used to represent the proportion of the number of cells generated after executing the network optimization function to the number of problem cells. It can be expressed as: the number of cells generated after executing the network optimization function / the number of problem cells * 100%.

The optimization ratio of problem rasters is used to represent the proportion of the number of rasters generated after executing the network optimization function to the number of problem rasters, which can be expressed as: the number of rasters generated after executing the network optimization function/the number of problem rasters *100%.

The optimization ratio of users with poor experience is used to represent the proportion of the number of users generated after executing the network optimization function to the number of problematic users, which can be expressed as: the number of users generated after executing the network optimization function/the number of problematic users * 100%.

With reference to Figure 2, in the first device, the execution module determines the first indicator of the autonomous system based on the first data of the autonomous system.

In summary, the first device can determine the first indicator of the autonomous system, and can present the optimization effect of the network optimization function of the autonomous system from multiple perspectives of quality performance and efficiency performance. Thus, the first device may perform S103 and/or S104.

S103. The first device determines the performance of the network optimization function of the autonomous system based on the first indicator.

Based on the description of S102, the first device can determine the performance of the network optimization function of the autonomous system based on the first indicator of the autonomous system, so that the first device can evaluate the impact of the introduction of autonomous technology in the network optimization function on the network performance of the autonomous system. Impact.

With reference to Figure 2, in the first device, the execution module can send the first indicator of the autonomous system to the monitoring module. The monitoring module can determine the performance of the network optimization function of the autonomous system based on the first indicator of the autonomous system.

S104. The first device sends the first indicator to the second device.

S105. The second device determines the performance of the network optimization function of the autonomous system based on the first indicator.

Based on the description of S102, the first device may send the first indicator of the autonomous system to the second device. Therefore, the second device can determine the performance of the network optimization function of the autonomous system based on the first indicator of the autonomous system, so that the second device can also evaluate the impact of introducing autonomous technology into the network optimization function on the network performance of the autonomous system. .

Among them, this application does not limit the sending method of the first device. In addition, the first device may also send the first data of the autonomous system and the first indicator of the autonomous system to the second device.

With reference to Figure 2, in the first device, the execution module may send the first indicator of the autonomous system to the second device.

In summary, for the network optimization function of the autonomous system, the first device or the second device can use the first indicator of the autonomous system to characterize the autonomy capability of the autonomous system, that is, the first indicator of the autonomous system is used as one of the autonomous capabilities of the autonomous system. Evaluation dimension to measure the impact of introducing autonomous technology into the network optimization function on the network performance of the autonomous system.

In addition, the first device or the second device can also use the first indicator of the autonomous system and the degree of automation of the autonomous system mentioned in related technologies as multiple evaluation dimensions of the autonomous capability of the autonomous system to comprehensively measure the network optimization function. The impact of the introduction of autonomous technology on the network performance of autonomous systems.

The method for determining the indicators of the network optimization function of the autonomous system provided by this application obtains the first data of the autonomous system through the first device. The first data includes: network performance statistical data and/or network optimization process data, so as to obtain the first data of the autonomous system through the first device. One data comprehensively covers the data before and after the autonomous system performs the network optimization function. The first device can determine the first indicator of the autonomous system based on the first data. The first indicator is used to characterize the performance of the network optimization function of the autonomous system. The performance of the network optimization function at least includes: the optimization of the network performance of the autonomous system and/ Or the optimization efficiency of the network optimization function. The first device can determine the performance of the network optimization function of the autonomous system based on the first indicator, so that the first device can estimate the impact of the network optimization function on the autonomous system. influence, and/or, the first device sends a first indicator to the second device, and the second device determines the performance of the network optimization function of the autonomous system based on the first indicator, so that the first device can estimate the network optimization function to the autonomous system impact.

Therefore, with the help of indicators used to characterize the performance of the network optimization function of the autonomous system, that is, the above-mentioned first indicator of the autonomous system, it can be used as an evaluation dimension of the autonomy capability of the network optimization function of the autonomous system, and the autonomy capability of the autonomous system can be quantitatively evaluated. The influence/optimization effect brought by the network optimization function to the network performance of the autonomous system (such as from multiple perspectives of quality performance and/or efficiency performance) avoids that the autonomous capability of the network optimization function of the telecommunications network in related technologies only represents automation The degree of limitations helps to rationally deploy autonomous systems that introduce different autonomous technologies into network optimization functions.

Please refer to FIG. 4 , which shows a signaling interaction diagram of a method for determining indicators of a network optimization function of an autonomous system provided by an embodiment of the present application.

As shown in Figure 4, the indicator determination method of the network optimization function of the autonomous system of this application may include:

S201. The first device obtains the first data of the autonomous system. The first data includes: network performance statistical data and/or network optimization process data.

S202. The first device determines the first indicator of the autonomous system based on the first data. The first indicator is used to characterize the performance of the network optimization function of the autonomous system. The performance of the network optimization function at least includes: the optimization status of the network performance of the autonomous system and / Or the optimization efficiency of network optimization functions.

Among them, S201 and S202 are implemented similarly to S101 and S102 in the embodiment of FIG. 3 respectively, and will not be described again in this application.

S203. The first device determines the second indicator of the autonomous system based on the first indicator. The second indicator is used to indicate whether the performance of the network optimization function of the autonomous system meets the preset conditions.

The first device can evaluate whether the performance of the network optimization function of the autonomous system meets the preset conditions based on the first indicator of the autonomous system, thereby determining the second indicator of the autonomous system.

The second indicator of the autonomous system is used to indicate whether the performance of the network optimization function of the autonomous system meets the preset conditions, and the preset conditions can be set according to the actual situation. In other words, the second indicator of the autonomous system refers to the degree of satisfaction of the performance of the network optimization function of the autonomous system. Therefore, according to the second indicator of the autonomous system, the performance of the network optimization function of the autonomous system can be statistically analyzed and comprehensively reflected. Performance of network optimization functions for autonomous systems. For ease of explanation, the second indicator of the autonomous system can be expressed as KEI_Fulfillment.

Among them, the first device can determine the second indicator of the autonomous system in various ways.

In some embodiments, the first device may determine the minimum value of the first indicator based on a preset indicator benchmark. Therefore, the first device can determine the minimum value of the first indicator as the second indicator of the autonomous system.

In other embodiments, the first device may determine the weighted value of the first indicator based on a preset indicator benchmark. Therefore, the first device can determine the weighted value of the first indicator as the second indicator of the autonomous system.

For ease of explanation, the preset indicator baseline can be expressed as KEI_Baseline. In addition, the preset indicator baseline can also be called the indicator baseline value.

For example, assume that 3 KEIs (KEI1, KEI2, KEI3) are selected to evaluate the performance of the network optimization function of the autonomous system. Correspondingly, the first indicators of the autonomous system include: KEI1_Real, KEI2_Real, and KEI3_Real. The aforementioned three indicators The corresponding preset indicator benchmarks are: KEI1_Baseline, KEI2_Baseline, and KEI3_Baseline.

If the minimum value method is adopted, then

The second indicator of the autonomous system KEI_Fulfillment=MIN(

MIN(KEI1_Real/KEI1_Baseline, 1),

MIN(KEI2_Real/KEI2_Baseline, 1),

MIN(KEI3_Real/KEI3_Baseline, 1)).

Among them, MIN() refers to the minimum value within an interval.

MIN(KEI1_Real/KEI1_Baseline, 1) refers to the minimum value among KEI1_Real/KEI1_Baseline and 1.

MIN(KEI2_Real/KEI2_Baseline, 1) refers to the minimum value among KEI2_Real/KEI2_Baseline and 1.

MIN(KEI3_Real/KEI3_Baseline, 1) refers to the minimum value among KEI3_Real/KEI3_Baseline and 1.

Correspondingly, the second indicator KEI_Fulfillment of the autonomous system is the minimum value among MIN(KEI1_Real/KEI1_Baseline, 1), MIN(KEI2_Real/KEI2_Baseline, 1), and MIN(KEI3_Real/KEI3_Baseline, 1).

If a weighted value selection method is adopted, then

The second indicator of the autonomous system KEI_Fulfillment=

MIN(KEI1_Real/KEI1_Baseline, 1)*MIN(KEI2_Real/KEI2_Baseline, 1)*MIN(KEI3_Real/KEI3_Baseline, 1).

Among them, MIN() refers to the minimum value within an interval.

Correspondingly, the second indicator KEI_Fulfillment of the autonomous system is the product of MIN(KEI1_Real/KEI1_Baseline, 1), MIN(KEI2_Real/KEI2_Baseline, 1), and MIN(KEI3_Real/KEI3_Baseline, 1).

With reference to Figure 2, in the first device, the execution module determines the second indicator of the autonomous system based on the first indicator of the autonomous system.

In summary, the first device can determine the first indicator of the autonomous system and the second indicator of the autonomous system. Therefore, the first device may perform S203 and/or S204.

S204. The first device determines the performance of the network optimization function of the autonomous system based on the first indicator and the second indicator.

Based on the description of S204, the first device can determine the performance of the network optimization function of the autonomous system based on the first indicator of the autonomous system and the second indicator of the autonomous system, and can comprehensively present the optimization effect of the performance of the network optimization function, so that the first device The device can evaluate the impact of introducing autonomous technology into the network optimization function on the network performance of the autonomous system.

Combined with Figure 2, in the first device, the execution module can send the first indicator of the autonomous system and the autonomous system to the monitoring module. The second indicator of the system. The monitoring module may determine the performance of the network optimization function of the autonomous system based on the first indicator of the autonomous system and the second indicator of the autonomous system.

S205. The first device sends the first indicator and the second indicator to the second device.

S206. The second device determines the performance of the network optimization function of the autonomous system based on the first indicator and the second indicator.

Based on the description of S204, the first device may send the first indicator of the autonomous system and the second indicator of the autonomous system to the second device. Therefore, the second device can determine the performance of the network optimization function of the autonomous system based on the first indicator of the autonomous system and the second indicator of the autonomous system, so that the second device can also evaluate the performance of the autonomous system after introducing autonomous technology into the network optimization function. impact on network performance.

Among them, this application does not limit the sending method of the first device. In addition, the first device can also jointly send the first data of the autonomous system, the first indicator of the autonomous system, and the second indicator of the autonomous system to the second device.

With reference to Figure 2, in the first device, the execution module may send the first indicator of the autonomous system and the second indicator of the autonomous system to the second device.

In summary, for the network optimization function of the autonomous system, the first device or the second device can use the first indicator of the autonomous system and the second indicator of the autonomous system to characterize the autonomous capability of the autonomous system, that is, the first indicator of the autonomous system and the second indicator of the autonomous system. The second indicator of the autonomous system is jointly used as an evaluation dimension of the autonomy capability of the autonomous system to measure the impact of the introduction of autonomous technology into the network optimization function on the network performance of the autonomous system.

In addition, the first device or the second device may also use the first indicator of the autonomous system, the second indicator of the autonomous system, and the degree of automation of the autonomous system mentioned in the related technology as multiple evaluation dimensions of the autonomous capability of the autonomous system. To comprehensively measure the impact of introducing autonomous technology into the network optimization function on the network performance of autonomous systems.

It should be noted that, in addition to the above implementation manner, in S205, the first device may also send the first indicator to the second device, so that the second device determines the second indicator of the autonomous system based on the first indicator. Therefore, the second device can determine the performance of the network optimization function of the autonomous system based on the first indicator of the autonomous system and the second indicator of the autonomous system.

The specific implementation method of the second device determining the second indicator of the autonomous system can be referred to the description of the first device determining the second indicator of the autonomous system in the embodiment of FIG. 3, which will not be described again here.

In this application, in addition to the first indicator of the autonomous system, the first device or the second device can also determine the second indicator of the autonomous system based on the first indicator of the autonomous system, and can perform statistics on the performance of the network optimization function of the autonomous system. Analyze and comprehensively present the optimization effect of the network optimization function of the autonomous system.

Therefore, with the help of indicators used to characterize the performance of the network optimization function of the autonomous system, that is, the above-mentioned first indicator of the autonomous system and the second indicator of the autonomous system, they can be jointly used as an evaluation dimension of the autonomy capability of the network optimization function of the autonomous system. , can accurately and quantitatively evaluate the impact/optimization effect of the network optimization function of the autonomous system on the network performance of the autonomous system, avoiding the limitation that the autonomy capability of the network optimization function of the telecommunications network in related technologies only represents the degree of automation, and has Helps in the rational deployment of autonomous systems that introduce different autonomous technologies into network optimization functions.

As an example, this application also provides an indicator determination device for the network optimization function of an autonomous system.

Please refer to FIG. 5 , which is a schematic structural diagram of an apparatus for determining indicators of a network optimization function of an autonomous system provided by an embodiment of the present application.

As shown in Figure 5, the indicator determination device 100 of the network optimization function of the autonomous system can exist independently or can be integrated in other devices, and can communicate with the second device in Figure 2 to achieve any of the above. In a method embodiment, corresponding to the operation of the first device, the indicator determination device 100 of the network optimization function of the autonomous system of the present application may include:

The acquisition module 101 is used to acquire the first data of the autonomous system. The first data includes: network performance statistical data and/or network optimization process data;

The determination module 102 is configured to determine the first indicator of the autonomous system based on the first data. The first indicator is used to characterize the performance of the network optimization function of the autonomous system. The performance of the network optimization function at least includes: the optimization of the network performance of the autonomous system. and/or the optimization efficiency of network optimization functions;

The determination module 102 is also used to determine the performance of the network optimization function of the autonomous system according to the first indicator;

The sending module 103 is used to send the first indicator to the second device.

In some embodiments, the optimization of the network performance of the autonomous system includes: the network performance after the autonomous system performs the network optimization function, the improvement compared with the network performance before the autonomous system performs the network optimization function, and the network optimization performed by the autonomous system Specifics of network performance after functionality.

In some embodiments, the determining module 102 is also used to:

In some embodiments, the determining module 102 is specifically used to:

Determine the minimum value or weighted value as the second indicator.

In some embodiments, the sending module 103 is also used to send the second indicator to the second device.

In some embodiments, the network performance statistical data includes: at least one of coverage performance data, capacity performance data, rate performance data, delay performance data, quality difference data, or experience difference data, and the quality difference data includes information used to characterize Data on whether the network performance of the autonomous system meets the preset performance quality standards. The poor experience data includes data used to characterize whether the network performance of the autonomous system meets the preset user experience standards;

In some embodiments, the first indicator includes: a quality performance indicator and/or an efficiency performance indicator. The quality performance indicator is used to characterize the optimization of network performance of the autonomous system, and the efficiency performance indicator is used to characterize the network of the autonomous system. Optimize the efficiency of the optimization function.

In some embodiments, quality performance indicators are determined based on network performance statistics;

In some embodiments, the quality performance indicators include at least one of: poor quality optimization rate, poor experience optimization rate, poor quality performance, or poor experience performance. The poor quality optimization rate is used to characterize the performance of the autonomous system after performing the network optimization function. The network performance, the quality improvement compared with the network performance before the autonomous system performs the network optimization function, the experience difference optimization rate is used to characterize the network performance after the autonomous system performs the network optimization function, compared with the network performance before the autonomous system performs the network optimization function User experience improvement compared with performance, poor performance is used to characterize autonomous system execution network Whether the network performance after the network optimization function reaches the quality standard, and poor experience performance is used to characterize whether the network performance after the autonomous system performs the network optimization function reaches the user experience standard;

In some embodiments, the poor quality optimization rate includes: at least one of a weak coverage optimization rate, a high load optimization rate, a low rate optimization rate, or a low latency optimization rate;

Please refer to FIG. 6 , which is a schematic structural diagram of an apparatus for determining indicators of a network optimization function of an autonomous system provided by an embodiment of the present application.

As shown in Figure 6, the indicator determination device 200 of the network optimization function of the autonomous system can exist independently or can be integrated in other devices. It can communicate with the first device in Figure 2 to achieve any of the above. In the method embodiment, corresponding to the operation of the second device, the indicator determination device 200 of the network optimization function of the autonomous system of the present application may include:

The receiving module 201 is configured to receive the first indicator of the autonomous system from the first device. The first indicator is used to characterize the performance of the network optimization function of the autonomous system. The performance of the network optimization function at least includes: the optimization of the network performance of the autonomous system and /or the optimization efficiency of the network optimization function;

The determination module 202 is configured to determine the performance of the network optimization function of the autonomous system according to the first indicator.

In some embodiments, the determining module 202 is also used to:

In some embodiments, the determining module 202 is specifically used to:

Determine the minimum value or weighted value as the second indicator.

In some embodiments, the receiving module 201 is also used to receive the second indicator from the first device;

The determination module 202 is specifically configured to determine the performance of the network optimization function of the autonomous system based on the first indicator and the second indicator.

In some embodiments, the quality performance indicators include at least one of: poor quality optimization rate, poor experience optimization rate, poor quality performance, or poor experience performance. The poor quality optimization rate is used to characterize the performance of the autonomous system after performing the network optimization function. The network performance, the quality improvement compared with the network performance before the autonomous system performs the network optimization function, the experience difference optimization rate is used to characterize the network performance after the autonomous system performs the network optimization function, compared with the network performance before the autonomous system performs the network optimization function The user experience improvement is compared with the performance. Poor quality performance is used to characterize whether the network performance of the autonomous system reaches the quality standard after executing the network optimization function. Poor experience performance is used to characterize whether the network performance of the autonomous system reaches the user experience after executing the network optimization function. standard;

The indicator determination device of the network optimization function of the autonomous system of the present application can be used to execute the technical solutions of the method embodiments shown above. Its implementation principles and technical effects are similar. The operation of each module can be further referred to the method embodiments. The relevant description will not be repeated here. The modules here can also be replaced by components or circuits.

This application can divide the first device or the second device into functional modules according to the above method examples. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or software function modules. It should be noted that the division of modules in each embodiment of the present application is schematic and is only a Logical function division can be divided in other ways during actual implementation.

As an example, this application also provides a communication device.

Please refer to FIG. 7 , which shows a schematic diagram of the hardware structure of a communication device provided by an embodiment of the present application.

As shown in Figure 7, the communication device 300 serves as a hardware support for the first device in Figure 2, can obtain the first data of the autonomous system from the second device, and send the first indicator of the autonomous system, or the first indicator of the autonomous system to the second device. The first indicator and the second indicator are used to implement the operation corresponding to the first device in any of the above method embodiments.

The communication device 300 of the present application may include: a memory 301 and a processor 302. The memory 301 and the processor 302 may be connected through a bus 303. Optionally, the processor 302 and the memory 301 are integrated together.

Memory 301, used to store program codes;

The processor 302 calls the program code. When the program code is executed, it is used to execute the method in any of the above embodiments. For details, please refer to the relevant descriptions in the foregoing method embodiments.

Optionally, this application also includes a communication interface 304, which can be connected to the processor 302 through the bus 303. The processor 302 can control the communication interface 304 to implement the above-mentioned receiving and sending functions of the communication device 300 .

The communication device in the embodiment of the present application can be used to execute the technical solutions in the above method embodiments. The implementation principles and technical effects are similar and will not be described again here.

As an example, this application also provides a communication device.

Please refer to FIG. 8 , which shows a schematic diagram of the hardware structure of a communication device provided by an embodiment of the present application.

As shown in Figure 8, the communication device 400 serves as a hardware support for the second device in Figure 2, can transmit the first data of the autonomous system to the first device, and receive the first indicator of the autonomous system from the first device, or the first indicator of the autonomous system. The first indicator and the second indicator are used to implement operations corresponding to the second device in any of the above method embodiments.

The communication device 400 of the present application may include: a memory 401 and a processor 402. The memory 401 and the processor 402 may be connected through a bus 403. Optionally, the processor 402 and the memory 401 are integrated together.

Memory 401, used to store program code;

The processor 402 calls the program code. When the program code is executed, it is used to execute the method in any of the above embodiments. For details, please refer to the relevant descriptions in the foregoing method embodiments.

Optionally, the present application includes a communication interface 404, which can be connected to the processor 402 through a bus 403. The processor 402 can control the communication interface 404 to implement the above-mentioned receiving and sending functions of the communication device 400 .

Exemplarily, this application also provides an indicator determination system for the network optimization function of an autonomous system, including: an autonomous system, and an indicator determination device for the network optimization function of the autonomous system in the embodiment of Figure 5; or, the autonomous system, Figure 5 The indicator determination device of the network optimization function of the autonomous system in the embodiment 5 and the indicator determination device of the network optimization function of the autonomous system in the embodiment of FIG. 6 .

Exemplarily, this application also provides an indicator determination system for the network optimization function of an autonomous system, including: a first device that implements the method in the previous embodiment, and a second device that implements the method in the previous embodiment.

Exemplarily, this application provides a chip, including: an interface circuit and a logic circuit. The interface circuit is used to receive signals from other chips other than the chip and transmit them to the logic circuit, or to transmit signals from the logic circuit. Sent to other chips outside the chip, the logic circuit is used to implement the methods in the previous embodiments.

Illustratively, the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor so that a communication device implements the method in the previous embodiments.

Exemplarily, the present application provides a computer program product, including: execution instructions, the execution instructions are stored in a readable storage medium, and at least one processor of the communication device can read the execution instructions from the readable storage medium, and the at least one processor Executing the execution instructions causes the communication device to implement the methods in the previous embodiments.

In the above embodiments, all or part of the functions may be implemented by software, hardware, or a combination of software and hardware. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. A computer program product includes one or more computer instructions. When computer program instructions are loaded and executed on a computer, processes or functions according to the present application are produced in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable device. Computer instructions may be stored in computer-readable storage media. Computer-readable storage media can be any available media that can be accessed by a computer or a data storage device such as a server, data center, or other integrated media that contains one or more available media. Available media may be magnetic media (eg, floppy disk, hard disk, tape), optical media (eg, DVD), or semiconductor media (eg, solid state disk (SSD)), etc.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments are implemented. This process can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, , may include the processes of the above method embodiments. The aforementioned storage media include: read-only memory (ROM) or random access memory (RAM), magnetic disks, optical disks and other media that can store program codes.

Claims

A method for determining indicators of the network optimization function of an autonomous system, which is characterized by including:

Obtain first data of the autonomous system, where the first data includes: network performance statistical data and/or network optimization process data;

According to the first data, a first indicator of the autonomous system is determined. The first indicator is used to characterize the performance of the network optimization function of the autonomous system. The performance of the network optimization function at least includes: the autonomous system The optimization of network performance and/or the optimization efficiency of the network optimization function;

Determine the performance of the network optimization function of the autonomous system according to the first indicator;

Or, send the first indicator to the second device.
The method according to claim 1, characterized in that the optimization of the network performance of the autonomous system includes: the network performance after the autonomous system performs the network optimization function, and the network performance before the autonomous system performs the network optimization function. Comparative performance improvement, and specific network performance after the autonomous system performs the network optimization function.
The method according to claim 1 or 2, characterized in that, the method further includes:

Determine a second indicator of the autonomous system according to the first indicator, where the second indicator is used to indicate whether the performance of the network optimization function of the autonomous system meets preset conditions;

Determining the performance of the network optimization function of the autonomous system according to the first indicator includes:

Based on the first indicator and the second indicator, the performance of the network optimization function of the autonomous system is determined.
The method of claim 3, wherein determining the second indicator of the autonomous system based on the first indicator includes:

Determine the minimum value or weighted value of the first indicator according to the preset indicator benchmark;

The minimum value or the weighted value is determined as the second index.
The method according to claim 3 or 4, characterized in that, the method further includes:

Send the second indicator to the second device.
The method according to any one of claims 1 to 5, characterized in that the network performance statistical data includes: coverage performance data, capacity performance data, rate performance data, delay performance data, poor quality data, or poor experience At least one of the data, the quality difference data includes data used to characterize whether the network performance of the autonomous system reaches a preset performance quality standard, and the experience difference data includes data used to characterize the network performance of the autonomous system Data on whether the preset user experience standards are met;

The network optimization process data includes: the start time of the network optimization function, the end time of the network optimization function, the scope of the network optimization function, the number of cells or grids or users that need to perform the network optimization function, and the cells or grids where the root cause is located. or the number of users, or at least one of the cells or grids generated after performing the network optimization function, or the number of users.
The method according to any one of claims 1 to 6, characterized in that the first indicator includes: a quality performance indicator and/or an efficiency performance indicator, and the quality performance indicator is used to characterize the autonomous system The optimization of network performance, the efficiency performance index is used to characterize the optimization efficiency of the network optimization function of the autonomous system.
The method according to claim 7, characterized in that:

The quality performance indicators are determined based on the network performance statistics;

The efficiency performance index is determined based on the network optimization process data.
The method according to claim 7 or 8, characterized in that,

The quality performance indicators include at least one of: poor quality optimization rate, poor experience optimization rate, poor quality performance, or poor experience performance. The poor quality optimization rate is used to characterize the performance of the autonomous system after executing the network optimization function. The network performance, the quality improvement compared with the network performance before the autonomous system performs the network optimization function, the experience difference optimization rate is used to characterize the network performance after the autonomous system performs the network optimization function, and the The user experience improvement compared with the network performance before the system performs the network optimization function. The poor performance is used to characterize whether the network performance after the autonomous system performs the network optimization function reaches the quality standard. The poor experience performance is used to characterize Whether the network performance of the autonomous system after performing the network optimization function meets user experience standards;

The efficiency performance indicators include: the optimization duration of the network optimization function, the optimization scope of the network optimization function, the root cause positioning ratio of the network optimization function, or the optimization ratio of the number of cells or grids or users generated after executing the network optimization function. at least one of.
The method according to claim 9, characterized in that:

The poor quality optimization rate includes: at least one of weak coverage optimization rate, high load optimization rate, low rate optimization rate, or low delay optimization rate;

The poor experience optimization rate includes: at least one of low-rate user optimization rate or low-latency user optimization rate;

The poor performance includes: at least one of: coverage performance after executing the network optimization function, capacity performance after executing the network optimization function, rate performance after executing the network optimization function, or delay performance after executing the network optimization function;

The poor experience performance includes: the proportion of low-speed users after executing the network optimization function, the proportion of high-latency users after executing the network optimization function, the proportion of users with up to standard speed after executing the network optimization function, or the proportion of users who reach the standard rate after executing the network optimization function. At least one of the proportion of users whose latency meets the standard.
A method for determining indicators of the network optimization function of an autonomous system, which is characterized by including:

A first indicator of the autonomous system is received from the first device. The first indicator is used to characterize the performance of the network optimization function of the autonomous system. The performance of the network optimization function at least includes: optimization of the network performance of the autonomous system. situation and/or optimization efficiency of said network optimization functions;

Based on the first indicator, the performance of the network optimization function of the autonomous system is determined.
The method according to claim 11, characterized in that the optimization of the network performance of the autonomous system includes: the network performance after the autonomous system performs the network optimization function, and the network performance before the autonomous system performs the network optimization function. Comparative performance improvement, and specific network performance after the autonomous system performs the network optimization function.
The method according to claim 11 or 12, characterized in that, the method further includes:

Determine a second indicator of the autonomous system according to the first indicator, where the second indicator is used to indicate whether the performance of the network optimization function of the autonomous system meets preset conditions;

Determining the performance of the network optimization function of the autonomous system according to the first indicator includes:

Based on the first indicator and the second indicator, the performance of the network optimization function of the autonomous system is determined.
The method according to claim 13, characterized in that, according to the first indicator, the The second indicator of the autonomous system includes:

Determine the minimum value or weighted value of the first indicator according to the preset indicator benchmark;

The minimum value or the weighted value is determined as the second index.
The method according to claim 13 or 14, characterized in that the method further includes:

receiving the second indicator from the first device;

Determining the performance of the network optimization function of the autonomous system according to the first indicator includes:

Based on the first indicator and the second indicator, the performance of the network optimization function of the autonomous system is determined.
The method according to any one of claims 11 to 15, characterized in that the network performance statistical data includes: coverage performance data, capacity performance data, rate performance data, delay performance data, poor quality data, or poor experience At least one of the data, the quality difference data includes data used to characterize whether the network performance of the autonomous system reaches a preset performance quality standard, and the experience difference data includes data used to characterize the network performance of the autonomous system Data on whether the preset user experience standards are met;

The network optimization process data includes: the start time of the network optimization function, the end time of the network optimization function, the scope of the network optimization function, the number of cells or grids or users that need to perform the network optimization function, and the cells or grids where the root cause is located. or the number of users, or at least one of the cells or grids generated after performing the network optimization function, or the number of users.
The method according to any one of claims 11 to 16, characterized in that the first indicator includes: a quality performance indicator and/or an efficiency performance indicator, and the quality performance indicator is used to characterize the autonomous system The optimization of network performance, the efficiency performance index is used to characterize the optimization efficiency of the network optimization function of the autonomous system.
The method according to claim 17, characterized in that:

The quality performance indicators are determined based on the network performance statistics;

The efficiency performance index is determined based on the network optimization process data.
The method according to claim 17 or 18, characterized in that,

The quality performance indicators include at least one of: poor quality optimization rate, poor experience optimization rate, poor quality performance, or poor experience performance. The poor quality optimization rate is used to characterize the performance of the autonomous system after executing the network optimization function. The network performance, the quality improvement compared with the network performance before the autonomous system performs the network optimization function, the experience difference optimization rate is used to characterize the network performance after the autonomous system performs the network optimization function, and the The user experience improvement compared with the network performance before the system performs the network optimization function. The poor performance is used to characterize whether the network performance after the autonomous system performs the network optimization function reaches the quality standard. The poor experience performance is used to characterize Whether the network performance of the autonomous system after performing the network optimization function meets user experience standards;

The efficiency performance indicators include: the optimization duration of the network optimization function, the optimization scope of the network optimization function, the root cause positioning ratio of the network optimization function, or the optimization ratio of the number of cells or grids or users generated after executing the network optimization function. at least one of.
The method according to claim 19, characterized in that:

The poor quality optimization rate includes: at least one of weak coverage optimization rate, high load optimization rate, low rate optimization rate, or low delay optimization rate;

The poor experience optimization rate includes: at least one of low-rate user optimization rate or low-latency user optimization rate;

The poor performance includes: at least one of: coverage performance after executing the network optimization function, capacity performance after executing the network optimization function, rate performance after executing the network optimization function, or delay performance after executing the network optimization function;

The poor experience performance includes: the proportion of low-speed users after executing the network optimization function, the proportion of high-latency users after executing the network optimization function, the proportion of users with up to standard speed after executing the network optimization function, or the proportion of users who reach the standard rate after executing the network optimization function. At least one of the proportion of users whose latency meets the standard.
A method for determining indicators of the network optimization function of an autonomous system, characterized in that it is applied to an indicator determination system of the network optimization function of an autonomous system, and the system includes: a first device and a second device; the method includes:

The first device obtains first data of the autonomous system, where the first data includes: network performance statistical data and/or network optimization process data;

The first device determines a first indicator of the autonomous system based on the first data. The first indicator is used to characterize the performance of the network optimization function of the autonomous system. The performance of the network optimization function at least includes : The optimization of the network performance of the autonomous system and/or the optimization efficiency of the network optimization function;

The first device sends the first indicator to the second device;

The second device determines the performance of the network optimization function of the autonomous system based on the first indicator.
The method according to claim 21, characterized in that the optimization of the network performance of the autonomous system includes: the network performance after the autonomous system performs the network optimization function, and the network performance before the autonomous system performs the network optimization function. Comparative performance improvement, and specific network performance after the autonomous system performs the network optimization function.
The method according to claim 21 or 22, characterized in that, the method further includes:

The first device determines a second indicator of the autonomous system based on the first indicator, and the second indicator is used to indicate whether the performance of the network optimization function of the autonomous system meets preset conditions;

The first device sends the second indicator to the second device;

The second device determines the performance of the network optimization function of the autonomous system based on the first indicator, including:

The second device determines the performance of the network optimization function of the autonomous system based on the first indicator and the second indicator.
The method according to claim 21 or 22, characterized in that, the method further includes:

The second device determines a second indicator of the autonomous system based on the first indicator, and the second indicator is used to indicate whether the performance of the network optimization function of the autonomous system meets preset conditions;

The second device determines the performance of the network optimization function of the autonomous system based on the first indicator, including:

The second device determines the performance of the network optimization function of the autonomous system based on the first indicator and the second indicator.
The method according to claim 23 or 24, characterized in that determining the second indicator of the autonomous system according to the first indicator includes:

Determine the minimum value or weighted value of the first indicator according to the preset indicator benchmark;

The minimum value or the weighted value is determined as the second index.
The method according to any one of claims 21 to 25, characterized in that the network performance statistical data includes: coverage performance data, capacity performance data, rate performance data, delay performance data, poor quality data, or poor experience At least one of the data, the quality difference data includes data used to characterize whether the network performance of the autonomous system reaches a preset performance quality standard, and the experience difference data includes data used to characterize the network performance of the autonomous system Data on whether the preset user experience standards are met;

The network optimization process data includes: the start time of the network optimization function, the end time of the network optimization function, the scope of the network optimization function, the number of cells or grids or users that need to perform the network optimization function, and the cells or grids where the root cause is located. or the number of users, or at least one of the cells or grids generated after performing the network optimization function, or the number of users.
The method according to any one of claims 21 to 26, characterized in that the first indicator includes: a quality performance indicator and/or an efficiency performance indicator, and the quality performance indicator is used to characterize the autonomous system The optimization of network performance, the efficiency performance index is used to characterize the optimization efficiency of the network optimization function of the autonomous system.
The method according to claim 27, characterized in that:

The quality performance indicators are determined based on the network performance statistics;

The efficiency performance index is determined based on the network optimization process data.
The method according to claim 27 or 28, characterized in that,

The quality performance indicators include at least one of: poor quality optimization rate, poor experience optimization rate, poor quality performance, or poor experience performance. The poor quality optimization rate is used to characterize the performance of the autonomous system after executing the network optimization function. The network performance, the quality improvement compared with the network performance before the autonomous system performs the network optimization function, the experience difference optimization rate is used to characterize the network performance after the autonomous system performs the network optimization function, and the The user experience improvement compared with the network performance before the system performs the network optimization function. The poor performance is used to characterize whether the network performance after the autonomous system performs the network optimization function reaches the quality standard. The poor experience performance is used to characterize Whether the network performance of the autonomous system after performing the network optimization function meets user experience standards;

The efficiency performance indicators include: the optimization duration of the network optimization function, the optimization scope of the network optimization function, the root cause positioning ratio of the network optimization function, or the optimization ratio of the number of cells or grids or users generated after executing the network optimization function. at least one of.
The method according to claim 29, characterized in that:

The poor quality optimization rate includes: at least one of weak coverage optimization rate, high load optimization rate, low rate optimization rate, or low delay optimization rate;

The poor experience optimization rate includes: at least one of low-rate user optimization rate or low-latency user optimization rate;

The poor performance includes: at least one of: coverage performance after executing the network optimization function, capacity performance after executing the network optimization function, rate performance after executing the network optimization function, or delay performance after executing the network optimization function;

The poor experience performance includes: the proportion of low-speed users after executing the network optimization function, the proportion of high-latency users after executing the network optimization function, the proportion of users with up to standard speed after executing the network optimization function, or the proportion of users who reach the standard rate after executing the network optimization function. At least one of the proportion of users whose latency meets the standard.
An indicator determination device for the network optimization function of an autonomous system, characterized in that it is applied to the first device Equipment, the device comprising: a module for performing the method according to any one of claims 1-10.
An indicator determination device for the network optimization function of an autonomous system, characterized in that it is applied to a second device, and the device includes: a module for executing the method according to any one of claims 11-20.
An indicator determination system for the network optimization function of an autonomous system, characterized by comprising: an autonomous system, and an indicator determination device for the network optimization function of an autonomous system as claimed in claim 31; or, an autonomous system, as claimed in claim 31 The indicator determination device for the network optimization function of the autonomous system and the indicator determination device for the network optimization function of the autonomous system as claimed in claim 32.
An indicator determination system for the network optimization function of an autonomous system, characterized in that it includes: a first device and a second device for executing the method according to any one of claims 21-30.
A communication device, characterized by including: a memory and a processor;

The memory is used to store program instructions;

The processor is configured to call program instructions in the memory to cause the communication device to execute the method described in any one of claims 1-10, and/or to cause the communication device to execute any one of claims 11-20. the method described.
A computer-readable storage medium, characterized by comprising computer instructions, which when the computer instructions are run on a communication device, cause the communication device to perform the method according to any one of claims 1-10, and/ Or, causing the communication device to perform the method according to any one of claims 11-20.
A chip, characterized in that it includes: an interface circuit and a logic circuit. The interface circuit is used to receive signals from other chips other than the chip and transmit them to the logic circuit, or to transmit signals from the logic circuit. Sent to other chips other than the chip, the logic circuit is used to implement the method as described in any one of claims 1-10; and/or, implement the method as described in any one of claims 11-20 .
A computer program product, characterized in that, when the computer program product is run on a computer, it causes the computer to execute the method according to any one of claims 1 to 10, and/or causes the computer to execute The method according to any one of claims 11-20.