WO2024146592A1

WO2024146592A1 - Poor-quality identification method and related apparatus

Info

Publication number: WO2024146592A1
Application number: PCT/CN2024/070535
Authority: WO
Inventors: 李昂; 蔡静宜; 彭斌; 王宇辰
Original assignee: 华为技术有限公司
Priority date: 2023-01-05
Filing date: 2024-01-04
Publication date: 2024-07-11
Also published as: CN118301013A

Abstract

Embodiments of the present application provide a poor-quality identification method and a related apparatus. The method comprises: a first apparatus analyzes key performance indicator (KPI) data of a user according to a first poor-quality identification model, so as to obtain an edge side poor-quality identification result and a confidence level corresponding to the edge side poor-quality identification result, wherein the confidence level is used for indicating the accuracy represented by the edge side poor-quality identification result; and the first apparatus may send to a second apparatus KPI data corresponding to a first confidence level in the confidence level, and may also send to the second apparatus a second poor-quality identification result corresponding to a second confidence level in the confidence level, wherein the accuracy of the first poor-quality identification result represented by the first confidence level is lower than the accuracy of the second poor-quality identification result represented by the second confidence level, and the first poor-quality identification result and the second poor-quality identification result belong to the edge side poor-quality identification result. By using the embodiments of the present application, a transmission bandwidth can be reduced without affecting the accuracy of poor-quality identification, and the communication cost is reduced.

Description

Poor quality identification method and related device

This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office of China on January 5, 2023, with application number 202310011197.8 and application name “Quality Difference Identification Method and Related Device”, and claims the priority of the Chinese patent application filed with the State Intellectual Property Office of China on March 21, 2023, with application number 202310283247.8 and application name “Quality Difference Identification Method and Related Device”, the entire contents of which are incorporated by reference in this application.

Technical Field

The present application relates to the field of Internet application technology, and in particular to a quality difference identification method and related devices.

Background technique

Home broadband networks are rapidly developing from digitalization to intelligence, and have expanded from providing basic services such as Internet access and voice to all aspects of users' work and life. According to reports, more than 510 million users in China use online services such as remote office, video browsing, and gaming entertainment through home broadband. Many services such as high-definition video, cloud gaming, enhanced display, and online video conferencing have high real-time requirements. Poor network connection quality or insufficient bandwidth can cause problems such as application freezes or delays, thereby affecting users' service experience. Operators need labels for home network quality to measure the network service experience on the user side and carry out corresponding services based on this. For example: increase the mining of potential customers, improve customer network security, and actively identify poor-quality users to improve user experience.

In order to measure the user's network quality, it is proposed to use a collection device to collect the user's application KPI data and report it in full to the computing device, which then uses a neural network model to identify whether the application used by the user has poor quality. The poor quality identification result can be used as a basis for evaluating the user's network quality, so as to proactively carry out subsequent operations or bandwidth upgrades for the user.

However, the amount of user application KPI data is large. When identifying poor quality, reporting all KPI data to the computing device for calculation will lead to high data communication and computing costs, high performance pressure, and cannot effectively support operators in carrying out corresponding businesses.

Summary of the invention

The embodiments of the present application provide a quality difference identification method and related devices, which can reduce the transmission bandwidth and communication costs without affecting the accuracy of quality difference identification.

In a first aspect, an embodiment of the present application provides a method for identifying quality differences, including:

The first device analyzes the user's key performance indicator KPI data according to the first quality difference identification model to obtain an edge side quality difference identification result and a confidence level corresponding to the edge side quality difference identification result, wherein the confidence level is used to indicate the degree of accuracy represented by the edge side quality difference identification result; the first device sends the KPI data corresponding to the first confidence level in the confidence level to the second device; the first device sends the second quality difference identification result corresponding to the second confidence level in the confidence level to the second device, wherein the accuracy of the first quality difference identification result represented by the first confidence level is lower than the accuracy of the second quality difference identification result represented by the second confidence level, and the first quality difference identification result and the second quality difference identification result belong to the edge side quality difference identification result.

It is easy to understand that KPI data contains various KPI indicators and has a large amount of data, while the data contained in the quality difference identification result obtained after analyzing the KPI data is relatively small. The complexity of the KPI data corresponding to the first confidence level is higher than that of the KPI data corresponding to the second confidence level. Therefore, sending the KPI data corresponding to the first confidence level to the second device and sending the quality difference result corresponding to the second confidence level to the second device can reduce the data communication cost without affecting the accuracy of the quality difference identification result.

In a possible implementation of the first aspect, the embodiment of the present application can be applied to a cloud-edge-end collaborative system, which includes a cloud, an edge-side device and a terminal, wherein the first device is a device deployed on the edge side, and the second device is a device deployed on the cloud. For the edge-side device, which also has computing power, it can use the computing power of the edge side to perform some less complex calculations, thereby reducing the transmission bandwidth. When cloud resources are limited, the consumption of cloud computing resources can be reduced.

In a possible implementation manner of the first aspect, before the first device sends a second quality difference identification result corresponding to a second confidence level in the confidence level to the second device, the method further includes:

The first device sets a data reporting method through an interface, and the reporting method includes a confidence threshold and a data ratio.

It can be seen that the present application can provide two data reporting methods, so the reporting method can be set through the interface according to the actual application, and the setting method is flexible and changeable.

In a possible implementation of the first aspect, when the reporting method is the confidence threshold, the first confidence is less than the confidence threshold, and the second confidence is greater than or equal to the confidence threshold.

It can be seen that the reporting method of the confidence threshold can make the amount of reported data change dynamically, and there will be no underreporting of low-confidence data, which can ensure the accuracy of poor quality identification.

In a possible implementation of the first aspect, when the reporting method is the data ratio, the first confidence is a confidence that is less than the data ratio in the confidence ranking, the second confidence is a confidence that is greater than or equal to the data ratio in the confidence ranking, and the confidence ranking is a numerical value corresponding to the confidence obtained by sorting the confidences corresponding to the edge side recognition results in ascending order.

It can be seen that the data ratio reporting method can ensure the constancy of the reported data volume, so that the second device can receive a sufficient amount of data.

In a possible implementation manner of the first aspect, before the first device analyzes the key performance indicator KPI data of the user according to the first quality difference identification model to obtain the edge side quality difference identification result and the confidence level corresponding to the edge side quality difference identification result, the method further includes:

The first device receives the first quality difference identification model from the second device, where the first quality difference identification model is trained by the second device.

It can be seen that, since the computing resources of the first device are limited, the model training can be performed on other devices, and only the trained model needs to be deployed on the first device. In this way, the computing resources of the first device can be fully utilized for quality difference identification.

In a second aspect, an embodiment of the present application provides a method for identifying quality differences, including:

The second device receives KPI data corresponding to the first confidence level from the first device; the second device receives a second quality difference identification result corresponding to the second confidence level from the first device, wherein the accuracy of the first quality difference identification result represented by the first confidence level is lower than the accuracy of the second quality difference identification result represented by the second confidence level;

The second device analyzes the KPI data corresponding to the first confidence level according to the second quality difference identification model to obtain a cloud quality difference identification result;

The second device obtains the quality difference identification result of the user according to the second quality difference identification result and the cloud quality difference identification result.

In a possible implementation of the second aspect, the first device is a device deployed on an edge side, and the second device is a device deployed on a cloud side.

In a possible implementation of the second aspect, the first confidence is less than a confidence threshold, and the second confidence is greater than or equal to the confidence threshold, wherein the confidence threshold is a reporting method set by the first device.

In a possible implementation of the second aspect, the first confidence is less than the data ratio in the confidence ranking, the second confidence is greater than or equal to the data ratio in the confidence ranking, and the confidence ranking is a numerical value corresponding to the confidence obtained by sorting the confidences in ascending order, wherein the confidence is the confidence corresponding to the edge-side quality difference identification result obtained after the first device analyzes the user's KPI data according to the first quality difference identification model, and the data ratio is the reporting method set by the first device.

In a possible implementation manner of the second aspect, the second device obtains a quality difference identification result of the user according to the second quality difference identification result and the cloud quality difference identification result, including:

The second device counts the quality difference results in the second quality difference identification result and the cloud quality difference identification result, and the quality difference results are used to indicate that the network situation reflected by the KPI data corresponding to the quality difference results is poor;

When the number of the poor quality results is greater than a preset threshold, the users corresponding to the poor quality results are poor quality users.

In a possible implementation manner of the second aspect, before the second device sends the first quality difference identification model to the first device, the further step includes:

The second device obtains the first quality difference recognition model and the second quality difference recognition model through training according to historical KPI data, wherein the recognition accuracy of the first quality difference recognition model is lower than the recognition accuracy of the second quality difference recognition model.

In a possible implementation manner of the second aspect, after the second device obtains the first quality difference identification model and the second quality difference identification model through training according to historical KPI data, the second device further includes:

The second device sends the first quality difference identification model to the first device, wherein the first device is used to obtain the edge side quality difference identification result and the confidence corresponding to the edge side quality difference identification result according to the first quality difference identification model, and the confidence is used to indicate the accuracy represented by the edge side quality difference result, and the confidence includes the first confidence and the second confidence.

In a third aspect, an embodiment of the present application provides a quality difference identification system, which includes a first device and a second device, wherein the first device is used to execute the method described in any one of the first aspects, and the second device is used to execute the method described in any one of the second aspects.

In a possible implementation manner of the third aspect, the first device is used to:

Analyzing the user's key performance indicator KPI data according to the first quality difference identification model to obtain an edge side quality difference identification result and a confidence level corresponding to the edge side quality difference identification result, wherein the confidence level is used to indicate the accuracy represented by the edge side quality difference identification result;

Sending KPI data corresponding to a first confidence level among the confidence levels to a second device;

Sending a second quality difference identification result corresponding to a second confidence level in the confidence level to the second device, wherein the accuracy of the first quality difference identification result represented by the first confidence level is lower than the accuracy of the second quality difference identification result represented by the second confidence level, and the first quality difference identification result and the second quality difference identification result belong to the edge-side quality difference identification result;

The second device is used for:

Receiving KPI data corresponding to a first confidence level from a first device;

receiving a second quality difference identification result corresponding to a second confidence level from the first device;

Analyze the KPI data corresponding to the first confidence level according to the second quality difference identification model to obtain a cloud quality difference identification result;

A quality difference identification result for the user is obtained according to the second quality difference identification result and the cloud quality difference identification result.

In another possible implementation of the third aspect, the first device is a device deployed on an edge side, and the second device is a device deployed on a cloud side.

In another possible implementation of the third aspect, the first device is further used to: set a data reporting method through an interface, where the reporting method includes a confidence threshold and a data ratio.

In another possible implementation of the third aspect, when the reporting method is the confidence threshold, the first confidence is less than the confidence threshold, and the second confidence is greater than or equal to the confidence threshold.

In another possible implementation of the third aspect, when the reporting method is the data ratio, the first confidence is a confidence that is less than the data ratio in the confidence ranking, the second confidence is a confidence that is greater than or equal to the data ratio in the confidence ranking, and the confidence ranking is a numerical value corresponding to the confidence obtained by sorting the confidences corresponding to the edge side recognition results in ascending order.

In yet another possible implementation manner of the third aspect, the second device is further used for:

The first quality difference recognition model and the second quality difference recognition model are obtained by training according to historical KPI data, wherein the recognition accuracy of the first quality difference recognition model is lower than the recognition accuracy of the second quality difference recognition model;

The first quality difference identification model is sent to the first device.

Counting the quality difference results in the second quality difference identification result and the cloud quality difference identification result, wherein the quality difference results are used to indicate that the network situation reflected by the KPI data corresponding to the quality difference results is poor;

In a fourth aspect, an embodiment of the present application provides a computing device, the computing device comprising a processing module and a communication module, wherein:

The processing module is used to analyze the key performance indicator KPI data of the user according to the first quality difference identification model to obtain the edge side quality difference identification result and the confidence corresponding to the edge side quality difference identification result, wherein the confidence is used to indicate the accuracy represented by the edge side quality difference identification result;

The communication module is used to send KPI data corresponding to a first confidence level among the confidence levels to the second device;

The communication module is also used to send a second quality difference identification result corresponding to a second confidence level in the confidence level to the second device, wherein the accuracy of the first quality difference identification result represented by the first confidence level is lower than the accuracy of the second quality difference identification result represented by the second confidence level, and the first quality difference identification result and the second quality difference identification result belong to the edge side quality difference identification result.

In another possible implementation of the fourth aspect, the computing device is a device deployed on an edge side, and the second device is a device deployed on a cloud side.

In yet another possible implementation manner of the fourth aspect, the processing module is further used to:

The data reporting method is set through the interface, and the reporting method includes a confidence threshold and a data ratio.

In another possible implementation of the fourth aspect, when the reporting method is the confidence threshold, the first confidence is less than the confidence threshold, and the second confidence is greater than or equal to the confidence threshold.

In another possible implementation of the fourth aspect, when the reporting method is the data ratio, the first confidence is a confidence that is less than the data ratio in the confidence ranking, the second confidence is a confidence that is greater than or equal to the data ratio in the confidence ranking, and the confidence ranking is obtained by sorting the confidences corresponding to the edge side recognition results in order from small to large. The value corresponding to the confidence level.

In yet another possible implementation of the fourth aspect, the communication module is further used to:

The first quality difference recognition model is received from the second device, where the first quality difference recognition model is obtained by training the second device. The computing device includes a processing module and a communication module, wherein:

The communication module is used to receive KPI data corresponding to the first confidence level from the first device;

The communication module is further configured to receive a second quality difference identification result corresponding to a second confidence level from the first device, wherein the accuracy of the first quality difference identification result represented by the first confidence level is lower than the accuracy of the second quality difference identification result represented by the second confidence level;

The processing module is used to analyze the KPI data corresponding to the first confidence level according to the second quality difference identification model to obtain a cloud quality difference identification result;

The processing module is further used to obtain the user's quality difference identification result according to the second quality difference identification result and the cloud quality difference identification result.

In a fifth aspect, an embodiment of the present application provides a computing device, the computing device comprising a processing module and a communication module, wherein:

In another possible implementation of the fifth aspect, the first device is a device deployed on the edge side, and the computing device is a device deployed on the cloud.

In another possible implementation of the fifth aspect, the first confidence is less than a confidence threshold, and the second confidence is greater than or equal to the confidence threshold, wherein the confidence threshold is a reporting method set by the first device.

In another possible implementation of the fifth aspect, the first confidence is less than the data ratio in the confidence ranking, the second confidence is greater than or equal to the data ratio in the confidence ranking, and the confidence ranking is a numerical value corresponding to the confidence obtained by sorting the confidences in ascending order, wherein the confidence is the confidence corresponding to the edge-side quality difference identification result obtained after the first device analyzes the user's KPI data according to the first quality difference identification model, and the data ratio is the reporting method set by the first device.

In yet another possible implementation manner of the fifth aspect, the processing module is specifically configured to:

In yet another possible implementation manner of the fifth aspect, the processing module is further used to:

The first quality difference recognition model and the second quality difference recognition model are obtained by training according to historical KPI data, wherein the recognition accuracy of the first quality difference recognition model is lower than the recognition accuracy of the second quality difference recognition model.

In yet another possible implementation of the fifth aspect, the communication module is further used to:

The first quality difference identification model is sent to the first device, wherein the first device is used to obtain an edge side quality difference identification result and a confidence level corresponding to the edge side quality difference identification result according to the first quality difference identification model, the confidence level is used to indicate the accuracy represented by the edge side quality difference result, and the confidence level includes the first confidence level and the second confidence level.

In a sixth aspect, an embodiment of the present application provides a computing device, comprising a processor and a memory; the processor is used to execute instructions stored in the memory so that the computing device implements the method described in any one of the first aspect or the method described in any one of the second aspect.

Optionally, the computing device further comprises a communication interface, wherein the communication interface is used to receive and/or send data, and/or the communication interface is used to provide input and/or output for the processor.

It should be noted that the above embodiment is described by taking a processor (or general-purpose processor) that executes the method by calling a computer specification as an example. In the specific implementation process, the processor can also be a dedicated processor, in which case the computer instructions have been pre-loaded in the processor. Optionally, the processor can also include both a dedicated processor and a general-purpose processor.

Optionally, the processor and the memory may also be integrated into one device, that is, the processor and the memory may also be integrated together.

In a seventh aspect, an embodiment of the present application further provides a computing device cluster, the computing device cluster comprising at least one computing device, each computing device comprising a processor and a memory;

The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method described in any one of the first aspect or the method described in any one of the second aspect.

In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium. When the instructions are executed on at least one processor, the method described in any one of the first aspect or the method described in any one of the second aspect is implemented.

In a ninth aspect, the present application provides a computer program product, which includes computer instructions. When the instructions are executed on at least one processor, the method described in any one of the first aspect or the method described in any one of the second aspect is implemented.

Optionally, the computer program product may be a software installation package or an image package. When the aforementioned method is required, the computer program product may be downloaded and executed on a computing device.

The beneficial effects of the technical solutions provided in the second to ninth aspects of the present application can refer to the beneficial effects of the technical solution of the first aspect, and will not be repeated here.

BRIEF DESCRIPTION OF THE DRAWINGS

The following is an introduction to the drawings used in the embodiments of the present application.

FIG1 is a schematic diagram of quality difference identification in the cloud provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of the architecture of a quality difference identification system 10 provided in an embodiment of the present application;

FIG3 is a schematic diagram of the architecture of a quality difference identification system 20 applied to a passive optical fiber network provided in an embodiment of the present application;

FIG4 is a schematic diagram of a possible quality difference identification method provided in an embodiment of the present application;

FIG5 is a flow chart of another quality difference identification method provided in an embodiment of the present application;

FIG6 is a schematic diagram of the structure of a computing device 60 provided in an embodiment of the present application;

FIG. 7 is a schematic diagram showing the structure of a computing device 70 provided in an embodiment of the present application.

Detailed ways

The embodiments of the present invention are described below in conjunction with the drawings in the embodiments of the present invention. It should be noted that in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs, and the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

For ease of understanding, the following examples provide some concepts related to the embodiments of the present application for reference, as shown below:

1. Data sampling, also known as data acquisition, is an interface that uses a device to collect data from outside the system and input it into the system.

2. Low quality refers to poor network quality, including playback freezes, slow upload speeds, slow download speeds, etc. Low-quality users refer to users who are dissatisfied with the service experience when using mobile communication network services due to network quality issues or other factors.

3. Neural network, composed of neural units, which can refer to a computing unit with x _s and intercept b as input, and the output of the computing unit can be:

Where s=1, 2, ...n, n is a natural number greater than 1, _ws is the weight of _xs , and b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolutional layer. The activation function can be a sigmoid function. A neural network is a network formed by connecting many of the above-mentioned single neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the characteristics of the local receptive field. The local receptive field can be an area composed of several neural units.

4. A fully connected layer is a neural network layer in which every node is connected to all nodes in the previous layer. A fully connected network means that for both n-1 and n layers, any node (also called a neuron) in the n-1 layer is connected to all nodes in the n layer.

5. Flow refers to encapsulated data transmitted in a layered network. For example, a one-way message flow transmitted between a source IP address and a destination IP address within a period of time. Flows are divided into two types: text flow and binary flow.

6. Passive optical network (PON) is a point-to-multipoint optical fiber transmission and access technology. A PON system can include an optical line terminal (OLT), an optical distribution network (ODN) and multiple optical network units (ONUs).

OLT is an optical line terminal, a telecommunications central office device used to connect to the network backbone and is a device for external and internal network entrances. It is placed at the central office and is used for traffic scheduling, buffer control, and providing user-oriented passive optical network interfaces and bandwidth allocation.

ONU is used to connect to the local area network or home users, selectively receive the broadcasts sent by the OLT, and collect and cache the data that the users need to send.

The so-called "passive" means that the ODN does not contain any active electronic devices or electronic power supplies, and is entirely composed of passive devices such as optical splitters.

The technical solution of the embodiments of the present application can be applied to various passive optical network (PON) systems, for example, next generation PON (NG-PON), NG-PON1, NG-PON2, gigabit capable PON (GPON), 10 gigabit per second PON (XG-PON), symmetric 10 gigabit passive optical network (XGS-PON), Ethernet PON (EPON), 10 gigabit per second EPON (10 gigabit per second EPON, 10G EPON), next generation EPON (NG-EPON), wavelength division multiplexing (WDM) PON, time and wavelength division stacking multiplexing (TWDM) PON, and other systems. Vision multiplexing, TWDM) PON, point to point (point to point, P2P) WDMPON (P2P-WDMPON), asynchronous transfer mode PON (asynchronous transfer mode PON, APON), broadband PON (broadband PON, BPON), etc., as well as 25 gigabit per second PON (25G-PON), 50 gigabit per second PON (50gigabitpersecondPON, 50G-PON), 100 gigabit per second PON (100gigabitpersecondPON, 100G-PON), 25 gigabit per second EPON (25gigabitpersecondEPON, 25G-EPON), 50 gigabit per second EPON (50gigabitpersecondEPON, 50G-EPON), 100 gigabit per second EPON (100gigabitpersecondEPON, 100G-EPON), and GPON, EPON of other rates.

Please refer to Figure 1, which is a schematic diagram of quality difference identification in the cloud provided by an embodiment of the present application. Among them, the first device 102 can be a device installed on the optical line terminal OLT101, specifically, the first device 102 can be a single board. The second device 103 can be located in the cloud, specifically, the second device 103 can be a server, and the server is equipped with a network performance management system, referred to as a network management system. As can be seen from Figure 1, OLT101 can forward the received upstream and downstream traffic to the first device 102. Among them, the upstream and downstream traffic is the user's traffic. The first device 102 can filter the upstream and downstream traffic according to the granularity of the PON port, thereby filtering the upstream and downstream traffic to obtain the filtered upstream and downstream traffic. The second device 103 located in the cloud can set the sampling frequency and send the sampling frequency to the first device 102. The first device 102 can sample the filtered uplink and downlink traffic according to the sampling frequency, and then perform application identification on the sampled uplink and downlink processes, and then perform application KPI analysis to obtain KPI data. The first device 102 sends the KPI data to the second device 103 located in the cloud. The second device 103 can use a neural network model to identify quality differences of the KPI data based on a deep learning algorithm to determine poor quality users.

It is easy to understand that users with poor quality caused by network quality problems will have lower satisfaction with network services, and may complain, switch networks, and so on. The probability of loss of poor quality user groups is high, but they are also potential customers that can be mined. Operators generally mine potential customers by carrying label data with home network quality, improving customer experience, and identifying poor quality users to reduce the complaint rate. However, the premise of the above business is to ensure the accuracy of poor quality identification. As can be seen from Figure 1, the current poor quality identification method is that the first device reports the KPI data of each application in full, and then uses a deep learning algorithm in the cloud to perform poor quality identification to obtain poor quality results. Since the poor quality identification method shown in Figure 1 requires that all KPI data of the application be uploaded to the cloud, the bandwidth resource requirements for data transmission are high. Using the full amount of data for poor quality identification in the cloud results in a large amount of concurrent input of the neural network model, which brings high computing cost and performance pressure. The applicant found that the KPI data of some applications are normal data, which can be accurately identified using a lightweight neural network model. However, the current poor quality identification method is basically to use a high-precision model for calculation in the cloud, so that the edge computing power cannot be fully utilized. This results in high consumption and waste of computing resources in the cloud. Therefore, under the condition of limited resources, the method shown in FIG1 cannot guarantee the accuracy of poor quality identification of users.

The embodiment of the present application provides a quality difference identification method and related devices. First, the first stage quality difference identification is performed by the first device to obtain the edge side quality difference identification result and the confidence corresponding to the edge side quality difference identification result. Then the first device sends the KPI data corresponding to the first confidence in the confidence or the second quality difference identification result corresponding to the second confidence in the confidence to the second device. Among them, the accuracy of the first quality difference identification result represented by the first confidence is lower than the accuracy of the second quality difference identification result represented by the second confidence, and the first quality difference identification result and the second quality difference identification result belong to the edge side quality difference identification result. Then, the second stage quality difference identification is performed by the second device, and the KPI data that cannot be prepared for identification is further judged, and the quality difference identification result can be finally obtained. The second device can send the quality difference identification result to the operator, and the operator can carry out operation and maintenance for the quality difference user based on the quality difference identification result. By coordinating the quality difference identification by the first device and the second device, the data communication and computing costs can be reduced, thereby achieving the effect of reducing the bandwidth and computing resource requirements without affecting the accuracy of the quality difference identification.

The following is an introduction to the system architecture of the embodiment of the present application. It should be noted that the system architecture and business scenarios described in this application are intended to more clearly illustrate the technical solution of this application, and do not constitute a limitation on the technical solution provided by this application. It is known to those skilled in the art that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution provided by this application is also applicable to similar technical problems.

Please refer to Figure 2, which is a schematic diagram of the architecture of a quality difference identification system 10 provided in an embodiment of the present application. The quality difference identification system 10 includes a second device 110, a first device 120 and a user device 130. Among them, the number of devices can be 1 or more. Figure 1 is for the convenience of description, taking a first device 120 and a user device 130 as an example, and the present application is also applicable to quality difference identification systems including other numbers of devices.

The second device 110 can be a cloud, which is also called a server or a cloud platform, and is a device with computing and communication capabilities. It usually concentrates more computing resources, which can include physical devices or virtual devices. For example, the second device 110 can be deployed in one or more servers (such as blade servers, rack servers, etc.). For another example, the cloud can include one or more virtual machines, containers, etc. The second device 110 can provide a variety of resources for the first device 120, such as computing resources, game resources, live broadcast resources, and on-demand resources. The second device 110 can also provide a variety of services for the user device 130, such as quality difference identification results, etc.

The first device 120 is a device with computing and/or communication capabilities. As a possible approach, the first device 120 may be an edge device. The edge is a device that provides a network entry point, for example, the device may include one or more of a router, a routing switch, an integrated access device (IAD), a multiplexer, a metropolitan area network (MAN) access device, or a wide area network (WAN) access device.

The user device 130 is a device with computing and/or communication capabilities, and may be a device used by an operator. As a possible approach, the user device 130 is generally used as an input and/or output device, and may be applied to a variety of application scenarios. For example, the device in the embodiment of the present application may be applied to a variety of application scenarios, such as the following application scenarios: smart buildings, mobile Internet, Internet of Vehicles, industrial control, unmanned driving, transportation safety, Internet of Things (IoT), smart cities, or smart homes, etc. For example, the device includes but is not limited to computers, handheld devices (such as tablets, or mobile phones, etc.), wearable devices (smart glasses, or smart watches, etc.), sensors (such as cameras, laser radars, or millimeter wave radars, etc.), home appliances (smart TVs, smart refrigerators, or smart security equipment, etc.), entertainment equipment (song request equipment, virtual reality equipment, or game consoles, etc.), mobile tools (vehicles, aircraft, ships, or drones, etc.), etc.

In an embodiment of the present application, the second device 110 can train a first quality difference identification model and a second quality difference identification model, and can send the second quality difference identification model to the first device 120. The first device 120 can obtain the user's key performance indicator KPI data from the data stream transmitted by itself, and analyze the KPI data according to the first quality difference identification model to obtain the edge side quality difference identification result and the confidence corresponding to the edge side quality difference identification result. Among them, the confidence is used to indicate the accuracy represented by the edge side quality difference identification result. It can be understood that the higher the confidence, the higher the credibility of the edge side quality difference identification result; the lower the confidence, the lower the credibility of the edge side quality difference identification result. Therefore, the first device 120 can send the KPI data corresponding to the first confidence in the confidence or the second quality difference identification result corresponding to the second confidence in the confidence to the second device 110. Among them, the accuracy of the first quality difference identification result represented by the first confidence level is lower than the accuracy of the second quality difference identification result represented by the second confidence level, and the first quality difference identification result and the second quality difference identification result belong to the edge side quality difference identification results.

After the second device 110 receives the KPI data corresponding to the first confidence level or the second quality difference identification result corresponding to the second confidence level from the first device, the second device 110 can analyze the KPI data corresponding to the first confidence level according to the second quality difference identification model to obtain the cloud quality difference identification result. Then, the second device 110 can obtain the user's quality difference identification result according to the second quality difference identification result and the cloud quality difference identification result. Finally, the second device 110 can send the quality difference identification result to the user equipment 130, and the operator can perform operation and maintenance on the poor quality user based on the quality difference identification result.

In an embodiment of the present application, the computing power of the edge side and the computing power of the cloud side are combined to process KPI data of different recognition difficulties in stages, and the transmission bandwidth and cloud computing resource requirements are rewarded without affecting the accuracy of quality difference recognition, thereby reducing resource waste.

Please refer to Figure 3, which is a schematic diagram of the architecture of a quality difference identification system 20 applied to a passive optical network provided in an embodiment of the present application. As shown in Figure 3, the quality difference identification system 20 includes at least one OLT121, at least one ODN122, multiple ONU123 and a cloud 110. Among them, OLT121 is an operator-side device, which can be the first device shown in Figure 2, providing a network-side interface for the quality difference identification system 20, and allocating network data of upper-layer services to the user side through ODN122. ONU123 is a user-side device, which can be the user device shown in Figure 2, providing a user-side interface for the quality difference identification system 20, and connected to ODN122 for receiving data sent by OLT121. ODN122 is a network composed of optical fibers and passive optical splitters, used to connect OLT121 devices and ONU123 devices, and used to distribute or multiplex data signals between OLT121 and ONU123. The cloud 110 is equipped with an intelligent management and control platform, which may be the second device shown in FIG. 2 . The intelligent management platform processes and analyzes the received data, and pushes the analysis results to the user-side APP to support the operator in conducting business.

ODN122 includes feeder optical fiber, optical splitter and branch line, which are respectively composed of different passive optical devices. The main passive optical devices include: single-mode optical fiber and optical cable, optical fiber ribbon and ribbon cable, optical connector, passive optical splitter, passive optical attenuator and optical fiber interface, etc.

In the quality difference identification system 20, the direction from OLT121 to ONU123 is defined as the downstream direction, and the direction from ONU123 to OLT121 is defined as the upstream direction. In the downstream direction, OLT121 uses time division multiplexing (TDM) to broadcast downstream data to multiple ONU123 managed by the OLT121, and each ONU123 only accepts data carrying its own identification. In the upstream direction, multiple ONU123 use time division multiple access (TMDA) to communicate with OLT121. Each ONU123 sends upstream data according to the time domain resources allocated to it by OLT121. Using the above mechanism, the downstream optical signal sent by OLT121 is a continuous optical signal, and the upstream optical signal sent by ONU123 is a burst optical signal.

The OLT 121 is usually located in a central office (CO), and can centrally manage at least one ONU 123 and transmit data between the ONU 123 and the upper network. Specifically, the OLT 121 can act as a medium between the ONU 123 and the upper network (such as the Internet, the public switched circuit network), forwarding the data received from the upper network to the ONU 123, and forwarding the data received from the ONU 123 to the upper network. The specific configuration of the OLT 121 may vary depending on the specific type of the PON system. For example, in one embodiment, the OLT 121 may include a transmitter and a receiver, the transmitter is used to send a downstream continuous optical signal to the ONU 123, and the receiver is used to receive an upstream burst optical signal from the ONU 123, wherein the downstream optical signal and the upstream optical signal can be transmitted through the ODN 122, but the embodiment of the present application is not limited thereto.

The OLT 121 may include a transportation management (TM) module, a media access control (MAC) module, and a central processing unit (CPU), etc. The TM module, MAC module, and CPU, etc. may be integrated on a first device, which may specifically be a single board or a chip, such as a system-on-chip (SOC) chip, etc. The single board may collect KPI data of each ONU 123 attached to the OLT 121 in real time.

The ONU123 can be distributedly arranged at the user side location (such as the user's premises). The ONU123 can be a network device for communicating with the OLT121 and the user. Specifically, the ONU123 can act as a medium between the OLT121 and the user. For example, the ONU123 can forward the data received from the OLT121 to the user, and forward the data received from the user to the OLT121. If the ONU123 specifically and directly provides the function of the user port, it is called an optical network terminal (ONT). In one possible implementation, the ONU123 includes a single family unit (SFU). The user devices connected to the SFU include: a personal computer (PC), an interactive network television (IPTV) and/or a fixed telephone. In another possible implementation, the ONU123 includes a multi-dwelling unit (MDU). The MDU communicates with user equipment via very high bit rate digital subscriber loop (VDSL) technology. The user equipment connected to the MDU includes: PC, IPTV and/or landline telephone. In another possible implementation, ONU123 includes a single business unit (SBU). The user equipment connected to the SBU includes: PC and/or landline telephone. In another possible implementation, ONU123 includes a base station unit splitter (cell base unit, CBU). The user equipment connected to the CBU includes a base station. The base station transmits Internet information and/or voice information through the CBU. It should be noted that the aforementioned ONU123 may also include other types of units or modules, which are not limited in the embodiments of the present application.

The ODN 122 may be a data distribution network, and may include optical fibers, optical couplers, optical splitters or other devices. In one possible implementation, the optical fibers, optical couplers, optical splitters or other devices may be passive devices. Specifically, the optical fibers, optical couplers, optical splitters or other devices may be devices that do not require power support when distributing data signals between the OLT 121 and the ONU 123. Specifically, taking an optical splitter as an example, the optical splitter may be connected to the OLT 121 via a trunk optical fiber, and connected to multiple ONUs 123 via multiple branch optical fibers, respectively, thereby realizing a point-to-multipoint connection between the OLT 121 and the ONU 123. In another possible implementation, the ODN 122 may also include one or more processing devices, such as an optical amplifier or a relay device. In addition, the ODN 122 may specifically extend from the OLT 121 to multiple ONUs 123, but may also be configured into any other point-to-multipoint structure, and the embodiments of the present invention are not limited thereto.

In one possible implementation, after the single board in OLT121 collects the KPI data of different management applications (applications, APPs) of home users, a lightweight quality difference identification model is first set on the single board, such as the first quality difference identification model. The single board can perform the first stage quality difference identification on the user's KPI data according to the first quality difference identification model to obtain the edge side quality difference identification result and the confidence level corresponding to the edge side quality difference identification result. Among them, the confidence level is used to indicate the accuracy represented by the edge side quality difference identification result. It can be understood that the higher the confidence level, the higher the credibility of the edge side quality difference identification result; the lower the confidence level, the lower the credibility of the edge side quality difference identification result.

In another possible implementation, the single board can set the reporting method and quantity of data through the interface. For edge-side quality difference recognition results with high confidence (for example, the second confidence in the confidence), the single board can directly send the second quality difference recognition result corresponding to the second confidence to the cloud 110. For edge-side quality difference recognition results with low confidence (for example, the first confidence in the confidence), the single board can send KPI data corresponding to the first confidence to the cloud 110. Among them, the accuracy of the first quality difference recognition result represented by the first confidence is lower than the accuracy of the second quality difference recognition result represented by the second confidence, and the first quality difference recognition result and the second quality difference recognition result belong to edge-side quality difference recognition results.

After receiving the data reported by the single board of OLT121, the cloud 110 can use a high-precision neural network model (such as a second quality difference identification model) to perform a second stage of quality difference identification, further judge the data that cannot be accurately identified by the OLT121 side, and output the quality difference identification result.

Operators can obtain the quality difference identification results on the user-side APP of the intelligent management and control platform, and carry out the quality difference user operation and maintenance services based on the quality difference identification results. The collaborative cloud-edge approach can fully utilize the edge computing power and reduce the data communication and cloud computing costs, thereby achieving the effect of reducing bandwidth and computing resource requirements without affecting the accuracy of quality difference identification.

Please refer to Figure 4, which is a flow chart of a possible quality difference identification method provided in an embodiment of the present application. Optionally, the quality difference identification method can be applied to the quality difference identification system shown in Figure 2 or Figure 4.

The above-mentioned quality difference identification method includes one or more steps from step S401 to step S404. It should be understood that for the convenience of description, this application is described in the order of S401 to S404, and is not intended to limit the execution to the above order. The embodiment of the present application does not limit the execution order, execution time, execution number, etc. of the above-mentioned one or more steps. S401 to step S404 are as follows:

Step S401: The first device analyzes the key performance indicator KPI data of the user according to the first quality difference identification model to obtain the edge side quality difference identification result and the confidence level corresponding to the edge side quality difference identification result.

The confidence level is used to indicate the accuracy of the edge-side quality difference identification result, that is, the accuracy of the quality difference identification of the KPI data can be distinguished based on the confidence level.

In one possible implementation, the first device is a device deployed on the edge side, and the second device is a device deployed on the cloud. The first quality difference recognition model is a lightweight neural network model. For data with a small amount of calculation, the first device can obtain a more accurate recognition result with a smaller amount of calculation through the first quality difference recognition model. However, for data with a large amount of calculation, a more accurate recognition result may not be obtained through the first quality difference recognition model. It can be understood that the higher the confidence level, the higher the accuracy represented by the edge-side quality difference recognition result corresponding to the confidence level; the lower the confidence level, the lower the accuracy represented by the edge-side quality difference recognition result corresponding to the confidence level.

In one possible implementation, the first device may be a board installed on the OLT. The first device may provide data including but not limited to the following types: application fingerprint identification; OLT ID and ONT ID; network KPI indicators, etc.

Among them, application fingerprint recognition is used to distinguish the type of APP currently used by the user. The first device can identify more than 100 different APPs in three major categories. For example, suppose that the data streams being transmitted by the first device include KPI data 1, KPI data 2, and KPI data 3. KPI data 1 is the KPI data sent by application 1 of the video playback type, KPI data 2 is the KPI data of application 2 of the live broadcast type, and KPI data 3 is the KPI data sent by application 3 of the game type. Then when the network device transmits KPI data 1, KPI data 2, and KPI data 3, it can identify application 1 as a video playback type application, application 2 as a live broadcast type application, and application 3 as a game type application based on application fingerprint recognition. After identifying the types of application 1, application 2, and application 3, the first device can determine the application types to which KPI data 1, KPI data 2, and KPI data 3 belong.

OLT ID and ONT ID can be used to determine the source of KPI data. It is understandable that the data stream transmitted by the first device contains KPI data, and the KPI data carries OLT ID and ONT ID. OLT is the central office equipment of the operator's passive optical network. Multiple ONTs are set on the single board of OLT, and each ONT corresponds to a user. Therefore, the first device can determine which user the current KPI data corresponds to based on the OLT ID and ONT ID pre-set on the single board.

Network KPI indicators can be used to indicate the current user's network status, including but not limited to total packet loss, round-trip time, effective rate, average rate, download rate, etc.

In a possible implementation, after the first device collects KPI data from the transmitted data stream, it pre-processes the data, such as aggregating features of multiple data streams, building relevant features, data default processing, data normalization, etc. Thus, the KPI indicators contained in the KPI data and the users and applications corresponding to the KPI data can be determined.

In a possible implementation, the first device may receive the first quality difference identification model from the second device, that is, the first quality difference identification model may be obtained by training the second device using historical data.

Next, the first device can input the KPI data of all applications of the user within the time window into the first quality difference identification model, and output the binary classification result and the confidence corresponding to the binary classification result. The binary classification result is used to indicate whether a quality difference event occurs at the current time. Based on the confidence, the first device can distinguish the accuracy of the KPI data identification.

Step S402: The first device sends KPI data corresponding to a first confidence level in the confidence level to the second device. Alternatively, the first device sends a second quality difference identification result corresponding to a second confidence level in the confidence level to the second device.

Similarly, the second device may receive KPI data corresponding to the first confidence level from the first device. Alternatively, the second device may receive a second quality difference identification result corresponding to the second confidence level from the first device.

Among them, the accuracy of the first quality difference identification result represented by the first confidence is lower than the accuracy of the second quality difference identification result represented by the second confidence, and the first quality difference identification result and the second quality difference identification result are edge-side quality difference identification results.

Therefore, a high confidence level indicates that the KPI data corresponding to the confidence level is highly certain to belong to the normal or poor quality category, and can be effectively identified using a lightweight model; a low confidence level indicates that it is difficult for the first device to accurately identify the poor quality of the KPI data corresponding to the confidence level, and further judgment is required using a high-precision model.

In another possible implementation, the first device can set a data reporting method in the interface between the collected data and the reported data, and add parameters such as "confidence threshold", "reporting ratio", and "reporting method". In this way, the first device can send data to the cloud according to the preset reporting method.

Therefore, the first device can determine the specific reporting method according to the "reporting method" parameter. If the reporting method corresponding to the "reporting method" parameter is a confidence threshold, the first device can send the KPI data corresponding to the confidence level less than the above confidence threshold to the second device according to the confidence threshold. The first device can send the second quality difference identification result corresponding to the confidence level greater than or equal to the above confidence threshold to the second device according to the confidence threshold. Therefore, the first confidence level is less than the above confidence threshold, and the second confidence level is greater than or equal to the above confidence threshold. It should be noted that the "equal to" situation can be placed in another branch of the judgment, such as "sending the KPI data corresponding to the confidence level less than or equal to the above confidence threshold".

If the reporting method corresponding to the "reporting method" parameter is the reporting ratio, the first device can send the KPI data corresponding to the confidence that is less than the above data ratio in the confidence ranking to the second device according to the reporting ratio. The first device can send the second quality difference recognition result corresponding to the confidence that is greater than or equal to the above data ratio in the confidence ranking to the second device according to the reporting ratio. Among them, the confidence ranking is the value corresponding to the confidence obtained by sorting the confidence corresponding to the edge side recognition result in order from small to large. It should be noted that the "equal to" situation can be placed in another branch of the judgment, such as "sending KPI data corresponding to the confidence that is less than or equal to the above data ratio in the confidence ranking".

In summary, assuming that the data stream being transmitted by the first device includes KPI data 1, KPI data 2, and KPI data 3, the first device collects KPI data 1, KPI data 2, and KPI data 3 from the data stream being transmitted, and preprocesses the above KPI data, and analyzes the above preprocessed KPI data 1, KPI data 2, and KPI data 3 according to the first quality difference identification model, respectively, to obtain the first edge side quality difference identification result corresponding to KPI data 1 and the confidence corresponding to the first edge side quality difference identification result, the second edge side quality difference identification result corresponding to KPI data 2 and the confidence corresponding to the first edge side quality difference identification result, and the third edge side quality difference identification result corresponding to KPI data 3 and the confidence corresponding to the first edge side quality difference identification result. If the confidence corresponding to the first edge side quality difference identification result belongs to the first confidence, indicating that the confidence is low, the first device sends KPI data 1 to the second device. If the confidence level corresponding to the second edge side quality difference recognition result belongs to the second confidence level, indicating that the confidence level is relatively high, the first device sends the second edge side quality difference recognition result to the second device. If the confidence level corresponding to the third edge side quality difference recognition result also belongs to the second confidence level, indicating that the confidence level is relatively high, the first device sends the third edge side quality difference recognition result to the second device.

It should be noted that, in actual applications, the preset threshold and data ratio can be determined according to the actual network conditions, and this application does not impose any restrictions.

In step S403, the second device analyzes the KPI data corresponding to the first confidence level according to the second quality difference identification model to obtain a cloud quality difference identification result.

Specifically, the second device can train a first quality difference recognition model and a second quality difference recognition model based on historical data. The first quality difference recognition model is a lightweight neural network model, and the second quality difference recognition model is a high-precision neural network model. Furthermore, the second quality difference recognition model includes three high-precision neural network models, corresponding to three major categories of applications: live broadcast, on-demand, and games. The second device can use a high-precision neural network model to further judge the KPI data that cannot be accurately identified, and obtain a cloud-based quality difference recognition result.

In practical applications, after obtaining the trained first quality difference recognition model and the second quality difference recognition model, the first quality difference recognition model and the second quality difference recognition model can be deployed online for application, for example, the second device sends the first quality difference recognition model to the first device. After the above-mentioned trained first quality difference recognition model and the second quality difference recognition model are deployed to the online application, the accuracy of the trained model can be regularly verified at preset time intervals (such as half a month, a month, etc.). If the accuracy of the model decreases significantly, the model is degraded and verified. Data can be re-collected to train the trained model, and the model parameters can be further optimized to improve the accuracy of the model and ensure the accuracy of quality difference recognition.

It is understandable that the KPI data corresponding to the first confidence level received by the second device is data pre-processed by the first device, so the KPI data carries the application type. Therefore, the second device can input the KPI data of the corresponding application into the corresponding quality difference recognition model. For example, the KPI data of the live broadcast type is input into the model corresponding to the live broadcast application, the KPI data of the on-demand type is input into the model corresponding to the on-demand application, and the KPI data of the game type is input into the model corresponding to the game type. The output of the model is a binary classification result, indicating whether a quality difference event occurs in the current time window.

Step S404: The second device obtains the quality difference identification result of the user according to the second quality difference identification result and the cloud quality difference identification result.

Specifically, the second device counts the second quality difference identification result reported by the first device and the quality difference result in the cloud quality difference identification result, wherein the quality difference result is used to indicate that the network situation reflected by the KPI data corresponding to the quality difference result is poor. When the value is , the user corresponding to the poor quality result is a poor quality user.

For example, the second quality difference identification result corresponding to user 1 includes a first number of quality difference results and a second number of non-quality difference results, and the cloud quality difference identification result corresponding to user 1 includes a third number of quality difference results and a fourth number of non-quality difference results. If the total number of the first number of quality difference results and the third number of quality difference results exceeds a preset threshold, user 1 is a quality difference user.

Please refer to Fig. 5, which is a flow chart of another quality difference identification method provided in an embodiment of the present application. Optionally, the quality difference identification method can be applied to the quality difference identification system shown in Fig. 2 or Fig. 3.

The above-mentioned application quality difference identification method includes one or more steps from step S1 to step S5. It should be understood that for the convenience of description, this application is described in the order of S1 to S5, and is not intended to limit the execution to the above order. The embodiment of the present application does not limit the execution order, execution time, execution number, etc. of the above-mentioned one or more steps.

As can be seen from Figure 5, the quality difference identification method mainly includes 5 main processes: Step S1: the first device collects and processes KPI data; Step S2: the first device outputs the edge side quality difference identification result and its corresponding confidence according to the KPI data; Step S3: the first device sets the data reporting method through the interface, and reports the data to the second device according to the reporting method; Step S4: the second device determines whether the user is a poor quality user according to the reported data; Step S5: the operator carries out poor quality user operation and maintenance services according to the quality difference identification result.

The specific implementation process of the above steps S1 to S5 can be as follows:

Step S11: With the development of Internet technology, various terminal applications emerge in an endless stream. In order to ensure the QoS of applications, operators need to manage application traffic. Performing quality evaluation and analysis on applications to identify poor quality users is a key step for operators to manage application traffic. Therefore, the first device can collect network KPI data of different applications in each user on a user-by-user basis;

Step S12: The first device may average and aggregate different flows in the original KPI data according to time and application, and construct new features from the original KPI data. The fields included in the processed KPI data are shown in Table 1:

Table 1: Fields

Step S21: From the KPI data outputted in step S12, the user's OLT ID and ONT ID can be extracted through the fields resID and clientLocation, and then the KPI data of the same OLT ID and ONT ID are aggregated in a summation manner according to the time point, that is, the KPI data of different applications at the same time point are aggregated, and the aggregated data is used as the input of the next step. It can be understood that the KPI data of the same OLT ID and ONT ID can be considered as the KPI data of the same user.

Step S22: The first device at the edge side performs quality difference identification according to the first quality difference identification model, that is, the KPI data outputted in step S21 is inputted into the first quality difference identification model for binary classification, and the model outputs the current KPI data edge side quality difference identification result and its corresponding location. Confidence. That is, a binary classification neural network model is used on the edge side to perform the first stage of quality difference identification. Further, the above neural network includes 4 full connections, and the multidimensional KPI feature vector (which can be a 30-dimensional KPI feature vector) aggregated by step S21 through time points is used as the input of the neural network. The last fully connected layer maps the output value to the range of 0 to 1 after the sigmoid function, where 1 represents poor quality and 0 represents non-poor quality. It can be understood that when training the first quality difference identification model, the labeled data can be used to train the neural network for binary classification tasks to reduce the binary cross entropy loss function. After the training, the neural network model is used to classify the corresponding KPI data in the existing network environment, and a threshold σ ₁ is selected as the boundary. If the output of the neural network is greater than σ ₁ , it can be determined as a quality difference event, and if the output is less than or equal to σ ₁ , it can be determined as a non-quality difference event. At the same time, the neural network has an output confidence to describe the accuracy of the quality difference identification of the KPI data.

Step S31: The first device can set the data reporting method through the interface. Furthermore, the first device can add parameters such as "confidence threshold", "data ratio" and "reporting method" in the interface for subscribing to data between collecting data and reporting data. For example, the field value of "reporting method" is set to 0, and the first device can send data to the second device according to the confidence threshold. The field value of "reporting method" is set to 1, and the first device can send data to the second device according to the data ratio. If the confidence threshold reporting method is selected, the confidence threshold can be set by setting the "confidence threshold" field value, and the field value setting range can be 0-1; if the data reporting method is selected, the reported data ratio can be set by setting the "reporting ratio" field value, and the field value setting range can be 0-1. Among them, the confidence threshold reporting is to report the original KPI number below the confidence threshold, and report the poor quality identification result above the confidence threshold. Data ratio reporting is to sort the confidence in ascending order, and report the original KPI data for the first X% according to the data ratio, and report the poor quality identification results for the last 1-X%.

Step S32: The first device can report the KPI data and the edge-side quality difference identification result to the second device located in the cloud in combination with the data reporting method set in step S31, the edge-side quality difference identification result and the confidence level output in step S22. According to the different values set in the "reporting method" field, the data reporting scenarios can be divided into confidence threshold reporting scenarios and data ratio reporting scenarios.

Confidence threshold reporting scenario: The first device can make a judgment based on the confidence threshold set by the interface. If the confidence output by the first quality difference identification model is lower than the confidence threshold, the original KPI data corresponding to the confidence is reported; if it is higher than the confidence threshold, the edge side quality difference identification result corresponding to the confidence is reported. In this scenario, the quantity set by the first device to the second device is in dynamic change.

Data ratio reporting scenario: The first device can sort the KPI data in ascending order according to the confidence level, and use the data ratio set by the interface to make judgments. The original KPI data corresponding to the sorting is reported if the sorting is lower than the data ratio, and the quality difference identification result corresponding to the sorting is reported if the sorting is higher than the ratio. In this scenario, the amount of data reported is constant.

Step S41: The second device receives the data reported by the first device, and corresponds the KPI data reported in step S32 to different application types. If it is KPI data, the second quality difference identification model is further used for quality difference identification. If it is a quality difference identification result, the result is retained and a comprehensive judgment is made with the identification result output by the second quality difference identification model. For KPI data, the second device can use APP ID (i.e., applicationSubType field) to filter out data about live broadcast, on-demand and games in the data stream, and select the data of the network KPI field in the data as the input of the second quality difference identification model.

Step S42: The second device inputs the KPI data outputted from step S41 into the corresponding second quality difference identification model, a binary classification neural network, for second-stage quality difference identification in the cloud, and outputs the cloud quality difference identification result of the current application KPI data. Furthermore, the second device can identify the quality difference of the three major applications of live broadcast, on-demand and games through three neural networks. Among them, the above-mentioned neural network is a high-precision neural network, which includes 9 fully connected layers. The first fully connected layer of the neural network accepts the multi-dimensional real-valued feature vector (which can be a 30-dimensional real-valued feature vector) outputted from step S41 as the input of the network, and the last fully connected layer maps the output value to the range of 0 to 1 after the sigmoid function. Among them, 1 can represent poor quality, and 0 can represent no poor quality. It can be understood that when training the model, the network can be trained with labeled laboratory construction data to perform binary classification tasks to reduce the binary cross entropy loss function. After the training, the trained neural network model is used to classify the corresponding application KPI data in the existing network environment, and a threshold σ ₂ is selected as the boundary. If the output of the neural network is greater than σ ₂ , it is judged as a quality-poor event, otherwise it is judged as a non-quality-poor event.

Step S43: The second device can combine the edge-side poor quality identification result reported in step S32 and the cloud-side poor quality identification result output in step S42 to make a poor quality user judgment. Further, the second device can set a time window length L (for example, L is one week), extract the user's OLT ID and ONT ID through the fields resID and clientLocation, and then count the number of all poor quality events corresponding to this user ID in window L. If the total number of poor quality events exceeds the threshold σ ₃ , it is determined that the user is a poor quality user.

Step S51: The second device may push the poor-quality user identification result outputted in step S43 to the user-side APP, thereby supporting the operation and maintenance of poor-quality users.

A person skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by a computer program to instruct the relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned method embodiments. The aforementioned storage medium includes: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

In summary, the embodiment of the present application can process different recognition difficulties in stages based on the lightweight model on the edge and the high-precision model on the cloud. KPI data. A lightweight model is used on the edge side to perform the first stage of quality difference identification on the KPI data, perform a preliminary screening of the quality difference time, and output the quality difference identification results and confidence. Based on the confidence and the reporting method set by the data reporting interface, the low-confidence original KPI data and the high-confidence quality difference identification results are reported to the cloud. A high-precision multi-neural network model is used on the cloud to perform the second stage of quality difference identification, and the KPI data that cannot be accurately identified is further judged to obtain the quality difference results. By using the edge-side computing power to screen data and report part of the data, it is possible to reduce the transmission bandwidth and motion computing resource requirements without affecting the accuracy of quality difference identification.

The method of the embodiment of the present application is described above, and the device of the embodiment of the present application is provided below.

Please refer to Figure 6, which is a schematic diagram of the structure of a computing device 60 provided in an embodiment of the present application. The computing device 60 may include a processing module 601 and a communication module 602. The computing device 60 is used to implement the aforementioned quality difference identification method, such as the data processing method in the embodiment shown in Figure 4 or Figure 5.

In a possible implementation, the computing device 60 is the first device in the aforementioned embodiment.

The processing module 601 is used to analyze the key performance indicator KPI data of the user according to the first quality difference identification model to obtain the edge side quality difference identification result and the confidence corresponding to the edge side quality difference identification result, wherein the confidence is used to indicate the accuracy represented by the edge side quality difference identification result;

The communication module 602 is used to send KPI data corresponding to a first confidence level among the confidence levels to a second device;

The communication module 602 is also used to send a second quality difference identification result corresponding to a second confidence level in the confidence level to the second device, wherein the accuracy of the first quality difference identification result represented by the first confidence level is lower than the accuracy of the second quality difference identification result represented by the second confidence level, and the first quality difference identification result and the second quality difference identification result belong to the edge side quality difference identification result.

In a possible implementation, the computing device 60 is a device deployed on the edge side, and the second device is a device deployed on the cloud.

In a possible implementation manner, the processing module 601 is further configured to:

In a possible implementation, when the reporting method is the confidence threshold, the first confidence is less than the confidence threshold, and the second confidence is greater than or equal to the confidence threshold.

In a possible implementation, when the reporting method is the data ratio, the first confidence is the confidence that is less than the data ratio in the confidence ranking, the second confidence is the confidence that is greater than or equal to the data ratio in the confidence ranking, and the confidence ranking is the numerical value corresponding to the confidence obtained by sorting the confidences corresponding to the edge side recognition results in ascending order.

In a possible implementation manner, the communication module 602 is further configured to:

The first quality difference identification model is received from the second device, where the first quality difference identification model is trained by the second device.

In another possible implementation, the computing device 60 is the second device in the aforementioned embodiment.

The communication module 602 is used to receive KPI data corresponding to a first confidence level from a first device;

The communication module 602 is further configured to receive a second quality difference identification result corresponding to a second confidence level from the first device, wherein the accuracy of the first quality difference identification result represented by the first confidence level is lower than the accuracy of the second quality difference identification result represented by the second confidence level;

The processing module 601 is used to analyze the KPI data corresponding to the first confidence level according to the second quality difference identification model to obtain a cloud quality difference identification result;

The processing module 601 is further configured to obtain a quality difference identification result of the user according to the second quality difference identification result and the cloud quality difference identification result.

In another possible implementation, the first device is a device deployed on the edge side, and the computing device 60 is a device deployed on the cloud.

In another possible implementation, the processing module 601 is specifically configured to:

In another possible implementation, the processing module 601 is further configured to:

In another possible implementation, the communication module 602 is further configured to:

Please refer to FIG. 7 , which is a schematic diagram of the structure of a computing device 70 provided in an embodiment of the present application.

The computing device 70 may be an independent device such as a server, a host, an edge device, or a device included in an independent device, such as a chip, a software module, or an integrated circuit. The computing device 70 may include at least one processor 701 and at least one memory 702. Optionally, a communication interface 703 may also be included. Further optionally, a connection line 704 may also be included, wherein the processor 701 and the memory 702 are connected via the connection line 704, and communicate with each other and transmit control and/or data signals via the connection line 704.

in:

(1) The processor 701 is a module for performing arithmetic operations and/or logical operations, and may specifically include one or more of the following devices: a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a complex programmable logic device (CPLD), a coprocessor (to assist the central processing unit in completing corresponding processing and applications), or a microcontroller unit (MCU), etc.

(2) Memory 702 is used to provide storage space, in which data such as operating system and computer program can be stored. Memory 702 can be one or a combination of random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM).

(3) The communication interface 703 may be used to provide information input or output for the at least one processor. In some possible scenarios, the communication interface 703 may include an interface circuit. And/or, the communication interface 703 may be used to receive data sent externally and/or send data to the outside. For example, the communication interface 703 may include a wired link interface such as an Ethernet cable, or a wireless link (Wi-Fi, Bluetooth, general wireless transmission and other short-range wireless communication technologies, etc.) interface. Optionally, the communication interface 703 may also include a transmitter (such as a radio frequency transmitter, antenna, etc.) coupled to the interface, or a receiver, etc.

Optionally, if the computing device 70 is an independent device, the communication interface 703 may include a receiver and a transmitter. The receiver and the transmitter may be the same component or different components. When the receiver and the transmitter are the same component, the component may be referred to as a transceiver.

Optionally, if the computing device 70 is a chip or a circuit, the communication interface 703 may include an input interface and an output interface, and the input interface and the output interface may be the same interface, or may be different interfaces.

Optionally, the function of the communication interface 703 may be implemented by a transceiver circuit or a dedicated transceiver chip. The processor 701 may be implemented by a dedicated processing chip, a processing circuit, a processor or a general-purpose chip.

The functions and actions of the modules or units in the computing device 70 listed above are only for illustrative purposes.

Each functional unit in the computing device 70 can be used to implement the aforementioned quality difference identification method. For example, the computing device can be the first device and/or the second device in the embodiment shown in FIG. 4 or FIG. 5. For details, please refer to the aforementioned. In order to avoid redundancy, the detailed description is omitted here.

Optionally, the processor 701 may be a processor specifically used to execute the aforementioned method (for convenience of distinction, referred to as a dedicated processor), or may be a processor that executes the aforementioned method by calling a computer program (for convenience of distinction, referred to as a dedicated processor). Optionally, the at least one processor may include both a dedicated processor and a general-purpose processor.

Optionally, in the case where the computing device includes at least one memory 702 , if the processor 701 implements the aforementioned quality difference identification method by calling a computer program, the computer program may be stored in the memory 702 .

The embodiment of the present application also provides a chip system, which includes a processor and a communication interface, wherein the communication interface is used to receive and/or send data, and/or the communication interface is used to provide input and/or output for the processor. The chip system is used to implement the aforementioned quality difference identification method, such as the method described in Figure 4 or Figure 5.

The embodiment of the present application further provides a computer-readable storage medium, in which instructions are stored, and the instructions are used to implement the aforementioned quality difference identification method, such as the method described in Figure 4 or Figure 5.

The embodiment of the present application further provides a computer program product, which includes computer instructions, and the computer instructions are used to implement the aforementioned quality difference identification method, such as the method described in Figure 4 or Figure 6.

In the embodiments of the present application, the words "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

In the embodiments of this application, "at least one" refers to one or more, and "more" refers to two or more. "Item(s)" or similar expressions refer to any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one item(s) of a, b, or c can be represented by: a, b, c, (a and b), (a and c), (b and c), or (a and b and c), where a, b, c can be single or plural. "And/or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and/or B can be represented by: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character "/" generally indicates that the associated objects before and after are in an "or" relationship.

Furthermore, unless otherwise specified, the ordinal numbers such as "first" and "second" used in the embodiments of the present application are used to distinguish multiple objects, and are not used to limit the order, timing, priority or importance of multiple objects. For example, the first device and the second device are only for the convenience of description, and do not represent the difference in structure, importance, etc. between the first device and the second device. In some embodiments, the first device and the second device can also be the same device.

In the above embodiments, the term "when..." can be interpreted as meaning "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modification, equivalent substitution, improvement, etc. made within the concept and principle of the present application shall be included in the protection scope of the present application.

A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

Claims

A method for identifying poor quality, characterized in that the method comprises:

The first device analyzes the key performance indicator KPI data of the user according to the first quality difference identification model to obtain an edge side quality difference identification result and a confidence level corresponding to the edge side quality difference identification result, wherein the confidence level is used to indicate the accuracy represented by the edge side quality difference identification result;

The first device sends KPI data corresponding to a first confidence level among the confidence levels to the second device;

The first device sends a second quality difference identification result corresponding to a second confidence level in the confidence level to the second device, wherein the accuracy of the first quality difference identification result represented by the first confidence level is lower than the accuracy of the second quality difference identification result represented by the second confidence level, and the first quality difference identification result and the second quality difference identification result belong to the edge side quality difference identification result.
The method according to claim 1 is characterized in that the first device is a device deployed on the edge side, and the second device is a device deployed on the cloud.
The method according to claim 1 or 2 is characterized in that before the first device sends the second quality difference identification result corresponding to the second confidence level in the confidence level to the second device, it also includes:

The first device sets a data reporting method through an interface, and the reporting method includes a confidence threshold and a data ratio.
The method according to claim 3 is characterized in that, when the reporting method is the confidence threshold, the first confidence is less than the confidence threshold, and the second confidence is greater than or equal to the confidence threshold.
The method according to claim 3 is characterized in that, when the reporting method is the data ratio, the first confidence is the confidence that is less than the data ratio in the confidence ranking, the second confidence is the confidence that is greater than or equal to the data ratio in the confidence ranking, and the confidence ranking is the numerical value corresponding to the confidence obtained by sorting the confidences corresponding to the edge-side recognition results in ascending order.
The method according to any one of claims 1 to 5 is characterized in that, before the first device analyzes the user's key performance indicator KPI data according to the first quality difference identification model to obtain the edge side quality difference identification result and the confidence corresponding to the edge side quality difference identification result, it also includes:

The first device receives the first quality difference identification model from the second device, where the first quality difference identification model is trained by the second device.
A method for identifying poor quality, characterized in that the method comprises:

The second device receives KPI data corresponding to the first confidence level from the first device;

The second device receives a second quality difference identification result corresponding to a second confidence level from the first device, wherein the accuracy of the first quality difference identification result represented by the first confidence level is lower than the accuracy of the second quality difference identification result represented by the second confidence level;

The second device analyzes the KPI data corresponding to the first confidence level according to the second quality difference identification model to obtain a cloud quality difference identification result;

The second device obtains the quality difference identification result of the user according to the second quality difference identification result and the cloud quality difference identification result.
The method according to claim 7 is characterized in that the first device is a device deployed on the edge side, and the second device is a device deployed on the cloud.
The method according to claim 7 or 8 is characterized in that the first confidence is less than a confidence threshold, and the second confidence is greater than or equal to the confidence threshold, wherein the confidence threshold is a reporting method set by the first device.
According to the method according to claim 7 or 8, it is characterized in that the first confidence is less than the data ratio in the confidence ranking, the second confidence is greater than or equal to the data ratio in the confidence ranking, and the confidence ranking is the numerical value corresponding to the confidence obtained after sorting the confidence in ascending order, wherein the confidence is the confidence corresponding to the edge side quality difference identification result obtained after the first device analyzes the user's KPI data according to the first quality difference identification model, and the data ratio is the reporting method set by the first device.
The method according to any one of claims 7 to 10 is characterized in that the second device obtains the user's quality difference identification result according to the second quality difference identification result and the cloud quality difference identification result, comprising:

The second device counts the quality difference results in the second quality difference identification result and the cloud quality difference identification result, and the quality difference results are used to indicate that the network situation reflected by the KPI data corresponding to the quality difference results is poor;

When the number of the poor quality results is greater than a preset threshold, the users corresponding to the poor quality results are poor quality users.
The method according to any one of claims 7 to 11, characterized in that before the second device sends the first quality difference identification model to the first device, it also includes:

The second device obtains the first quality difference recognition model and the second quality difference recognition model through training according to historical KPI data, wherein the recognition accuracy of the first quality difference recognition model is lower than the recognition accuracy of the second quality difference recognition model.
The method according to claim 12 is characterized in that after the second device obtains the first quality difference identification model and the second quality difference identification model through training according to historical KPI data, it also includes:

The second device sends the first quality difference identification model to the first device, wherein the first device is used to obtain the edge side quality difference identification result and the confidence corresponding to the edge side quality difference identification result according to the first quality difference identification model, and the confidence is used to indicate the accuracy represented by the edge side quality difference result, and the confidence includes the first confidence and the second confidence.
A quality difference identification system, characterized in that the system comprises a first device and a second device, wherein:

The first device is used to execute the method according to any one of claims 1 to 6, and the second device is used to execute the method according to any one of claims 7 to 13.
A computing device, characterized in that the computing device comprises a processing module and a communication module, wherein:

The processing module is used to analyze the key performance indicator KPI data of the user according to the first quality difference identification model to obtain the edge side quality difference identification result and the confidence corresponding to the edge side quality difference identification result, wherein the confidence is used to indicate the accuracy represented by the edge side quality difference identification result;

The communication module is used to send KPI data corresponding to a first confidence level among the confidence levels to the second device;

The communication module is also used to send a second quality difference identification result corresponding to a second confidence level in the confidence level to the second device, wherein the accuracy of the first quality difference identification result represented by the first confidence level is lower than the accuracy of the second quality difference identification result represented by the second confidence level, and the first quality difference identification result and the second quality difference identification result belong to the edge side quality difference identification result.
The device according to claim 15 is characterized in that the computing device is a device deployed on the edge side, and the second device is a device deployed on the cloud.
The device according to claim 15 or 16, characterized in that the processing module is further used to:

The data reporting method is set through the interface, and the reporting method includes a confidence threshold and a data ratio.
The device according to claim 17 is characterized in that, when the reporting method is the confidence threshold, the first confidence is less than the confidence threshold, and the second confidence is greater than or equal to the confidence threshold.
The device according to claim 17 is characterized in that, when the reporting method is the data ratio, the first confidence is the confidence that is less than the data ratio in the confidence ranking, the second confidence is the confidence that is greater than or equal to the data ratio in the confidence ranking, and the confidence ranking is the numerical value corresponding to the confidence obtained by sorting the confidences corresponding to the edge-side recognition results in ascending order.
The device according to any one of claims 15 to 19, characterized in that the communication module is further used for:

The first quality difference identification model is received from the second device, where the first quality difference identification model is trained by the second device.
A computing device, characterized in that the computing device comprises a processing module and a communication module, wherein:

The communication module is used to receive KPI data corresponding to the first confidence level from the first device;

The communication module is further configured to receive a second quality difference identification result corresponding to a second confidence level from the first device, wherein the accuracy of the first quality difference identification result represented by the first confidence level is lower than the accuracy of the second quality difference identification result represented by the second confidence level;

The processing module is used to analyze the KPI data corresponding to the first confidence level according to the second quality difference identification model to obtain a cloud quality difference identification result;

The processing module is further used to obtain the user's quality difference identification result according to the second quality difference identification result and the cloud quality difference identification result.
The device according to claim 21 is characterized in that the first device is a device deployed on the edge side, and the computing device is a device deployed on the cloud.
The device according to claim 21 or 22 is characterized in that the first confidence is less than a confidence threshold, and the second confidence is greater than or equal to the confidence threshold, wherein the confidence threshold is a reporting method set by the first device.
The device according to claim 21 or 22 is characterized in that the first confidence is less than the data ratio in the confidence ranking, the second confidence is greater than or equal to the data ratio in the confidence ranking, and the confidence ranking is a numerical value corresponding to the confidence obtained by sorting the confidence in ascending order, wherein the confidence is the confidence corresponding to the edge side quality difference identification result obtained by the first device after analyzing the user's KPI data according to the first quality difference identification model, and the data ratio is the reporting method set by the first device.
The device according to any one of claims 21 to 24, characterized in that the processing module is specifically used to:

Counting the quality difference results in the second quality difference identification result and the cloud quality difference identification result, wherein the quality difference results are used to indicate that the network situation reflected by the KPI data corresponding to the quality difference results is poor;

When the number of the poor quality results is greater than a preset threshold, the users corresponding to the poor quality results are poor quality users.
The device according to any one of claims 21 to 25, characterized in that the processing module is further used to:

The first quality difference recognition model and the second quality difference recognition model are obtained by training according to historical KPI data, wherein the recognition accuracy of the first quality difference recognition model is lower than the recognition accuracy of the second quality difference recognition model.
The device according to claim 26, characterized in that the communication module is further used to:

The first quality difference identification model is sent to the first device, wherein the first device is used to obtain an edge side quality difference identification result and a confidence level corresponding to the edge side quality difference identification result according to the first quality difference identification model, the confidence level is used to indicate the accuracy represented by the edge side quality difference result, and the confidence level includes the first confidence level and the second confidence level.
A computing device, characterized in that the computing device comprises a processor and a memory, and the processor is used to execute instructions stored in the memory so that the computing device implements the method described in any one of claims 1 to 13.
A computer-readable storage medium, characterized in that program instructions are stored in the computer-readable storage medium, and when the program instructions are executed on a computer or a processor, the method described in any one of claims 1 to 13 is implemented.
A computer program product, characterized in that the computer program product comprises program instructions, and when the program instructions are executed on a computer or a processor, the method described in any one of claims 1 to 13 is implemented.