CN114268985B

CN114268985B - Quality evaluation method and device for 5G private network, electronic equipment and storage medium

Info

Publication number: CN114268985B
Application number: CN202111425916.8A
Authority: CN
Inventors: 王杉; 冯毅; 张勍; 程奥林
Original assignee: China United Network Communications Group Co Ltd
Current assignee: China United Network Communications Group Co Ltd
Priority date: 2021-11-26
Filing date: 2021-11-26
Publication date: 2024-02-13
Anticipated expiration: 2041-11-26
Also published as: CN114268985A

Abstract

The application provides a quality evaluation method, a device, electronic equipment and a storage medium of a 5G private network, which can evaluate the quality of the provided service and the communication quality of the whole 5G private network according to different types of services provided by the 5G private network, particularly calculate service evaluation parameters of communication services by using different first weighting values for communication indexes of different service types under the service types of different 5G private networks, and calculate the integral communication quality parameters of the 5G private network by using a second weighting value according to the service flow of all a plurality of communication services of the 5G private network, thereby realizing the evaluation of the communication quality of the 5G private network, and further enabling operators to better set and maintain the 5G private network according to quality evaluation results and ensuring normal operation and daily maintenance of the 5G private network.

Description

Quality evaluation method and device for 5G private network, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of communications technologies, and in particular, to a quality evaluation method and apparatus for a 5G private network, an electronic device, and a storage medium.

Background

With the continuous development of communication technology, the communication services provided by operators for users are more and more abundant, and meanwhile, the 5G private network can provide safe, reliable and customized network services for the users according to special service requirements, so that more accurate, professional and network communication is realized.

In order to ensure normal operation and daily maintenance of the 5G private network, an operator needs to evaluate the communication quality of the 5G private network, and meanwhile, as the 5G private network adjusts the specific communication service quality provided by the 5G private network according to various services deployed according to the needs of users, the requirements of the different services on the communication quality are not identical. For example, some services have higher requirements on packet loss rate, lower latency requirements, and some services have higher latency requirements, etc. Therefore, how to evaluate the communication quality of the 5G private network more flexibly and effectively under the condition that the service handled by the 5G private network is more complex and various is a technical problem to be solved in the field.

Disclosure of Invention

The application provides a quality evaluation method, a quality evaluation device, electronic equipment and a storage medium for a 5G private network, which are used for evaluating the communication quality of the 5G private network.

The first aspect of the present application provides a quality assessment method for a 5G private network, including: acquiring a plurality of communication services executed by a 5G private network to be evaluated and a plurality of communication indexes of each communication service; classifying the plurality of communication services according to a plurality of preset service categories; determining a first weight value of the plurality of communication indexes corresponding to each business category in the plurality of business categories; weighting the communication indexes of the plurality of communication services according to the first weight value to obtain service evaluation parameters of the plurality of communication services; determining a second weight value corresponding to each communication service in the plurality of communication services; weighting the service evaluation parameters of the plurality of communication services according to the second weight value to obtain communication quality parameters; the communication quality parameter is used for evaluating the communication quality of the 5G private network.

In an implementation manner of the first aspect of the present application, the determining a second weight value corresponding to each communication service in the plurality of communication services includes: and taking the ratio of the communication flow of each communication service in the 5G private network when the 5G private network executes the plurality of communication services in the total communication flow of the 5G private network as a second weight value corresponding to each communication service in the plurality of communication services.

In an implementation manner of the first aspect of the present application, the communication index includes: communication speed, communication time delay, communication jitter and packet loss rate; the communication service class includes: uplink large bandwidth service, downlink large bandwidth service, real-time interactive service and large connection service.

In an implementation manner of the first aspect of the present application, the obtaining a plurality of communication services performed by the 5G private network to be evaluated, and a communication index of each communication service includes: acquiring service flow, message content and communication parameters of the 5G private network in a preset time period according to a Deep Packet Inspection (DPI) technology; and determining the communication service carried out by the 5G private network according to the service flow, the message content and the communication parameters, and calculating the communication index of each communication service.

In an implementation manner of the first aspect of the present application, the classifying the plurality of communication services according to a preset plurality of service classes includes: classifying each of the plurality of communication services into one or more of the plurality of service classes according to a preset plurality of service classes.

In an implementation manner of the first aspect of the present application, for a first communication service classified into a plurality of service classes in the plurality of communication services, the step of obtaining a service evaluation parameter of the first communication service includes: according to the first weight value, weighting the communication index corresponding to each service class in the first communication service to obtain service evaluation sub-parameters of the first communication service corresponding to a plurality of service classes; and weighting the service evaluation sub-parameters of the plurality of service categories according to the third weight value of each service category corresponding to the first communication service to obtain the service evaluation parameters of the first communication service.

In an implementation manner of the first aspect of the present application, after the obtaining the communication quality parameter, the method further includes: determining the communication quality level of the 5G private network according to a first numerical interval in which the communication quality parameter is located and a first mapping relation; the first mapping relation comprises a plurality of first numerical intervals and communication quality levels corresponding to the first numerical intervals; determining a communication quality level of each communication service according to a second numerical interval and a second mapping relation of a service evaluation parameter of each communication service in the plurality of communication services; the second mapping relation comprises a plurality of second numerical intervals and communication quality levels corresponding to the second numerical intervals.

In one implementation manner of the first aspect of the present application, the method further includes: and displaying the communication quality level of the 5G private network and/or the communication quality levels of the plurality of communication services on a display interface.

A second aspect of the present application provides a quality assessment device for a 5G private network, including: the acquisition module is used for acquiring a plurality of communication services executed by the 5G private network to be evaluated and a plurality of communication indexes of each communication service; the classification module classifies the plurality of communication services according to a plurality of preset service categories; a first determining module, configured to determine a first weight value of the plurality of communication indexes corresponding to each service class in the plurality of service classes; the service evaluation module is used for carrying out weighting processing on the communication indexes of the plurality of communication services according to the first weight value to obtain service evaluation parameters of the plurality of communication services; a second determining module, configured to determine a second weight value corresponding to each communication service in the plurality of communication services; the quality evaluation module is used for carrying out weighting processing on the service evaluation parameters of the plurality of communication services according to the second weight value to obtain communication quality parameters; the communication quality parameter is used for evaluating the communication quality of the 5G private network.

A third aspect of the present application provides an electronic device, comprising: a processor and a memory; wherein the memory has stored therein a computer program which, when executed by the processor, is operable to perform a method of quality assessment of a 5G private network according to any of the first aspects of the present application.

A fourth aspect of the present application provides a computer readable storage medium storing a computer program which when executed is operable to perform a method of quality assessment of a 5G private network according to any one of the first aspects of the present application.

In summary, the quality evaluation method, the device, the electronic equipment and the storage medium of the 5G private network provided by the application can evaluate the quality of the provided service and the communication quality of the whole 5G private network according to different types of services provided by the 5G private network, particularly, under the service types of different 5G private networks, the communication indexes of different service types are calculated by using different first weighting values to calculate the service evaluation parameters of the communication service, and according to the service flow of all the plurality of communication services of the 5G private network, the second weighting value is used to calculate the communication quality parameters of the whole 5G private network, so that the evaluation of the communication quality of the 5G private network is realized, and further, an operator can better set and maintain the 5G private network according to the quality evaluation result, and the normal operation and daily maintenance of the 5G private network are ensured.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive faculty for a person skilled in the art.

FIG. 1 is a quality of service aware mapping model;

fig. 2 is a method for establishing a quality evaluation model of a 5G private network provided in the present application;

fig. 3 is a flow chart of an embodiment of a quality evaluation method of a 5G private network provided in the present application;

FIG. 4 is a system deployment architecture diagram for DPI acquisition of a 5G private network;

FIG. 5 is a schematic diagram illustrating the effect of an embodiment of a display interface provided in the present application;

fig. 6 is a flow chart of another embodiment of a quality evaluation method of a 5G private network provided in the present application;

fig. 7 is a schematic structural diagram of an embodiment of a quality evaluation device for a 5G private network provided in the present application;

fig. 8 is a schematic structural diagram of an embodiment of an electronic device provided in the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.

The terms "first," "second," "third," "fourth" and the like in the description and in the claims of this application and in the above-described figures, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that embodiments of the present application described herein may be capable of operation in sequences other than those illustrated or described herein, for example. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

With the continuous development of communication technology, communication services provided by operators to users are becoming more and more abundant, for example, in personal mobile communication services provided by operators to users, internet services such as web browsing, video transmission, games and voice calls occupy more than 80% of the traffic. As another example, with the widespread use of 5G technology, a 5G private network may provide secure, reliable, and customized network services to individual users. The 5G private network can provide more accurate, professional and private network communication for users according to special service requirements.

Because the 5G private network is used for adjusting the specific communication service quality provided by the 5G private network according to various services deployed according to the requirements of users, the requirements of the different services on the communication quality are different, for example, the video services (machine vision, video monitoring and the like) provided by the 5G private network have higher requirements on packet loss rate and lower time delay requirements; the control industry such as industrial production control data, thing networking sensor collection is higher to the time delay requirement. Therefore, how to evaluate the communication quality of the 5G private network more flexibly and effectively under the condition that the service handled by the 5G private network is more complex and various is a technical problem to be solved in the field.

Aiming at the technical problems, the method for evaluating the quality of the 5G private network is provided, and can evaluate the quality of the provided service and the communication quality of the whole 5G private network according to different types of services provided by the 5G private network, so that a provider can better set and maintain the 5G private network according to an evaluation result, and normal operation and daily maintenance of the 5G private network are ensured. The technical scheme of the present application is described in detail below with specific examples. The following embodiments may be combined with each other, and some embodiments may not be repeated for the same or similar concepts or processes.

Aiming at a quality evaluation system of a 5G private network, the method provides a category division rule of the 5G private network service, a selection rule of a communication index of the 5G private network service, an evaluation rule of the 5G private network service and a template of a service index influence factor. These rules and templates for quality assessment of the 5G private network are first described below.

In some embodiments, the application may divide the communication traffic performed by the 5G private network into at least the following traffic classes: large bandwidth (Enhanced Mobile Broadband, abbreviated as eMBB) traffic, real-time interactive (Ultra-reliable and Low Latency Communications, abbreviated as ul lc) traffic, and large connectivity (Massive MachineType Communication, abbreviated as mctc) traffic. The large bandwidth service may be further divided into: upstream large bandwidth traffic (or traffic without significant downstream bandwidth requirements, such as video backhaul, etc.) and downstream large bandwidth traffic (or traffic without significant upstream bandwidth requirements, such as streaming media playback, etc.).

Illustratively, in the application scenario of smart manufacturing, the smart manufacturing-related communication services performed by the 5G private network include: video monitoring, machine vision, robot control, programmable logic controller (Programmable Logic Controller, abbreviated as PLC) industrial control, data acquisition and the like. The communication services can be divided into one of the above four service categories according to the service requirements of the communication services, and the division results are shown in table 1.

TABLE 1

Wherein, the machine vision and robot control need to transmit high definition video and various sensor data in an uplink way, can be divided into "+" in Table 1 indicated uplink large bandwidth class service; and PLC industrial control of the device can be divided into real-time interactive services indicated by "+.f in table 1. Meanwhile, when processing on the network side, similar to the machine vision composite service, the composite service is generally carried on different slices or QoS flows (QoS flows) according to the multi-service requirement and treated as different services, so that the communication services such as the machine vision can be divided into a plurality of different service categories. Specifically, the process of obtaining the service class by performing service analysis on the communication service may be shown in the following table 2.

TABLE 2

In some embodiments, in order to measure the communication quality of the 5G private network, the present application further provides a communication indicator for measuring the quality of an abnormal event of a communication service performed by the 5G private network. Specifically, fig. 1 is a service quality perception mapping model, and as shown in fig. 1, a service index (Key Quality Indicator, abbreviated as KQI) corresponding to a perception index quality of service (Quality of Service, abbreviated as QoE) affecting user experience, a network index KPI that may be specifically applied includes: communication rate, communication delay, communication jitter, and packet loss rate. In a typical smart manufacturing scenario, for example, the services such as video backhaul, video on demand, PLC real-time control, and power grid differential protection, the network key index KQI that affects the user experience QoE can be summarized as the effects of rate, delay, jitter, and packet loss. In addition, for the expandability (integrity) of the network quality assessment template, special indexes (such as access performance, mobility performance and the like) of the service are introduced as network quality assessment supplementary factors for private network service with special index requirements, and the network quality assessment supplementary factors can be supplemented into the communication indexes of the template as required. In the embodiment of the present application, these effects are recorded as communication indexes, and the following table 3 shows a specific correspondence relationship between communication indexes (KPIs) corresponding to different service classes.

TABLE 3 Table 3

In some embodiments, the present application further provides a network index evaluation template applied to the 5G private network, for quantitatively evaluating the quality of the 5G private network. In the application scenario of the 5G private network, since the monitoring of the communication index is more strict than that of the personal mobile communication network, the set quantization threshold value should be lower, so that the quantization value lower than the threshold mark is considered to be the processing such as the degradation of the communication quality and the departure of the subsequent alarm.

In some embodiments, the present application may divide the quality of a 5G private network into three communication quality levels, including: the excellent, good and alert, and the communication quality for each communication index can also be divided into the three levels described above. Illustratively, when the communication quality parameter for indicating the communication quality of the 5G private network is recorded as index Data (Sampling Data), after the communication quality parameter is calculated when the quality of the 5G private network is evaluated, the communication quality level of the 5G private network may be determined according to the correspondence relationship shown in the following table 4.

TABLE 4 Table 4

Wherein 3 corresponds to an excellent communication quality level, and the acquired network index Data is better than an excellent Threshold (Threshold 1), which indicates that the network performance index has more abundant safety margin; 2 corresponds to a good communication quality level, the acquired network index data is between a 'excellent' Threshold (Threshold 1) and an 'alarm' Threshold (Threshold 2), which indicates that the network index is normal, the normal working requirement of the service can be met, but the safety margin is smaller; 1 corresponds to the notification quality level of the alarm, and the acquired network index Data is inferior to an alarm Threshold (Threshold 2), which indicates that the network index cannot meet the normal operation of the service and needs to prompt professionals for network performance investigation; 0 is abnormal, indicating that the index is not considered in the business evaluation.

In the same manner as in table 4 above, four communication indexes, communication rate, communication delay, communication jitter, and communication quality level of packet loss rate, may also be defined, respectively, as shown in table 5 below:

TABLE 5

In addition, the present application provides only the communication quality levels of the four communication indexes as an example, and it can be understood that the special indexes (such as access performance, mobility performance, etc.) applicable to the 5G private network are also evaluated and designed in the same manner as described above, and the relevant threshold parameters are supplemented, and finally the special indexes are incorporated into the evaluation system according to the service requirements of the 5G private network.

In some embodiments, the present application further provides a template of a traffic indicator impact factor, which may be used to weight different traffic indicators differently when evaluating the quality of a traffic according to the traffic indicator of the traffic. For the characteristics of different service types in the service of the 5G private network, different weight values can be given to the communication services of different service types, and the communication services are marked as a first weight value or as influencing weights/influencing factors. Illustratively, table 6 below provides a specific implementation of the first weight values provided herein.

TABLE 6

The uplink large bandwidth service comprises services such as video monitoring, video recording and storage, live broadcast and streaming, and the like, and is characterized in that real-time transmission is carried out according to video quality such as frame rate, code rate, resolution and the like of video coding, and the rate requirement of network transmission is required to meet the code rate requirement, so that the transmission delay is required to be certain. Therefore, in the first weight values shown in table 6, the uplink rate is a key indicator, and the first weight value of the rate is set to be 5. The highest first weight value of the packet loss rate affecting the multimedia backhaul quality is set to 3, and the highest first weight value of the delay and jitter is set to 2. The downlink transmission requirement is not high, and the highest first weight value is set to be 1 or 0.

For downlink large bandwidth service, the service is the same as the streaming media transmission service model of the traditional 2C, the requirement on the downlink transmission rate is higher, the first weight value is set to 5, the first weight value of downlink packet loss is set to 3, and the first weight value of delay and jitter is set to 2. The uplink transmission requirement is not high, and the first weight value is set to be 1 or 0.

For real-time interactive service, mainly packet transmission service, no requirement is imposed on network bandwidth, and mainly the requirement of ultra-low time delay is met. For industrial-grade uRLLC service, the requirements on experiments and jitter are more severe. Therefore, the first weight value of the bidirectional delay and jitter is set to 5, and the first weight value of the uplink and downlink rates is set to 2. Since the industrial control command is generally guaranteed not to be lost by the upper layer protocol through multiple sending and retransmission mechanisms, the first weight value of packet loss can be set to 0.

For large connection traffic, mctc traffic packet size and throughput requirements are typically small, but device density is high. The biggest challenges and concerns are both the network's support capability for high connection density and the network's energy efficiency. The conventional service quality evaluation is to evaluate the service from the service experience and transmission quality view, but the mMTC service has low transmission requirement, but has high cost on network resources, and has high influence on the network energy consumption utilization rate, and is also a key evaluation dimension from the service networking view, and the aim is that the quality can be ensured, and meanwhile, the resource occupation is minimum and the energy cost is lowest. The service mainly depends on an application layer to ensure the reliability of data transmission, the requirements on bandwidth and time delay are not high, the first weight value can be set to be 1, and the first weight value of packet loss can be set to be 2.

By combining the above embodiments, the service types, the selected communication indexes and the formulated evaluation rules for dividing the communication service of the 5G private network provided by the application can evaluate the communication quality of the 5G private network. Before the evaluation, a general evaluation model or an evaluation template can be established, so that different 5G private networks can be provided with different templates, the application of the evaluation method of the 5G private networks is more flexible, and the application is wider and more general. For example, fig. 2 is a method for establishing a quality assessment model of a 5G private network, where the method includes the following steps:

s1: and determining the communication service executed by the 5G private network to be evaluated, namely, performing service combing on the communication service deployed by the 5G private network to obtain the communication service executed by the 5G private network, or executing part of main stream service, etc., for example, in S1, it may be determined that I communication tasks executed by the 5G private network are as shown in the following table 7.

TABLE 7

S2: and classifying the communication service determined in the S1 according to a plurality of preset service categories such as uplink large bandwidth service, downlink large bandwidth service, real-time interaction service, large connection service and the like, and dividing each communication service into one or more of the preset service categories. For example, after splitting the I traffic in table 7 into J traffic subclasses, the traffic classes corresponding to the K different traffic classes may be respectively obtained, and finally, the classification result shown in table 8 is obtained.

TABLE 8

S3: determining the communication index of each communication service in S2, establishing the corresponding relation between the communication service and the corresponding communication index in form of a table or matrix, namely generating a network quality evaluation table, realizing service index selection and grading definition of each classified service, giving out the key index grading standard of the related service, and defining a network index evaluation template; and generates a network quality assessment table of the private network service according to the private network service classification dimension as shown in table 9 below.

TABLE 9

When the above communication indexes of the plurality of communication services of the 5G private network are determined, the communication indexes can be substituted into a list of network index evaluation in table 9, and the communication indexes filled into the network index evaluation in table 9 are marked as a network index evaluation result matrix R, which can be represented by the following formula one:

wherein r is used to represent a communication index corresponding to one service or a subclass of one service filled in table 9, J is the subclass number divided by a plurality of communication services, K is the number of communication indexes affecting the communication service, and K corresponds to up 4 communication indexes, down 4 communication indexes and 1 service specific index in table 9 in total to 9 columns.

S4: determining a first weight value of the current 5G private network, namely generating a network index weight table: the default weights/impact factors templates of the various indexes are checked, and the factors are adjusted to form the network index weights/impact factors as shown in table 10, it being understood that table 6 is an example implementation of a specific value of table 10.

Table 10

Wherein f is the weight/influence factor of a certain type of service index, and is marked as a first weight value. It can be understood that in the practical application process, the weight/influence factor of the service index corresponding to the 5G private network can be adjusted for the service subclass after different services in the private network to be evaluated, so as to form fj, k in table 10, and the service subclass weight/influence factor matrix F finally forming the private network can be represented by the following formula two:

s5: setting an inter-service initial weight table: the weight adjustment between each service is performed for the private network traffic occupied by each service, and a service weight matrix shown in table 11 is formed and denoted as W.

TABLE 11

Wherein w is _,j And for the service subclass weight, W is a service subclass weight matrix, and the service subclass weight matrix can be expressed by the following formula III, wherein the initial default weight in the formula III is the same as each type of service weight, and can be automatically adjusted in real time according to the service duty ratio of the service in the private network.

S6: and (3) establishing a calculation model for 5G private network quality assessment according to the parameters set in S1-S5.

Specifically, for each communication service executed by the 5G private network, a calculation formula defining a service evaluation parameter related to the communication service is expressed as the following formula four:

Wherein q is a service evaluation parameter of each communication service, J is the number of service subclasses, R is a matrix generated by the formula one, and F is a matrix generated by the formula two. Then, after the subclasses belonging to the same communication service are weighted and averaged according to the following formula five, a quality score of the communication service which is executed by the 5G private network and can be divided into a plurality of service classes can be obtained and recorded as a service evaluation parameter of the communication service.

Where i is the ith communication service in the 5G private network, which can be broken down into m to n service subclasses in table 8. And finally, calculating the service evaluation parameters of all the communication services to obtain a service evaluation matrix with the following formula six, wherein I is the number of the communication services executed by the 5G private network.

Subsequently, a communication quality evaluation threshold for the communication service of the 5G private network to be evaluated may be set, and the communication quality level of each communication service may be determined by comparing the service evaluation parameter of each communication service in the sixth formula with the communication quality evaluation threshold, where the correspondence is as shown in table 12 below.

Table 12

In some embodiments, for the overall communication quality of the 5G private network, when calculating the communication quality parameter related to the communication traffic, a weight value of each communication task for the communication quality parameter of the 5G private network may be determined for the communication traffic of the plurality of communication traffic performed by the 5G private network and recorded as a second weight value.

For example, the traffic duty matrix T may be defined for I traffic performed by the 5G private network as represented by the following equation seven:

wherein UP_SERVICE __ TRAFF_RATIO _i The ith communication service of the 5G private network accounts for the proportion of the total service flow of the 5G private network. And then, combining the formula six and the formula seven, calculating a second weight value correction considering the traffic flow ratio through the following formula eight, wherein the communication quality parameter of the 5G private network is calculated:

or, the formula III and the formula IV can be combined, and the communication quality parameter of the 5G private network without considering the service flow ratio is calculated through the following formula III:

Q _PN ＝[Q] ^T x[W]formula nine

Finally, a communication quality evaluation threshold for the 5G private network to be evaluated may be set, and the communication quality parameters calculated by the above formula eight or formula nine are compared with the communication quality evaluation threshold to determine the communication quality level of the 5G private network, where the correspondence is shown in the following table 13.

TABLE 13

According to the quality evaluation model of the 5G private network established by the method shown in fig. 2, the present application further provides a method for performing quality evaluation on the 5G private network by using the model, and the execution subject of the method may be electronic equipment such as a computer, a server, etc., and the present application describes a process of executing the quality evaluation method of the 5G private network by using the electronic equipment as the execution subject. For example, fig. 3 is a flow chart of an embodiment of a quality evaluation method for a 5G private network provided in the present application, and as shown in fig. 3, the quality evaluation method for a 5G private network provided in the present embodiment includes:

S101: the electronic equipment acquires a plurality of communication services executed by the 5G private network to be evaluated and a plurality of communication indexes of each communication service.

In some embodiments, the electronic device may perform quality assessment on the 5G private network to be assessed according to an instruction of a worker of the operator, or the electronic device may perform quality assessment on the preset 5G private network at a specified time according to a preset task. In S101, in order to perform quality assessment on the 5G private network, a plurality of communication services executed by the 5G private network and a communication index of each communication service are first determined, where the communication index includes: communication rate, communication delay, communication jitter, packet loss rate, etc.

In some embodiments, in S101, the electronic device may specifically obtain, by means of deep packet inspection (Deep packet inspection, abbreviated as DPI), a traffic flow, a message content, a communication parameter, etc. of the 5G private network to be evaluated within a preset period of time, and then determine a communication service of the 5G private network according to the obtained traffic flow, message content, and communication parameter, and calculate a communication index of each communication service.

For example, fig. 4 is a system deployment structure diagram for DPI acquisition of a 5G private network, where a DPI signaling acquisition system performs bypass spectroscopy on related network interface links, performs real-time analysis on data in the links, and generates signaling details (SDR, for the signaling portion) and traffic details (FDR, for the user plane traffic data). And all detail lists (SDR and FDR, which are called XDR collectively) are transmitted to a DPI synthesizer for data association, and XDR original ticket of a signaling plane and a user plane is generated. And filtering the XDR ticket belonging to the private network through the IMSI belonging to the private network. The service-based network quality assessment index is generated based primarily on user plane XDR ticket calculations. And then, extracting service related indexes according to the dimensions of 5 minutes, 1 hour and 24 hours to form a user face width table, combining the user face width table by taking a service name (SERVNAME) as a reference as shown in the following table 14 to obtain XDR network indexes counted according to the service dimensions, and obtaining related network performance KPIs, wherein the time granularity is correspondingly 5 minutes, 1 hour and 24 hours.

TABLE 14

NCI	string	cellid
			DNN	string	apn
MCC	string	National code
			MNC	string	Operator network code
GPSI	string	msisdn
			SUPI	STRING	imsi
PEI	string	pei
			tacid	string	8 first bits of pei
SERVNAME	string	Unique value of business subclass
			SERVICE_TYPE	string	Destination IP
Network index 1	…	…
			Network index 2	…	…
Network index 3	…	…
			…	…	…

Then, based on the obtained content, a communication index of the 5G private network may be calculated, wherein the traffic flow: the uplink traffic flow up_service_ul_traff, the downlink traffic flow up_service_dl_traff and the TOTAL traffic flow up_service_total_traff can be obtained through the user face width table parameters, and then the traffic volume ratio of each communication SERVICE i in a plurality of communication SERVICEs can be calculated through the following formula ten, so that a traffic volume ratio matrix T in the formula seven is finally formed.

The traffic rate, i.e., the communication rate in the communication index, is calculated by the following formula eleven, wherein the average traffic rate of each user is calculated by calculating the traffic rate due to the difference of traffic flow and TIME duration of each user, up_service_ul/dl_traffic is the UP/down traffic flow, up_ul_dura_total_time is the UP/down TIME duration TOTAL TIME duration.

The average rate of traffic is calculated by the following formula twelve, where L is the number of users using the traffic in the time granularity.

The communication DELAY in the communication index is calculated according to the following formula thirteen, wherein for TCP services, such as HTTP web browsing, the uplink and downlink TCP RTT DELAYs are selected as the end-to-end DELAYs of the services, so that the uplink RTT TOTAL DELAY UP_UL_RTT_TOTAL_TIME_DELAY and the downlink RTT TOTAL DELAY UP_DL_RTT_TOTAL_TIME_DELAY can be obtained, and for services borne on UDP, the application protocol above UDP needs to be analyzed to objectively reflect the service DELAYs. Such as multimedia services, require analysis of the RTP protocol, etc.

Up_rtp_ul/dl_packet is the RTP UP/down report number, and up_rtp_ul/dl_duration is the RTP UP/down service DURATION.

The communication Jitter in the obtained communication index comprises an uplink Jitter time delay UP_UL_jitter_Sum, a downlink Jitter time delay UP_DL_jitter_Sum, an uplink Jitter statistical number UP_UL_jitter_count, a downlink Jitter statistical number UP_DL_jitter_count, an uplink maximum Jitter UP_UL_jitter_Max and a downlink maximum Jitter UP_DL_jitter_Max.

Calculating the packet loss rate in the communication index according to the following formula fourteen, wherein TCP ensures no packet loss by means of a retransmission mechanism; services carried on UPD need to rely on upper layer protocols to count packet losses. For example, multimedia services, an RTP protocol needs to be analyzed, and an RTP uplink PACKET loss number ul_rtp_ul/dl_low_packet and an RTP uplink/downlink TOTAL PACKET number up_rtp_ul/dl_total_packet can be obtained.

S102: the electronic device classifies the plurality of communication services acquired in S101, and classifies the plurality of communication services into one or more service classes according to a plurality of preset service classes.

In some embodiments, the traffic class includes: uplink large bandwidth service, downlink large bandwidth service, real-time interactive service and large connection service. The electronic device may classify the plurality of communication services in the manner shown in table 9 in S102, and determine the communication index corresponding to each communication service, so as to obtain a network index evaluation result matrix R according to a formula for subsequent calculation.

S103: the electronic equipment determines first weight values of a plurality of communication indexes corresponding to each business category in the business categories.

In some embodiments, the electronic device may determine the first weight values of the plurality of communication indexes corresponding to different traffic classes shown in table 10 in S103, and form the first weight value matrix F in the second formula for subsequent calculation according to the first weight values of the plurality of communication indexes of all traffic classes.

In some embodiments, the first weight value may be preset or may be specified by an operator, for example, may be stored in the electronic device in advance, or may be acquired by the electronic device through the internet. Alternatively, the first weight value may be set according to different conditions, for example, one first weight value is used by the 5G private network in the a region, another weight value is used by the 5G private network in the B region, and so on; or the granularity of the first weight value is increased by using the first weight value in daytime, using the other weight value at night, and the like, so that the accuracy of quality assessment is improved.

S104: and the electronic equipment performs weighting processing on the communication indexes of the plurality of communication services according to the first weight value to obtain service evaluation parameters of the plurality of communication services.

In some embodiments, the electronic device specifically performs weighting processing on the matrix R obtained in S102 according to the matrix F obtained in S103, calculates a service evaluation parameter of each of the plurality of communication services according to a fourth formula, and forms a service evaluation parameter matrix in a sixth formula.

More specifically, for a first communication service classified into a plurality of service classes, the first communication service is a communication service corresponding to a plurality of subclasses in table 11, after calculating service evaluation sub-parameters of the plurality of service classes corresponding to the first communication service through a formula four, obtaining a service evaluation parameter corresponding to the first communication service through a method of weighting and averaging all the service evaluation sub-parameters through a formula five, wherein a weighted value is marked as a third weighted value.

S105: the electronic equipment determines a second weight value corresponding to each communication service in the plurality of communication services. The second weight value corresponding to each communication service can be preset and can be specified by an operator; alternatively, the second weight value may be further determined according to a ratio of communication traffic of a plurality of communication services of the 5G private network in the total communication traffic.

For example, in some embodiments, the electronic device may determine, in S105, a traffic duty cycle matrix T for subsequent computation in a manner shown by equation seven, where the matrix includes a duty cycle of traffic flows (or referred to as traffic flows) of I traffic of the 5G private network at traffic flows of the 5G private network. The communication service with larger communication service flow can be distributed with larger weight value when the quality of the 5G private network is evaluated, so that the quality of the 5G private network can be evaluated more comprehensively and effectively.

S106: and (5) weighting the service evaluation parameters of the plurality of communication services according to the second weight value determined in the step (105) to obtain the communication quality parameters of the 5G private network.

In some embodiments, in S106, the electronic device may specifically calculate the communication quality parameter of the 5G private network through a calculation manner of the formula eight, so that the calculated communication quality parameter may be used for subsequent evaluation of the communication quality of the 5G private network.

Then, after obtaining the communication quality parameters of the 5G private network, the electronic device may determine, according to the manner shown in table 13, that the communication quality level of the 5G private network is excellent, good or alarm according to the first numerical interval in which the calculated communication quality parameters are located and the first mapping relationship in table 13.

And after obtaining the service evaluation parameters of the plurality of communication services, the electronic device may also determine that the communication quality level of the communication service is excellent, good or alarm according to the second mapping relationship in table 12 and the second numerical interval where the calculated service evaluation parameters are located in a manner as shown in table 12.

In some embodiments, after the electronic device determines the communication quality level of the 5G private network and the communication quality level of each communication service, the obtained communication quality level of the 5G private network and/or the communication quality levels of the plurality of communication services are displayed on a display interface such as a display. For example, fig. 5 is a schematic diagram showing the effect of an embodiment of the display interface provided in the present application, where the electronic device displays that the communication quality level of the 5G private network is "good" and displays the communication quality level of each communication service 1-I and detailed indicators of the subclass corresponding to each communication service on the display interface.

Fig. 6 is a flowchart of another embodiment of a quality evaluation method for a 5G private network provided in the present application, which illustrates a specific implementation manner of the method shown in fig. 4, where after DIP data collection is performed by an electronic device as an executing body, DIP data analysis is performed and network basic performance indexes and the like are extracted, so as to determine a plurality of communication services and communication indexes of the 5G private network. Then, the electronic device inputs the communication indexes into the network index evaluation template established in fig. 2 to obtain index scores shown in table 9, and further inputs the index scores shown in table 9 into the network quality evaluation template determined by the first weight value in table 10, so as to calculate service evaluation parameters of each service, and obtain the quality overall score of the whole 5G private network through a formula nine under the condition that the flow of each communication service is not considered. And when the communication flow of each service is considered, the flow of each communication service in the 5G private network can be obtained according to DIP data analysis, a service volume duty ratio matrix in a formula seven is obtained, then a second weight value is adjusted according to the service volume duty ratio, and finally the quality overall score of the 5G private network, namely the communication quality parameter, is obtained through a formula eight.

In summary, the quality evaluation method for a 5G private network provided in the embodiment of the present application can evaluate, for different types of services provided by the 5G private network, the quality of the provided service and the communication quality of the entire 5G private network, and particularly, under the service types of the different 5G private networks, calculate, by using different first weighting values, service evaluation parameters of communication services for communication indexes of the different service types, and calculate, by using second weighting values, the communication quality of the entire 5G private network according to service flows of all multiple communication services of the 5G private network. The operator can set and maintain the 5G private network better according to the evaluation result, and normal operation and daily maintenance of the 5G private network are ensured. In addition, the quality evaluation method of the 5G private network can adaptively adjust and calculate the weight value of the communication quality parameter in real time according to the change of the service flow of the 5G private network, so that the current communication quality of the 5G private network is evaluated more truly and effectively, and the evaluation result has accuracy and timeliness.

In the foregoing embodiments, the method for evaluating quality of a 5G private network provided in the embodiments of the present application is described, and in order to implement each function in the method for evaluating quality of a 5G private network provided in the embodiments of the present application, an electronic device as an execution body may include a hardware structure and/or a software module, and implement each function in the form of a hardware structure, a software module, or a hardware structure and a software module. Some of the functions described above are performed in a hardware configuration, a software module, or a combination of hardware and software modules, depending on the specific application of the solution and design constraints.

For example, fig. 7 is a schematic structural diagram of an embodiment of a quality evaluation device for a 5G private network provided in the present application, and as shown in fig. 7, the present application further provides a quality evaluation device for a 5G private network, including: an acquisition module 1001, a classification module 1002, a first determination module 1003, a traffic assessment module 1004, a second determination module 1005 and a quality assessment module 1006. The acquiring module 1001 is configured to acquire a plurality of communication services executed by the 5G private network to be evaluated, and a plurality of communication indexes of each communication service; the classification module 1002 is configured to classify the plurality of communication services according to a plurality of preset service classes; the first determining module 1003 is configured to determine a first weight value of the plurality of communication indexes corresponding to each traffic class in the plurality of traffic classes; the service evaluation module 1004 is configured to perform weighting processing on the communication indexes of the plurality of communication services according to the first weight value, so as to obtain service evaluation parameters of the plurality of communication services; the second determining module 1005 is configured to determine a second weight value corresponding to each communication service in the plurality of communication services; the quality evaluation module 1006 is configured to perform weighting processing on service evaluation parameters of the plurality of communication services according to the second weight value, so as to obtain communication quality parameters; the communication quality parameter is used for evaluating the communication quality of the 5G private network.

Specifically, the specific principle and implementation manner of the above steps executed by each module in the quality assessment device of the 5G private network may refer to the description in the quality assessment method of the 5G private network in the foregoing embodiments of the present application, and will not be repeated.

It should be noted that, it should be understood that the division of the modules of the above apparatus is merely a division of a logic function, and may be fully or partially integrated into a physical entity or may be physically separated. And these modules may all be implemented in software in the form of calls by the processing element; or can be realized in hardware; the method can also be realized in a form of calling software by a processing element, and the method can be realized in a form of hardware by a part of modules. The function of the above determination module may be implemented as a processing element that is set up separately, or may be integrated into a chip of the above apparatus, or may be stored in a memory of the above apparatus in the form of program codes, and may be called and executed by a processing element of the above apparatus. The implementation of the other modules is similar. In addition, all or part of the modules can be integrated together or can be independently implemented. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in a software form.

For example, the modules above may be one or more integrated circuits configured to implement the methods above, such as: one or more specific integrated circuits (application specific integrated circuit, ASIC), or one or more microprocessors (digital signal processor, DSP), or one or more field programmable gate arrays (field programmable gate array, FPGA), or the like. For another example, when a module above is implemented in the form of a processing element scheduler code, the processing element may be a general purpose processor, such as a central processing unit (central processing unit, CPU) or other processor that may invoke the program code. For another example, the modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), etc.

For example, fig. 8 is a schematic structural diagram of an embodiment of an electronic device provided in the present application, and as shown in fig. 8, the present application further provides an electronic device 200, including: a processor 201 and a memory 202 connected by a bus; wherein the memory 202 has stored therein a computer program which, when executed by the processor 201, is operable to perform a method of quality assessment of a 5G private network as in any of the previous embodiments of the present application. In some embodiments, the processor 201 may also interact with other devices through the communication interface 203.

The present application also provides a computer readable storage medium storing a computer program which when executed is operable to perform a quality assessment method for a 5G private network as in any of the previous embodiments of the present application.

The embodiment of the application also provides a chip for running the instructions, and the chip is used for executing the quality assessment method of the 5G private network, which is executed by the electronic equipment in any embodiment of the application.

Those of ordinary skill in the art will appreciate that: all or part of the steps for implementing the method embodiments described above may be performed by hardware associated with program instructions. The foregoing program may be stored in a computer readable storage medium. The program, when executed, performs steps including the method embodiments described above; and the aforementioned storage medium includes: various media that can store program code, such as ROM, RAM, magnetic or optical disks.

Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. The quality evaluation method of the 5G private network is characterized by comprising the following steps of:

acquiring a plurality of communication services executed by a 5G private network to be evaluated and a plurality of communication indexes of each communication service;

classifying the plurality of communication services according to a plurality of preset service categories;

determining a first weight value of the plurality of communication indexes corresponding to each business category in the plurality of business categories;

weighting the communication indexes of the plurality of communication services according to the first weight value to obtain service evaluation parameters of the plurality of communication services;

determining a second weight value corresponding to each communication service in the plurality of communication services;

Weighting the service evaluation parameters of the plurality of communication services according to the second weight value to obtain communication quality parameters; the communication quality parameter is used for evaluating the communication quality of the 5G private network;

the communication index includes: communication speed, communication time delay, communication jitter and packet loss rate;

the communication service class includes: uplink large bandwidth service, downlink large bandwidth service, real-time interactive service and large connection service;

the classifying the plurality of communication services according to a plurality of preset service categories includes:

classifying each of the plurality of communication services into one or more of the plurality of service classes according to a preset plurality of service classes;

for a first communication service classified into a plurality of service classes in the plurality of communication services, the step of obtaining a service evaluation parameter of the first communication service includes:

according to the first weight value, weighting the communication index corresponding to each service class in the first communication service to obtain service evaluation sub-parameters of the first communication service corresponding to a plurality of service classes;

And weighting the service evaluation sub-parameters of the plurality of service categories according to the third weight value of each service category corresponding to the first communication service to obtain the service evaluation parameters of the first communication service.

2. The method of claim 1, wherein determining a second weight value for each of the plurality of communication services comprises:

and taking the ratio of the communication flow of each communication service in the 5G private network when the 5G private network executes the plurality of communication services in the total communication flow of the 5G private network as a second weight value corresponding to each communication service in the plurality of communication services.

3. The method according to claim 1, wherein the obtaining a plurality of communication services performed by the 5G private network to be evaluated, and a communication index of each of the communication services, comprises:

acquiring service flow, message content and communication parameters of the 5G private network in a preset time period according to a Deep Packet Inspection (DPI) technology;

and determining the communication service carried out by the 5G private network according to the service flow, the message content and the communication parameters, and calculating the communication index of each communication service.

4. The method according to claim 1, wherein after the obtaining the communication quality parameter, further comprising:

determining the communication quality level of the 5G private network according to a first numerical interval in which the communication quality parameter is located and a first mapping relation; the first mapping relation comprises a plurality of first numerical intervals and communication quality levels corresponding to the first numerical intervals;

determining a communication quality level of each communication service according to a second numerical interval and a second mapping relation of a service evaluation parameter of each communication service in the plurality of communication services; the second mapping relation comprises a plurality of second numerical intervals and communication quality levels corresponding to the second numerical intervals.

5. The method according to claim 4, wherein the method further comprises:

and displaying the communication quality level of the 5G private network and/or the communication quality levels of the plurality of communication services on a display interface.

6. A quality assessment device for a 5G private network, comprising:

the acquisition module is used for acquiring a plurality of communication services executed by the 5G private network to be evaluated and a plurality of communication indexes of each communication service;

The classification module is used for classifying the plurality of communication services according to a plurality of preset service categories;

a first determining module, configured to determine a first weight value of the plurality of communication indexes corresponding to each service class in the plurality of service classes;

the service evaluation module is used for carrying out weighting processing on the communication indexes of the plurality of communication services according to the first weight value to obtain service evaluation parameters of the plurality of communication services;

a second determining module, configured to determine a second weight value corresponding to each communication service in the plurality of communication services;

the quality evaluation module is used for carrying out weighting processing on the service evaluation parameters of the plurality of communication services according to the second weight value to obtain communication quality parameters; the communication quality parameter is used for evaluating the communication quality of the 5G private network;

the classification module is specifically configured to classify each communication service of the plurality of communication services into one or more service classes of the plurality of service classes according to a preset plurality of service classes;

The service evaluation module is specifically configured to, when a service evaluation parameter of a first communication service is obtained for a first communication service classified into a plurality of service classes in the plurality of communication services, perform a weighting process on a communication index corresponding to each service class in the first communication service according to the first weight value, so as to obtain a service evaluation subparameter of the first communication service corresponding to the plurality of service classes; and weighting the service evaluation sub-parameters of the plurality of service categories according to the third weight value of each service category corresponding to the first communication service to obtain the service evaluation parameters of the first communication service.

7. An electronic device, comprising: a processor and a memory; wherein the memory has stored therein a computer program which, when executed by the processor, is operable to perform the quality assessment method of a 5G private network according to any of claims 1-5.

8. A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program, which when executed is operable to perform the quality assessment method of a 5G private network according to any of claims 1-5.