CN111343484A

CN111343484A - IPTV/OTT intelligent quality alarm method based on artificial intelligence

Info

Publication number: CN111343484A
Application number: CN201811553346.9A
Authority: CN
Inventors: 马学嘉
Original assignee: Vixtel Technologies Beijing Co ltd
Current assignee: Vixtel Technologies Beijing Co ltd
Priority date: 2018-12-19
Filing date: 2018-12-19
Publication date: 2020-06-26

Abstract

The invention discloses an IPTV/OTT intelligent quality alarm method based on artificial intelligence, belonging to the technical field of communication. The invention is based on the reported data of the set-top box soft probe, and adopts the data of the integrated resource management system of the associated operator and the like to generate 4-dimensional resource trees and related label information of networks, geography, platforms, terminal manufacturers and the like, and the invention comprises the following steps: a) the real-time stream processing and artificial intelligence Diversity Density (DD) algorithm is adopted, so that the clustering accuracy is improved while the real-time performance and the high efficiency of data processing are ensured, and the computational power consumption is reduced. b) The system self-summarizes a set of alarm system including alarm indexes, threshold values, clustering proportions and the like to form an alarm model through continuous testing verification and simulated fault data deep learning, and continuous self-learning and self-calibration are completed through daily production and use.

Description

IPTV/OTT intelligent quality alarm method based on artificial intelligence

Technical Field

The invention belongs to the technical field of communication, relates to an IPTV/OTT quality monitoring, quality alarm monitoring and automatic operation and maintenance system, and relates to IPTV/OTT and operator related network management and support systems.

Background

IPTV/OTT becomes an important basic business of operators in various regions in the digital home market strategy. At present, a soft probe is embedded into a mainstream through a set top box, so that the IPTV/OTT quality monitoring and fault fast delimiting and positioning capabilities are realized; however, as the user scale rapidly increases, mass data generated by the soft probe also brings great challenges to quality analysis and fault convergence; the main manifestations are as follows:

1) the data volume is huge, and early warning real-time nature requires highly: an operator of a million-level customer reports various audience rating and quality data in real time, each piece of data needs to be matched with about ten million pieces of resource data in total, then various dimensions are gathered, the real-time calculation amount is large, and according to the activity of 50%, 500 million and 500 million pieces of data need to be subjected to cross matching and real-time clustering in each period;

2) the noise data is many: due to uncertainty of user behaviors, the soft probe can report various useless and invalid data caused by the user behaviors, for example, the watching time is extremely short, a large amount of noise data can be generated when the user repeatedly enters or exits, and the noise data identification and cleaning are another big challenge;

3) index diversity: the IPTV/OTT live broadcast, on-demand broadcast and review of the existing network adopt completely different protocols, each type of service has more than 100 indexes and the main indexes are completely different, the index with the strongest correlation of the corresponding cluster type needs to be found out, and the strongest correlation index can be changed at every moment and under different service scenes and application;

the system for warning based on the clear rule consumes huge computing resources, meanwhile, the warning accuracy rate of quality degradation caused by uncertainty of user behaviors and index diversity is extremely low, a large amount of wrong warnings are generated, and under the background, the IPTV OTT quality warning is optimized and improved by innovatively utilizing the artificial intelligence technology.

Disclosure of Invention

The invention adopts a set-top box soft probe to report data base, generates 4-dimensional resource trees and related label information of networks, geography, platforms, terminal manufacturers and the like by adopting data of a related operator comprehensive resource management system and the like, outputs equipment and link alarm data of each layer according to set-top box degradation data and live broadcast channel degradation data, and adopts the following technologies to solve the problems:

1) the real-time stream processing and artificial intelligence Diversity Density (DD) algorithm is adopted, the clustering accuracy is improved while the real-time performance and the high efficiency of data processing are ensured, the computational power consumption is reduced, and the full data real-time alarm clustering time granularity is 5 minutes as shown in the attached figure 1;

2) the intelligent alarm model: the system self-summarizes a set of alarm system including alarm indexes, threshold values, clustering proportions and the like to form an alarm model through continuous testing verification and simulated fault data deep learning, and continuous self-learning and self-calibration are completed through daily production and use.

Drawings

FIG. 1 is a diagram of a raw mass data model architecture;

FIG. 2 is a block diagram of a data acquisition layer module;

FIG. 3 is a block diagram of a deep learning layer;

FIG. 4 is a diagram of an intelligent decision presentation architecture.

Detailed Description

In actual production, production data including user behaviors, time sequences, network batch marks, service indexes, terminal performance, complaint data, QoE indexes, network logs and the like are acquired by arranging a data acquisition layer, a basic data sharing pool is formed as shown in figure 2, data in the sharing pool is equivalent to food for machine learning, and the machine learning capacity is fed by the food. And performing data analysis and data mining on the metropolitan area network link information, the metropolitan area network service information and the set top box information by using a big data technology, presenting the metropolitan area network link quality, the service quality and the user access hot door program access quality for the user in real time, analyzing and counting network related complaints, and displaying the types and the occupation ratios of the complaints.

The following detailed description of embodiments of the invention, which mainly covers three parts, will be described in detail, and the associated drawings referred to in the description are shown in the accompanying drawings.

1. Deep learning layer

And the machine learning layer acquires data from the basic data sharing pool, sorts the data, eliminates abnormal data, classifies and stores the data according to time dimension, type and index dimension, and facilitates web search and query calling of the data. This stage corresponds to digestion and filtration of the food to obtain the nutrients therein, and the study diagram is shown in fig. 3.

(1) IPTV/OTT data analysis: and carrying out multi-dimensional analysis on the data of the same type, carrying out time day-to-ring ratio, week-to-ring ratio and month-to-ring ratio on the data, and carrying out lifting marking on the data.

(2) IPTV/OTT quality difference score: and (4) scoring the data of different types and indexes according to set interval values, distributing the index scores by different weights, and finally obtaining the comprehensive score condition of the service.

2 Intelligent judgment presentation layer

The layer is divided into three functional modules, the logic judgment functional module judges the condition of the real environment according to the perception score obtained by the machine learning layer, judges whether the data indexes are generally ascending or descending, and judges the influence on other environmental factors or other data through data. And the judgment and verification module continuously judges and verifies the future data according to the continuity of the data, and simultaneously calculates the accuracy of the judgment result and continuously corrects the judgment mechanism. The presentation module presents the data and the judgment result in the form of graphics and text, as shown in fig. 4.

3 intelligent IPTV OTT alarm model

The algorithm adopted by the intelligent IPTV OTT alarm is a Diversity Density (DD) algorithm proposed by O.Maron et al [5] in 1998. The basic idea is as follows: assuming that one IPTV/OTT soft probe data instance is represented by one feature vector, the space made up of all instance features is referred to as the instance feature space. An example is considered as a point in space, a packet has a number of examples, and the locus of points in space for these examples is considered as a manifold. A target concept point, i.e. the intersection of multiple positive packet manifolds, is found that satisfies the condition that each positive packet manifold passes through this point, and that no negative packet manifold passes through this point. The DD algorithm considers the examples and packets as obeying a certain probability distribution, defining a diversity density function, the function value of a point (example) in the feature space, that is, the probability value that this point satisfies the potential distribution of positive and negative packets. One example has a DD value, and the example with the largest DD value (existing only in the positive packet) is found as the target concept point. Then, taking the target concept point as a reference, calculating the distance between each example in the new package and the point, and judging the label of the new package according to whether the minimum distance is within a threshold value range.

Order to

An ith positive packet representing the composition of data collected by the system,

represents the jth instance of the ith positive packet,

a k attribute value representing a j example of an i-th positive packet; at the same time, order

Represents the ith negative packet (the initial negative packet can be set by adopting a manual intervention mode and referring to relevant empirical data),

represents the jth instance of the ith negative packet,

the kth attribute value of the jth example representing the ith negative packet. The concept point t represents the point with the maximum DD value, the target concept point t is determined by maximizing the objective function, the packages are mutually independent, all examples have the same prior probability, and the point t with the maximum DD function can be determined by the following formula according to the Bayesian decision theory.

Wherein Ω is a training packet set for machine learning of the system, in practical use, at least one example in the positive packet is taken as a positive example, all examples in the negative packet are taken as negative examples, and the probability that an example becomes a potential target concept point t is defined as the distance between the example and the point t, that is, the distance between the example and the point t

Since some features may be irrelevant or some features need to have a higher weight, the distance between weighted Euclidean distance measure examples is used, the weight W_kThe weight of the k-th feature is expressed by the following formula

The DD algorithm takes each example in all the positive packets as an initial search point, obtains a plurality of local extreme points by using gradient ascending iterative search for a plurality of times, and finally obtains an optimal value through comparison, wherein the optimal value is the optimal value presented by the system under the current learning capability.

An alarm logic rule is continuously formed with deep learning through a DD algorithm for adaptation, the obtained optimal value is the maximum fault position probability point, namely an alarm point, and 4-dimensional alarm output including network dimension, geographical dimension, terminal dimension and service dimension is formed through the DD algorithm.

Claims

1. The IPTV/OTT alarm point based on the optimal value of the Diversity Density (DD) algorithm is a key point of the description, the key point is formed by a large number of verification and practice results, a large number of artificial intelligence algorithms and machine learning mechanisms are tried, and meanwhile the method is protected from being applied to other operation and maintenance alarm scenes.

2. The method is characterized in that the optimal value of the DD algorithm is used as an alarm point, an operation and maintenance system and a system are an industrial innovation mechanism, the consumption of alarm calculation resources is greatly reduced, and the alarm efficiency is improved. The mechanism has the same value for other systems and service alarm optimization.