WO2024066720A1 - Procédé et appareil de détermination de seuil d'indicateur, support de stockage et appareil électronique - Google Patents

Procédé et appareil de détermination de seuil d'indicateur, support de stockage et appareil électronique Download PDF

Info

Publication number
WO2024066720A1
WO2024066720A1 PCT/CN2023/110331 CN2023110331W WO2024066720A1 WO 2024066720 A1 WO2024066720 A1 WO 2024066720A1 CN 2023110331 W CN2023110331 W CN 2023110331W WO 2024066720 A1 WO2024066720 A1 WO 2024066720A1
Authority
WO
WIPO (PCT)
Prior art keywords
indicator
threshold
data
coordinate
value
Prior art date
Application number
PCT/CN2023/110331
Other languages
English (en)
Chinese (zh)
Inventor
杨伟伟
冯媛
邵敏峰
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2024066720A1 publication Critical patent/WO2024066720A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
    • G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/067—Enterprise or organisation modelling
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06—Management of faults, events, alarms or notifications
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06—Management of faults, events, alarms or notifications
    • H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis

Definitions

  • the present disclosure relates to the field of big data and artificial intelligence technology, and in particular, to a method, device, storage medium and electronic device for determining an indicator threshold.
  • threshold setting such as anomaly detection, root cause analysis, data prediction, alarm management, intelligent recovery, and perception evaluation.
  • wireless network operators set thresholds for service indicators mainly based on fixed empirical thresholds of indicators or dynamic thresholds obtained from relatively complex statistical distributions. Even if dynamic thresholds based on mathematical methods such as statistical distribution are used, the threshold solution problem is converted into a threshold setting problem in another dimension, which makes it difficult to accurately and objectively measure the pros and cons of service indicators, and thus effectively guide network O&M and analysis and achieve the goal of maximizing data value.
  • the embodiments of the present disclosure provide a method, device, storage medium and electronic device for determining an indicator threshold, so as to at least solve the problem of how to determine the indicator threshold.
  • a method for determining an indicator threshold comprising: obtaining aggregated indicator data corresponding to a target indicator; determining an indicator data set from the aggregated indicator data, sorting first indicator data in the indicator data set to obtain second indicator data, clustering the second indicator data to obtain a plurality of clustered groups, and fitting the indicator data of each of the plurality of groups to obtain a piecewise function corresponding to each group, wherein the same indicator data set represents indicator data of the same monitored object; and determining an indicator threshold from a set of intersection coordinates of the piecewise functions according to the indicator bias of the target indicator.
  • a device for determining an indicator threshold comprising: an acquisition module configured to acquire aggregate indicator data corresponding to a target indicator; a first determination module configured to determine a target indicator from the aggregate indicator data; Determine an indicator data set, sort the first indicator data in the indicator data set to obtain second indicator data, cluster the second indicator data to obtain multiple groups after clustering, and fit the indicator data of each group of the multiple groups to obtain a piecewise function corresponding to each group, wherein the same indicator data set represents the indicator data of the same monitored object; a second determination module is configured to determine an indicator threshold from the intersection coordinate set of the piecewise function according to the indicator bias of the target indicator.
  • a computer-readable storage medium in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned method for determining the indicator threshold value when running.
  • an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the method for determining the indicator threshold through the computer program.
  • aggregated indicator data corresponding to a target indicator is obtained; an indicator data set is determined from the aggregated indicator data, first indicator data in the indicator data set is sorted to obtain second indicator data, the second indicator data is clustered to obtain a plurality of clustered groups, and the indicator data of each of the plurality of groups is fitted to obtain a piecewise function corresponding to each group, wherein the same indicator data set represents indicator data of the same monitored object; an indicator threshold is determined from the intersection coordinate set of the piecewise function according to the indicator bias of the target indicator, thereby solving the technical problem of how to determine the indicator threshold.
  • FIG1 is a hardware structure block diagram of a computer terminal of a method for determining an indicator threshold value according to an embodiment of the present disclosure
  • FIG2 is a flow chart of a method for determining an indicator threshold according to an embodiment of the present disclosure
  • FIG3 is a schematic diagram of two-dimensional discrete points according to an embodiment of the present disclosure.
  • FIG. 4 is a structural block diagram of a device for determining an indicator threshold according to an embodiment of the present disclosure.
  • FIG1 is a hardware structure block diagram of a computer terminal of the method for determining the indicator threshold of the embodiment of the present disclosure.
  • the computer terminal may include one or more (only one is shown in FIG1 ) processors 202 (the processor 202 may include but is not limited to a microprocessor (Microprocessor Unit, referred to as MPU) or a programmable logic device (Programmable logic device, referred to as PLD) and a memory 204 configured to store data.
  • MPU microprocessor Unit
  • PLD programmable logic device
  • the above-mentioned computer terminal may also include a transmission device 206 and an input and output device 208 configured to have a communication function.
  • a transmission device 206 and an input and output device 208 configured to have a communication function.
  • the structure shown in FIG1 is only for illustration and does not limit the structure of the above-mentioned computer terminal.
  • the computer terminal may also include more or fewer components than those shown in FIG1 , or have a different configuration with the same function as that shown in FIG1 or more functions than those shown in FIG1 .
  • the memory 204 may be configured to store computer programs, for example, software programs and modules of application software, such as computer programs corresponding to the method for determining the index threshold value in the embodiment of the present disclosure, and the processor 202 executes various functional applications and data processing by running the computer programs stored in the memory 204, that is, to implement the above method.
  • the memory 204 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.
  • the memory 204 may further include a memory remotely arranged relative to the processor 202, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
  • the transmission device 206 is configured to receive or send data via a network.
  • Specific examples of the above-mentioned network may include a wireless network provided by a communication provider of a computer terminal.
  • the transmission device 206 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet.
  • the transmission device 206 can be a radio frequency (Radio Frequency, referred to as RF) module, which is configured to communicate with the Internet wirelessly.
  • RF Radio Frequency
  • TCP Transmission Control Protocol
  • transmission control protocol transmission control protocol
  • RTT Round-Trip Time, round-trip delay
  • CPU Central Processing Unit
  • central processing unit central processing unit
  • FIG2 is a flow chart of a method for determining an indicator threshold according to an embodiment of the present disclosure. As shown in FIG2 , the steps of the method include:
  • Step S202 obtaining aggregated indicator data corresponding to the target indicator.
  • Step S204 determining an indicator data set from the aggregated indicator data, and performing a The indicator data are sorted to obtain second indicator data, the second indicator data are clustered to obtain multiple groups after clustering, and the indicator data of each group of the multiple groups are fitted to obtain a piecewise function corresponding to each group, wherein the same indicator data set represents the indicator data of the same monitoring object.
  • clustering algorithms for clustering the second indicator data may include Kmeans clustering algorithm, DBSCAN-density-based spatial clustering algorithm, spectral clustering algorithm, GMM-Gaussian mixture model clustering algorithm, MeanShift-mean migration clustering algorithm, hierarchical clustering, etc., but are not limited to these.
  • Step S206 determining an indicator threshold from the intersection coordinate set of the piecewise function according to the indicator bias of the target indicator.
  • the disclosed embodiment obtains aggregated indicator data corresponding to the target indicator; determines an indicator data set from the aggregated indicator data, sorts the first indicator data in the indicator data set to obtain second indicator data, clusters the second indicator data to obtain a plurality of clustered groups, and fits the indicator data of each of the plurality of groups to obtain a piecewise function corresponding to each group, wherein the same indicator data set represents the indicator data of the same monitored object; determines an indicator threshold from the intersection coordinate set of the piecewise function according to the indicator bias of the target indicator, thereby solving the problem of how to determine the indicator threshold.
  • the following implementation steps are proposed: determine the pre-set monitoring dimension, the monitoring object of the monitoring dimension, the indicator category of the target indicator, the initial indicator data under the indicator category, and the time aggregation granularity corresponding to the target indicator; determine the indicator data to be aggregated according to the pre-set monitoring dimension, the monitoring object of the monitoring dimension, the indicator category of the target indicator, and the initial indicator data under the indicator category; aggregate the indicator data to be aggregated according to the time aggregation granularity corresponding to the target indicator to obtain the aggregated indicator data corresponding to the target indicator.
  • a technical solution for aggregating the indicator data to be aggregated according to the time aggregation granularity corresponding to the target indicator to obtain the aggregated indicator data corresponding to the target indicator, which specifically includes: obtaining a first time granularity of the indicator data to be aggregated; when it is determined that the first time granularity is smaller than the time aggregation granularity, obtaining the first indicator data of the indicator data to be aggregated within the first time granularity, and aggregating multiple first time granularities into the time aggregation granularity; aggregating multiple first indicator data within the multiple first time granularities into first aggregated indicator data within the time aggregation granularity, and determining the first aggregated indicator data as the aggregated indicator data corresponding to the target indicator.
  • first indicator data of the indicator data to be aggregated within the first time granularity can be obtained, and the first indicator data can be determined as the aggregated indicator data corresponding to the target indicator.
  • the second indicator data before clustering the second indicator data, can be further standardized to obtain a plurality of standardized indicator values, wherein each standardized indicator value corresponds to a sorting number; for each standardized indicator value, the sorting number corresponding to the standardized indicator value is determined as the horizontal coordinate, and the standardized indicator value is determined as the vertical coordinate to obtain the coordinate point corresponding to the standardized indicator value; the coordinate slopes between the two adjacent coordinate points are determined to obtain a plurality of coordinate slopes, and for each of the plurality of coordinate slopes, a smoothing value of each coordinate slope is determined to obtain a plurality of smoothing values; based on the plurality of smoothing values, The sliding value determines third indicator data, and determines the third indicator data as the updated second indicator data.
  • the above-mentioned standardization processing may include normalization processing.
  • the second indicator data may be standardized by using a normalization processing method to compress the range to within the range of [0, 1], so as to standardize the data and improve the data processing efficiency.
  • the following technical solution is proposed: clustering the multiple coordinate slopes according to a preset clustering algorithm to obtain multiple groups of slope values; for each group of slope values, determining the mean of the coordinate slopes of each group of slope values as the smoothing value of the coordinate slope of each group of slope values.
  • the above-mentioned preset clustering algorithms may include Kmeans clustering algorithm, DBSCAN-density-based spatial clustering algorithm, spectral clustering algorithm, GMM-Gaussian mixture model clustering algorithm, MeanShift-mean shift clustering algorithm, hierarchical clustering, etc., and the present disclosure does not limit this.
  • a technical solution is also proposed, and the specific steps include: when it is determined that there is a target group slope value among the multiple groups of slope values, the smoothed value of the coordinate slope of the adjacent group slope value adjacent to the target group slope value is determined as the smoothed value of the coordinate slope within the target group slope value, or the smoothed value of the coordinate slope within the target group slope value is determined according to a preset smoothing value, wherein the number of coordinate slopes within the target group slope value is different from the number of coordinate slopes within each group of slope values.
  • multiple groups of means can be obtained according to the mean of the indicator data of each group, and a mean set can be determined according to the multiple groups of means, wherein the mean set includes the means corresponding to each piecewise function; the intersection coordinates of the intersection coordinate set of the piecewise function are determined, the left derivative and the right derivative corresponding to the intersection coordinates are determined, and the first mean of the first piecewise function corresponding to the left derivative in the mean set and the second mean of the second piecewise function corresponding to the right derivative in the mean set are determined; based on the first mean and the second mean, it is determined whether to retain the intersection coordinates within the indicator threshold set.
  • the following technical solution is proposed to illustrate the implementation process of determining whether to retain the intersection coordinates within the indicator threshold set based on the first mean and the second mean: determine a first absolute distance value between the intersection coordinates and the origin coordinates; determine first coordinate information corresponding to the first mean based on the first absolute distance value and the first mean, wherein the first coordinate information represents the independent variable value of the first piecewise function; determine second coordinate information corresponding to the second mean based on the first absolute distance value and the second mean, wherein the second coordinate information represents the independent variable value of the second piecewise function; when it is determined that the first coordinate information is the same as the second coordinate information, retain the intersection coordinates within the indicator threshold set; when it is determined that the first coordinate information is different from the second coordinate information, retain the intersection coordinates within the indicator threshold set.
  • a technical solution for implementing the above step S206 of determining the indicator threshold from the intersection coordinate set of the piecewise function according to the indicator bias of the target indicator, specifically including: determining the set of non-differentiable points of the piecewise function and the coordinate points whose second-order derivatives are target values; determining the indicator threshold set based on the set of non-differentiable points, the coordinate points whose second-order derivatives are target values, and the intersection coordinate set of the piecewise function; determining the set of indicator thresholds according to the set of non-differentiable points, the coordinate points whose second-order derivatives are target values, and the intersection coordinate set of the piecewise function; determining the set of The indicator bias of the target indicator determines the indicator threshold from the indicator threshold set.
  • target value may be, for example, 0, but is not limited thereto.
  • the process of determining an indicator threshold from an indicator threshold set according to the indicator bias of the target indicator can be implemented in a variety of ways, specifically including: Way 1, when it is determined that the indicator bias of the target indicator is negative, if the type of the indicator threshold is determined to be an alarm threshold, then the maximum value in the indicator threshold set is determined as the indicator threshold; if the type of the indicator threshold is determined to be a preferential threshold, then the minimum value in the indicator threshold set is determined as the indicator threshold.
  • Method 2 When it is determined that the indicator bias of the target indicator is positive, if the type of the indicator threshold is determined to be an alarm threshold, the minimum value in the indicator threshold set is determined as the indicator threshold; if the type of the indicator threshold is determined to be a preferential threshold, the maximum value in the indicator threshold set is determined as the indicator threshold.
  • the alarm threshold can be understood as a threshold when the performance corresponding to the indicator data of the target indicator is poor.
  • the alarm threshold of the CPU usage is set to 80%, at which time the CPU occupies more resources and the performance is poor.
  • the optimal threshold can be understood as a threshold when the performance corresponding to the indicator data of the target indicator is better.
  • the optimal threshold of the network delay is set to 10%, at which time the network delay is small and the performance is better. In particular, for multiple optimal thresholds, the smallest one is selected as the optimal delay.
  • the monitoring dimensions and indicators are determined, and the key KPI indicator system for operation and maintenance scenarios ⁇ KPI1, KPI2, ..., KPIn ⁇ is constructed; then, according to the actual real-time requirements of business operation and maintenance, the time aggregation granularity is determined, and the time granularity indicator aggregation is performed on each dimension-KPI data to construct a five-tuple data of ⁇ dimension (i.e. monitoring dimension), object (i.e. monitoring object), time granularity (i.e. time aggregation granularity), indicator (i.e. indicator category of target indicator), data (i.e. initial indicator data under indicator category) ⁇ ; among them, the business indicator configuration information needs to clarify the bias of the indicator and the normal range of the indicator.
  • ⁇ dimension i.e. monitoring dimension
  • object i.e. monitoring object
  • time granularity i.e. time aggregation granularity
  • indicator i.e. indicator category of target indicator
  • data i.e.
  • Step a construct a single-dimensional single-object or multi-object KPI indicator data set (the object range selection depends on the actual application scenario), sort the data values of the time series indicators (from small to large or from large to small), and obtain a two-dimensional sequence of ID values and KPI indicator values ⁇ i: Valuei ⁇ , i ⁇ [1, N], N is the number of samples in the data set, and the ID value is the corresponding serial number of the sorted KPI indicator, starting from 1 and increasing by an interval of 1.
  • the two-dimensional sequence can be expressed as a two-dimensional discrete point image with the sample ID and KPI index value as the coordinate axis.
  • the coordinate point is the ID as the horizontal axis and the KPI index value as the vertical axis.
  • Step b Considering that the KPI indicator value may fluctuate greatly, in order to facilitate subsequent processing, the indicator is first standardized.
  • the KPI indicator value is standardized by a normalization processing method, and the range is compressed to the range of [0, 1].
  • clustering algorithms can avoid excessive parameter settings in the process of building the algorithm model, and can classify data with differences in the sequence, making it easier to obtain turning points later.
  • the present disclosure does not specifically limit the type of clustering algorithms.
  • Step d For the G clustering result sets of step c, curve fitting is performed respectively to obtain a piecewise function f(x) having G fitting functions.
  • the curve fitting method can quickly obtain an approximate piecewise function, which provides an effective way to solve the turning point in the subsequent automatic threshold calculation process.
  • Step a Solve the intersection coordinate information of each adjacent piecewise fitting function. For the intersection coordinates, solve the x ⁇ [1, N] interval, the f(x) non-differentiable point set C and the coordinate points where the second-order derivative is 0 to form the threshold set T. For x ⁇ C, calculate the left derivative k1' and the right derivative k2' respectively, and calculate the absolute distance between k1' and k2' and k ⁇ K to determine the category. If both the left and right derivatives belong to the same group, it means that the point is not the turning point we want to find, and then remove the point from the threshold set T.
  • Step b Combine the indicator bias information provided in the business indicator configuration information in step 1 and take the maximum or minimum value in the threshold set T as the threshold solution.
  • the monitoring dimension of this embodiment is the server
  • the objects are server A, server B, and server C
  • the indicators are CPU usage (%), memory usage (%), disk usage (%), and network rate (kbps).
  • the above indicators are constructed into a key KPI indicator system for server equipment operation and maintenance scenarios;
  • the time aggregation granularity is determined to be 1 hour. Taking the server's CPU utilization rate (%) as the target indicator, the indicator is aggregated at the time granularity to construct a five-tuple data of ⁇ dimension, object, time granularity, indicator, data ⁇ ; the business indicator configuration information clearly states that the bias of the CPU utilization rate (%) indicator is negative, and the normal range of the indicator is 0 to 100.
  • Table 1 Index data record table
  • mapping function is solved according to the discrete sample points.
  • server A and server B as objects, select their corresponding indicator data sets, sort the indicator data in the set according to the values, and obtain a two-dimensional sequence ⁇ i: Valuei ⁇ , i ⁇ [1, 48] of the combination of ID values and CPU usage (%) values, and construct a two-dimensional discrete point image with the horizontal axis as the sample ID and the vertical axis as the CPU usage (%).
  • the CPU usage (%) indicator value is normalized.
  • the minimum-maximum scaling method can be used to compress the indicator range to the range of [0, 1].
  • the normalization function is as follows:
  • the two-dimensional sequence discrete points within the above two-dimensional sequence can be understood as coordinate points corresponding to the above standardized index values.
  • the slope between two consecutive points is calculated in segments (equivalent to the above-mentioned coordinate slope), and the slope is smoothed by the three-point mean slope k’.
  • the three-point mean smoothed slope cannot be calculated for the last two points, and the three-point mean smoothed slope of the previous point can be used instead to obtain a new sequence of ID values and three-point mean smoothed slopes ⁇ i:k’i ⁇ , i ⁇ [1,48].
  • the threshold is automatically learned.
  • intersection coordinate set X ⁇ (10, 0.3), (30, 0.5), (40, 0.8) ⁇ of each adjacent piecewise fitting function.
  • the intersection coordinates solve the x ⁇ [1, 48] interval, the set C of non-differentiable points of f(x) and the coordinate points where the second-order derivative is 0.
  • the threshold set T is formed.
  • the sets C and T are also ⁇ (10, 0.3), (30, 0.5), (40, 0.8) ⁇ .
  • the left derivative k1' and the right derivative k2' are calculated respectively.
  • the absolute distances with k ⁇ K are calculated for k1' and k2' respectively.
  • the k selects the mean of the smooth slopes of the groups corresponding to the piecewise functions on the left and right sides of the coordinate point. According to the calculated absolute distance, the belonging category is determined. For the coordinate point (10, 0.3), the left derivative k1' and the right derivative k2' are 0.04 and 0.01 respectively. The mean of the smooth slopes of the groups corresponding to the piecewise functions on the left and right sides of the coordinate point (10, 0.3) are 0.046 and 0.012 respectively. According to the absolute distance calculation, the left derivative k1' and the right derivative k2' belong to different groups, and the point is not removed from the threshold set T. The same goes for other coordinate points, and the final threshold set T is ⁇ (10, 0.3), (30, 0.5), (40, 0.8) ⁇ .
  • the usage scenario is to find the CPU usage rate (%) that needs to issue an alarm, that is, the worse threshold, so the maximum value of the smooth slope of the three-point mean in the threshold set T is selected as 0.8 as the reference value for generating the threshold. Based on this value, the CPU usage rate (%) indicator value before normalization is obtained in reverse, which is 90.72, which is the required threshold solution.
  • the monitoring dimension of this embodiment is the cell
  • the objects are cell 622001, cell 622002, cell 622003, ..., cell 622099
  • the indicators are TCP connection success rate (%), TCP retransmission rate (%), TCP disorder rate (%), TCP average RTT delay (ms);
  • the time aggregation granularity is determined to be 1 hour.
  • the indicator is aggregated at the time granularity to construct the five-tuple data of ⁇ dimension, object, time granularity, indicator, data ⁇ ; the business indicator configuration information clearly states that the bias of the TCP average RTT delay (ms) indicator is negative, and the normal range of the indicator is greater than or equal to 0.
  • mapping function is solved according to the discrete sample points.
  • 50 cells including cell 622001, cell 622002, cell 622003, ..., cell 622050, are determined as objects, and their corresponding indicator data sets are selected.
  • the indicator data in the set are sorted according to the values to obtain a two-dimensional sequence ⁇ i: Valuei ⁇ , i ⁇ [1, 1200] of the combination of ID value and TCP average RTT delay (ms) value.
  • a two-dimensional discrete point image is constructed with the horizontal axis as sample ID and the vertical axis as TCP average RTT delay (ms).
  • the indicator value of TCP average RTT delay (ms) is normalized, and the minimum-maximum scaling method can be used to compress the indicator range to the range of [0, 1].
  • the threshold is automatically learned.
  • the minimum value of the smooth slope of the three-point mean in the threshold set T 0.123, is selected as the reference value for generating the threshold. Based on this value, the TCP average RTT delay (ms) indicator value of 2.5 before normalization is obtained in reverse, which is the required threshold solution.
  • the unilaterality of the indicators is first divided to distinguish between positive and negative indicators. Secondly, the indicator data is preprocessed and the rate of change of the curve is calculated. Then, the machine learning algorithm is used to train the indicator data model. Finally, the threshold learning is transformed into a solution problem based on the turning point of the unilaterality of the indicator, so as to realize the intelligent generation of thresholds for different types of indicators with lower cost and higher accuracy.
  • the problem that the threshold or the converted threshold needs to be manually set can be solved more thoroughly.
  • the technical solution disclosed in the present invention has better applicability and accuracy, and provides a strong guarantee for the operation and maintenance support and operation analysis of mobile operators, which not only helps mobile operators to perform operation and maintenance support and operation analysis more accurately, but also greatly saves labor costs.
  • the present disclosure relates to the field of big data and artificial intelligence technology, and in particular to the field of communication big data and engineering operation and maintenance in the Internet, the Internet of Things, etc., where a large number of indicator thresholds need to be set in a targeted manner, such as the operation and maintenance support and operation analysis of mobile operators, such as anomaly detection, root cause analysis, data prediction, alarm management, intelligent recovery and perception evaluation.
  • the setting method of indicator thresholds on the market is currently widely used, and the indicator threshold is mainly set based on the fixed empirical threshold of the indicator or the dynamic threshold obtained by relatively complex statistical distribution.
  • the present disclosure converts the calculation problem of the indicator threshold into an image solving problem, and combines artificial intelligence algorithms for training and prediction.
  • the artificial intelligence algorithms used also have more choices in practical applications, such as neural network, clustering, classification and other algorithms.
  • the constructed model has good accuracy and broad application prospects, which provides a premise for the precision and intelligence of mobile operator engineering operation and maintenance, and also clarifies the direction for reducing labor costs.
  • This disclosure is aimed at the field of operation and maintenance, especially large and complex architecture systems. It includes IT equipment operation and maintenance based on underlying monitoring indicators and business system operation and maintenance based on model-based KPI/KQI indicators. By collecting and cleaning key system indicators, building monitoring dimension models and automatically learning indicator thresholds, it is possible to identify faults or risks in the system, thereby facilitating network optimization personnel to handle or avoid faults in advance.
  • a device for determining an indicator threshold is also provided, which is used to implement the above embodiments and preferred implementation methods.
  • the term “module” may be a combination of software and/or hardware that implements a predetermined function.
  • the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and contemplated.
  • FIG4 is a structural block diagram of a device for determining an indicator threshold according to an embodiment of the present disclosure. As shown in FIG4 , the device for determining an indicator threshold includes:
  • An acquisition module 42 is configured to acquire aggregated indicator data corresponding to a target indicator
  • a first determination module 44 is configured to determine an indicator data set from the aggregated indicator data, sort the first indicator data in the indicator data set to obtain second indicator data, cluster the second indicator data to obtain a plurality of clustered groups, and fit the indicator data of each of the plurality of groups to obtain a piecewise function corresponding to each group, wherein the same indicator data set represents indicator data of the same monitored object;
  • clustering algorithms for clustering the second indicator data may include Kmeans clustering algorithm, DBSCAN-density-based spatial clustering algorithm, spectral clustering algorithm, GMM-Gaussian mixture model clustering algorithm, MeanShift-mean migration clustering algorithm, hierarchical clustering, etc., but are not limited to these.
  • the second determination module 46 is configured to determine an indicator threshold from the intersection coordinate set of the piecewise function according to the indicator bias of the target indicator.
  • aggregated indicator data corresponding to the target indicator is obtained; an indicator data set is determined from the aggregated indicator data, the first indicator data in the indicator data set is sorted to obtain second indicator data, the second indicator data is clustered to obtain multiple clustered groups, and the indicator data of each group of the multiple groups is fitted to obtain a piecewise function corresponding to each group, wherein the same indicator data set represents the indicator data of the same monitored object; an indicator threshold is determined from the intersection coordinate set of the piecewise function according to the indicator bias of the target indicator, thereby solving the problem of how to determine the indicator threshold.
  • the acquisition module 42 is further configured to: determine a preset monitoring dimension, a monitoring object of the monitoring dimension, an indicator category of the target indicator, the initial indicator data under the indicator category, and a time aggregation granularity corresponding to the target indicator; determine the indicator data to be aggregated according to the preset monitoring dimension, the monitoring object of the monitoring dimension, the indicator category of the target indicator, and the initial indicator data under the indicator category; aggregate the indicator data to be aggregated according to the time aggregation granularity corresponding to the target indicator to obtain the aggregated indicator data corresponding to the target indicator.
  • the acquisition module 42 is further configured to: acquire the first time granularity of the indicator data to be aggregated; when determining that the first time granularity is smaller than the time aggregation granularity, acquire the first indicator data of the indicator data to be aggregated within the first time granularity, and aggregate multiple first time granularities into the time aggregation granularity; aggregate multiple first indicator data within the multiple first time granularities into first aggregate indicator data within the time aggregation granularity, and determine the first aggregate indicator data as the aggregate indicator data corresponding to the target indicator.
  • the above-mentioned acquisition module 42 is also configured to: when it is determined that the first time granularity is equal to the time aggregation granularity, obtain the first indicator data of the indicator data to be aggregated within the first time granularity, and determine the first indicator data as the aggregation indicator data corresponding to the target indicator.
  • the acquisition module 42 is further configured as follows: before clustering the second indicator data, the second indicator data is standardized to obtain a plurality of standardized indicator values, wherein each standardized indicator value corresponds to a sorting number; for each standardized indicator value, the sorting number corresponding to the standardized indicator value is determined as the horizontal coordinate, and the standardized indicator value is determined as the vertical coordinate to obtain the coordinate point corresponding to the standardized indicator value; the coordinate slopes between two adjacent coordinate points are determined to obtain a plurality of coordinate slopes, and for each of the plurality of coordinate slopes, a smoothing value of each coordinate slope is determined to obtain a plurality of smoothing values; the third indicator data is determined according to the plurality of smoothing values, and the third indicator data is determined as the updated second indicator data.
  • the above-mentioned standardization processing may include normalization processing.
  • the second indicator data may be standardized by using a normalization processing method to compress the range to within the range of [0, 1], so as to standardize the data and improve the data processing efficiency.
  • the acquisition module 42 is further configured to: in the process of determining the smoothing value of each coordinate slope in the multiple coordinate slopes to obtain multiple smoothing values, cluster the multiple coordinate slopes according to a preset clustering algorithm to obtain multiple groups of slope values; for each group of slope values, determine the mean of the coordinate slopes of each group of slope values as the smoothing value of the coordinate slope of each group of slope values.
  • the above-mentioned preset clustering algorithms may include Kmeans clustering algorithm, DBSCAN-density-based spatial clustering algorithm, spectral clustering algorithm, GMM-Gaussian mixture model clustering algorithm, MeanShift-mean shift clustering algorithm, hierarchical clustering, etc., and the present disclosure does not limit this.
  • the acquisition module 42 is further configured to: when it is determined that there is a target group slope value among the multiple groups of slope values, determine the smoothed value of the coordinate slope of the adjacent group slope value adjacent to the target group slope value as the smoothed value of the coordinate slope within the target group slope value, or determine the smoothed value of the coordinate slope within the target group slope value according to a preset smoothing value, wherein the number of coordinate slopes within the target group slope value is different from the number of coordinate slopes within each group of slope values.
  • the first determination module 44 is further configured to: obtain multiple groups of means according to the mean of the indicator data of each group, and determine a mean set according to the multiple groups of means, wherein the mean set includes the means corresponding to each piecewise function; determine the intersection coordinates of the intersection coordinate set of the piecewise function, determine the left derivative and the right derivative corresponding to the intersection coordinates, and determine the first mean of the first piecewise function corresponding to the left derivative in the mean set and the second mean of the second piecewise function corresponding to the right derivative in the mean set; determine whether to retain the intersection coordinates within the indicator threshold set based on the first mean and the second mean.
  • the first determination module 44 is further configured to: determine a first absolute distance value between the intersection coordinates and the origin coordinates; determine first coordinate information corresponding to the first mean according to the first absolute distance value and the first mean, wherein the first coordinate information represents the independent variable value of the first piecewise function; determine second coordinate information corresponding to the second mean according to the first absolute distance value and the second mean, wherein the second coordinate information represents the independent variable value of the second piecewise function; when it is determined that the first coordinate information is the same as the second coordinate information, retain the intersection coordinates within the indicator threshold set; when it is determined that the first coordinate information is different from the second coordinate information, retain the intersection coordinates within the indicator threshold set.
  • the second determination module 46 is further configured to: determine the set of non-differentiable points of the piecewise function and the coordinate points where the second-order derivative is the target value; determine the indicator threshold set based on the set of non-differentiable points, the coordinate points where the second-order derivative is the target value, and the coordinate set of the intersection of the piecewise function; and determine the indicator threshold from the indicator threshold set according to the indicator bias of the target indicator.
  • the second determination module 46 is further configured as follows: when it is determined that the indicator bias of the target indicator is negative, if the type of the indicator threshold is determined to be an alarm threshold, the maximum value in the indicator threshold set is determined as the indicator threshold; if the type of the indicator threshold is determined to be a preferential threshold, the minimum value in the indicator threshold set is determined as the indicator threshold.
  • the above-mentioned second determination module 46 is also configured as follows: when it is determined that the indicator bias of the target indicator is positive, if the type of the indicator threshold is determined to be an alarm threshold, the minimum value in the indicator threshold set is determined as the indicator threshold; if the type of the indicator threshold is determined to be a preferential threshold, the maximum value in the indicator threshold set is determined as the indicator threshold.
  • the present disclosure proposes a threshold intelligent learning and operation and maintenance device based on curve image calculation, which can solve the core problem of replacement of threshold self-learning in the industry (i.e., converting one threshold automatic learning process into the threshold setting of another threshold), and truly achieve automatic identification and operation and maintenance of thresholds without human intervention.
  • the indicator data of the monitoring object based on the granularity of the actual application scenario is obtained; the constructed data is displayed in a sequence graphical manner, the distribution data is converted into a curve image, and by solving the turning point of the distribution, combined with the actual indicator business characteristics, the indicator threshold self-learning function is further realized.
  • a first aspect of the present disclosure provides a model building unit based on collected indicators, which is configured to implement functions such as data cleaning, aggregate model description, and core business indicator configuration item description.
  • the second aspect of the present disclosure provides a method for solving a mapping function based on discrete sample points, the method comprising: converting the time series data after model construction into the image representation required for solving the threshold in the present disclosure. It should be noted that the image is not actually drawn here, but the converted data sequence can express the characteristics of the image; through clustering and image fitting algorithms, a mapping function of the image based on the sample sequence is obtained.
  • the third aspect of the present disclosure provides a threshold automatic learning calculation method based on curve image calculation, the method comprising: solving the slope change rate of the function curve obtained in the above steps, combining the business characteristics of the indicator, to obtain the threshold intelligent recognition result.
  • the fourth aspect of the present disclosure provides a threshold automatic learning device based on curve image calculation, the device comprising: a real-time data aggregation module, configured to perform real-time data cleaning and indicator aggregation for key KPI indicators of each entity node of the multi-dimensional system of the operation and maintenance system; a threshold intelligent identification module, configured to execute the method in the above steps.
  • a fifth aspect of the present disclosure provides an electronic device, the electronic device comprising a computer processor and a memory: the computer memory is configured to store a computer program;
  • the processor is configured to implement the functions implemented by the model building unit described in the first aspect above, and to execute a method for solving a mapping function based on discrete sample points described in the second aspect above and a threshold automatic learning calculation method based on curve image calculation described in the third aspect.
  • the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method.
  • the technical solution of the present disclosure, or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium (such as ROM/RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present disclosure.
  • a readable storage medium such as ROM/RAM, a magnetic disk, or an optical disk
  • the above-mentioned computer-readable storage medium may include, but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.
  • An embodiment of the present disclosure further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
  • the processor may be configured to perform the following steps through a computer program:
  • the electronic device may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
  • modules or steps of the present disclosure can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation.
  • the present disclosure is not limited to any specific combination of hardware and software.

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Human Resources & Organizations (AREA)
  • Strategic Management (AREA)
  • Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Theoretical Computer Science (AREA)
  • Development Economics (AREA)
  • Educational Administration (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Tourism & Hospitality (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Marketing (AREA)
  • Game Theory and Decision Science (AREA)
  • General Business, Economics & Management (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

Des modes de réalisation de la présente divulgation concernent un procédé et un appareil de détermination de seuil d'indicateur, un support de stockage et un appareil électronique. Le procédé consiste à : acquérir des données d'indicateur agrégées correspondant à un indicateur cible ; déterminer un ensemble de données d'indicateur à partir des données d'indicateur agrégées, trier les premières données d'indicateur dans l'ensemble de données d'indicateur pour obtenir des secondes données d'indicateur, regrouper les secondes données d'indicateur pour obtenir une pluralité de groupes regroupés, et ajuster les données d'indicateur dans chaque groupe de la pluralité de groupes pour obtenir une fonction par morceaux correspondant à chaque groupe, un même ensemble de données d'indicateur représentant des données d'indicateur d'un même objet de surveillance ; et déterminer un seuil d'indicateur à partir d'un ensemble de coordonnées d'intersection de la fonction par morceaux selon une polarisation d'indicateur de l'indicateur cible. La solution technique résout le problème de la manière de déterminer un seuil d'indicateur dans les technologies associées.
PCT/CN2023/110331 2022-09-30 2023-07-31 Procédé et appareil de détermination de seuil d'indicateur, support de stockage et appareil électronique WO2024066720A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211225305.3A CN117875746A (zh) 2022-09-30 2022-09-30 指标阈值的确定方法、装置、存储介质及电子装置
CN202211225305.3 2022-09-30

Publications (1)

Publication Number Publication Date
WO2024066720A1 true WO2024066720A1 (fr) 2024-04-04

Family

ID=90475973

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/110331 WO2024066720A1 (fr) 2022-09-30 2023-07-31 Procédé et appareil de détermination de seuil d'indicateur, support de stockage et appareil électronique

Country Status (2)

Country Link
CN (1) CN117875746A (fr)
WO (1) WO2024066720A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118051800A (zh) * 2024-04-16 2024-05-17 山东峰华智能科技有限公司 基于云边协同的矿内监测数据智能处理方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150127455A1 (en) * 2013-11-06 2015-05-07 Globys, Inc. Automated entity classification using usage histograms & ensembles
CN110489306A (zh) * 2019-08-26 2019-11-22 北京博睿宏远数据科技股份有限公司 一种报警阈值确定方法、装置、计算机设备及存储介质
CN111985815A (zh) * 2020-08-21 2020-11-24 国网能源研究院有限公司 一种能源电力运营评价指标的筛选方法及装置
CN114780371A (zh) * 2022-05-10 2022-07-22 平安科技（深圳）有限公司 基于多曲线拟合的压测指标分析方法、装置、设备及介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150127455A1 (en) * 2013-11-06 2015-05-07 Globys, Inc. Automated entity classification using usage histograms & ensembles
US20150127454A1 (en) * 2013-11-06 2015-05-07 Globys, Inc. Automated entity classification using usage histograms & ensembles
CN110489306A (zh) * 2019-08-26 2019-11-22 北京博睿宏远数据科技股份有限公司 一种报警阈值确定方法、装置、计算机设备及存储介质
CN111985815A (zh) * 2020-08-21 2020-11-24 国网能源研究院有限公司 一种能源电力运营评价指标的筛选方法及装置
CN114780371A (zh) * 2022-05-10 2022-07-22 平安科技（深圳）有限公司 基于多曲线拟合的压测指标分析方法、装置、设备及介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118051800A (zh) * 2024-04-16 2024-05-17 山东峰华智能科技有限公司 基于云边协同的矿内监测数据智能处理方法

Also Published As

Publication number Publication date
CN117875746A (zh) 2024-04-12

Similar Documents

Publication Publication Date Title
WO2020077672A1 (fr) Procédé et dispositif d'entraînement de modèle d'évaluation de qualité de service
CN114298863B (zh) 一种智能抄表终端的数据采集方法及系统
EP4024261A1 (fr) Procédé, appareil et système d'apprentissage de modèle
WO2021057382A1 (fr) Procédé et appareil de détection d'anomalie, terminal et support de stockage
CN109446783B (zh) 一种基于机器众包的图像识别高效样本采集方法及系统
WO2024066720A1 (fr) Procédé et appareil de détermination de seuil d'indicateur, support de stockage et appareil électronique
CN113408087B (zh) 一种基于云边系统和视频智能分析的变电站巡视方法
CN112365366B (zh) 一种基于智能化5g切片的微电网管理方法及系统
WO2021007871A1 (fr) Système d'optimisation d'opération de production d'alumine et procédé utilisant une collaboration côté nuage
US20210065021A1 (en) Working condition state modeling and model correcting method
CN108243435B (zh) 一种lte小区场景划分中的参数优化方法及装置
CN109494757B (zh) 一种电压无功运行预警方法和系统
CN111176953A (zh) 一种异常检测及其模型训练方法、计算机设备和存储介质
CN110555619A (zh) 一种基于智能配电网的供电能力评估方法
CN118017509B (zh) 基于数字孪生空间的大规模配电网并行优化方法
CN115689320A (zh) 基站运行的健康管理方法、装置及计算机可读存储介质
CN107666403B (zh) 一种指标数据的获取方法及装置
CN111343271A (zh) 一种基于dtu设备的人工智能云控制方法及系统
CN108123436B (zh) 基于主成分分析和多元回归算法的电压越限预测模型
CN110647086A (zh) 一种基于运行大数据分析的智能运维监控系统
CN112860768B (zh) 一种电磁频谱可用频率推荐方法
CN115328870A (zh) 一种面向云制造的数据共享方法及系统
CN106980925B (zh) 一种基于大数据的区域电网调度方法
CN111949246A (zh) 新能源电力行业应用的创建方法及装置
CN116993734B (zh) 基于视觉成像进行分析的电池隔热棉裁切质量预测系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23869966

Country of ref document: EP

Kind code of ref document: A1