CN117176489B - Broadband piecewise linear fitting method, device, electronic equipment and storage medium - Google Patents
Broadband piecewise linear fitting method, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117176489B CN117176489B CN202311452079.7A CN202311452079A CN117176489B CN 117176489 B CN117176489 B CN 117176489B CN 202311452079 A CN202311452079 A CN 202311452079A CN 117176489 B CN117176489 B CN 117176489B
- Authority
- CN
- China
- Prior art keywords
- broadband
- values
- value
- cdn node
- fitting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000005070 sampling Methods 0.000 claims abstract description 66
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 30
- 238000012886 linear function Methods 0.000 claims abstract description 12
- 230000011218 segmentation Effects 0.000 claims description 25
- 238000004590 computer program Methods 0.000 claims description 16
- 238000012417 linear regression Methods 0.000 claims description 8
- 238000010586 diagram Methods 0.000 description 9
- 238000004364 calculation method Methods 0.000 description 6
- 238000012163 sequencing technique Methods 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Landscapes
- Complex Calculations (AREA)
Abstract
The invention relates to a broadband piecewise linear fitting method, a device, electronic equipment and a storage medium, wherein the method comprises the following steps: and acquiring a plurality of CDN nodes and a plurality of corresponding broadband values of each CDN node in the broadband charging period according to the broadband sampling period, wherein each CDN node has a corresponding broadband value at each time point in the broadband charging period. And constructing a small root heap data structure of a first capacity based on a plurality of broadband values of each CDN node in the broadband charging period so as to dynamically acquire the broadband values of each CDN node ordered from big to small, wherein the first capacity is free sampling points in the broadband charging period. And selecting the broadband value of which the rank is not lower than a first threshold value according to the broadband values of which the ranks are ordered from large to small of each CDN node, and calling a piecewise linear fitting algorithm to linearly fit the broadband value of which the rank is not lower than the first threshold value. The method can splice and fit discrete charging points by using a simple linear function, and reduces the complexity of program calculation.
Description
Technical Field
The present invention relates to the field of broadband processing technologies, and in particular, to a broadband piecewise linear fitting method, a device, an electronic apparatus, and a storage medium.
Background
Currently, the main stream charging mode of the bandwidth of the CDN node in the market is 95 charging, namely: the average bandwidth value of each sampling period (usually 5 minutes) under one charging period (usually one natural month) of the node is sampled, then the sampled bandwidth values are ordered, and the first peak value after the highest 5% is removed as the charging value. One feature of this billing method is: each billing cycle has 5% of the time that can run as high as possible and eventually will not be billed, called the free duration.
Under the 95 charging mode, the dynamic historical bandwidth curve of k (the total sampling number of the charging period is 5% +1) before the node is sequenced from large to small, has important significance for charging cost control, and is an important reference basis for planning and adjusting the charging cost of the node. However, these ordering history bandwidths are discrete points, and are not convenient to directly use in some decision scenarios, which may cause problems of large calculation scale and increased program calculation complexity.
Therefore, the conventional broadband calculation method has higher program calculation complexity in a scene with larger calculation scale.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a wideband piecewise linear fitting method, apparatus, electronic device, and storage medium that can reduce the complexity of wideband computation.
The invention provides a broadband piecewise linear fitting method, which comprises the following steps:
acquiring a plurality of CDN nodes and a plurality of corresponding broadband values of each CDN node in a broadband charging period according to a broadband sampling period, wherein each CDN node has a corresponding broadband value at each time point in the broadband charging period;
constructing a small root heap data structure of a first capacity based on a plurality of broadband values of each CDN node in the broadband charging period so as to dynamically acquire the broadband values of each CDN node ordered from big to small, wherein the first capacity is free sampling points in the broadband charging period;
and selecting the broadband value of each CDN node with the ranking not lower than a first threshold value according to the broadband values of the CDN nodes in the sequence from big to small, and calling a piecewise linear fitting algorithm to linearly fit the broadband value of each CDN node with the ranking not lower than the first threshold value.
In one embodiment, the obtaining a plurality of CDN nodes according to a wideband sampling period and a plurality of wideband values corresponding to each CDN node in a wideband charging period includes:
Summarizing historical broadband values of physical broadband of CDN nodes of the large data platform where the CDN nodes are located, and determining the broadband sampling period and the broadband charging period;
acquiring a broadband value corresponding to each time point of each CDN node in the broadband charging period according to the broadband sampling period dynamic increment;
the historical broadband values comprise a plurality of broadband values corresponding to each CDN node.
In one embodiment, the constructing a small root heap data structure of the first capacity based on a plurality of broadband values of each CDN node in the broadband charging period to dynamically obtain broadband values of each CDN node ordered from large to small includes:
based on the CDN nodes, free sampling points in a broadband charging period are obtained to determine the first capacity;
constructing the small root heap data structure of the first capacity, sorting the broadband values of each CDN node in the broadband charging period according to the sequence from big to small of the broadband values through the small root heap data structure, and dynamically sorting the broadband values of the dynamic increment according to the size of the broadband values;
the small root heap data structure adopts a complete binary tree structure, and the broadband value corresponding to each father node is smaller than or equal to the broadband value corresponding to the child node.
In one embodiment, the constructing a small root heap data structure of the first capacity based on a plurality of broadband values of each CDN node in the broadband charging period to dynamically obtain broadband values of each CDN node ordered from large to small, further includes:
judging whether the number of nodes in the small root pile data structure reaches the first capacity or not; if yes, then
Comparing the broadband value of the dynamic increment with the broadband value corresponding to the original node in the small root heap data structure, and replacing the broadband value corresponding to the original node when the broadband value of the dynamic increment is larger than the broadband value corresponding to the original node; and
discarding the broadband value of the dynamic increment when the broadband value of the dynamic increment is smaller than the broadband value corresponding to the original node; if not, then
The dynamically incremented wideband value is inserted directly into the small root heap data structure.
In one embodiment, the selecting, according to the bandwidth values of the sequences from large to small, the bandwidth value of each CDN node with the ranking not lower than the first threshold, and invoking a piecewise linear fitting algorithm to perform linear fitting on the bandwidth value of each CDN node with the ranking not lower than the first threshold, includes:
Calculating the first threshold according to the first capacity, wherein the first threshold is the product of the total CDN node number and the first percentage in the broadband charging period plus a first numerical value;
acquiring fitting parameters, and carrying out pure linear regression on the broadband values with the ranks not lower than a first threshold value based on the fitting parameters so as to calculate the root mean square deviation of the fitting linear function and the broadband values of the original CDN nodes;
wherein the fitting parameters at least comprise fitting precision and maximum segmentation number.
In one embodiment, the selecting, according to the bandwidth values of the sequences from large to small, the bandwidth value of each CDN node with the ranking not lower than the first threshold, and invoking a piecewise linear fitting algorithm to perform linear fitting on the bandwidth value of each CDN node with the ranking not lower than the first threshold, further includes:
dividing the broadband value with the rank not lower than a first threshold value into a plurality of sections according to the maximum number of sections, and calling the piecewise linear fitting algorithm to perform piecewise linear fitting to obtain the root mean square error;
and determining the maximum allowable error of the root mean square error according to the fitting precision, and marking CDN nodes corresponding to the broadband values with the rank not lower than a first threshold value when the root mean square error exceeds the maximum allowable error.
In one embodiment, the method further comprises:
obtaining the minimum segmentation number meeting the fitting precision through a dichotomy, dividing the broadband value of the CDN node meeting the fitting precision in the broadband charging period into a plurality of sections according to the minimum segmentation number, and performing piecewise linear fitting to obtain a fitting result;
and feeding back the minimum segmentation number and the fitting result when the fitting result is equal to the maximum allowable error, and adjusting the minimum segmentation number when the fitting result is greater than or less than the maximum allowable error.
The invention also provides a broadband piecewise linear fitting device, which comprises:
the data sampling module is used for acquiring a plurality of CDN nodes and a plurality of corresponding broadband values of each CDN node in a broadband charging period according to a broadband sampling period, wherein each CDN node has a corresponding broadband value at each time point in the broadband charging period;
the data structuring module is used for constructing a small root heap data structure of a first capacity based on a plurality of broadband values of each CDN node in the broadband charging period so as to dynamically acquire the broadband values of each CDN node in a sequence from large to small, wherein the first capacity is free sampling points in the broadband charging period;
And the linear fitting module is used for selecting the broadband value of each CDN node, the ranking of which is not lower than a first threshold value, according to the broadband value of each CDN node which is sequenced from big to small, and calling a piecewise linear fitting algorithm to perform linear fitting on the broadband value of each CDN node, the ranking of which is not lower than the first threshold value.
The invention also provides an electronic device comprising a memory and a processor, the memory storing a computer program, the processor implementing the wideband piecewise linear fitting method as described in any one of the above when executing the computer program.
The invention also provides a computer storage medium storing a computer program which when executed by a processor implements a wideband piecewise linear fitting method as described in any one of the above.
According to the broadband piecewise linear fitting method, the device, the electronic equipment and the storage medium, the plurality of CDN nodes and the plurality of broadband values corresponding to each CDN node in the broadband charging period are obtained according to the broadband sampling period, and the small root heap data structure of free sampling point capacity in the broadband charging period is constructed based on the plurality of CDN nodes and the broadband values corresponding to each CDN node in the broadband charging period, so that the broadband values of each CDN node in sequence from large to small are dynamically obtained. And then, selecting the broadband value of each CDN node with the ranking not lower than the set threshold according to the broadband values of each CDN node ordered from large to small, and calling a piecewise linear fitting algorithm to linearly fit the broadband value with the ranking not lower than the set threshold of each CDN node, so that the fitting piecewise number of the broadband value of each CDN node and the cutting-off points and slopes of the pieces can be determined. The method can splice and fit discrete charging sampling points by using a plurality of simple linear functions, thereby facilitating the efficient decision processing of the related program under the condition that the related program keeps certain precision on the original data. For example, for a 95 billing node with a billing period of one month (the month is counted by 30 days), and a sampling period of 5 minutes, the threshold is set to 433 (30×24×12×5++1), and when the billing cost plan of the node needs to be adjusted at the end of the month, 432 decision branches will be generated directly according to the ranking history bandwidth curve processing. After the piecewise linear function fitting divided into 3 sections is used, calculation can be performed by only 3 decision branches, so that the complexity of program calculation is greatly reduced.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a wideband piecewise linear fitting method provided by the invention;
FIG. 2 is a schematic diagram of a small root pile data structure of a wideband piecewise linear fitting method in accordance with an embodiment of the present invention;
FIG. 3 is a schematic diagram of a piecewise linear fitting flow of a wideband piecewise linear fitting method in accordance with one embodiment of the present invention;
FIG. 4 is an exemplary graph of a linear fit curve for a wideband piecewise linear fitting method in accordance with one embodiment of the present invention;
FIG. 5 is a second schematic flow chart of the wideband piecewise linear fitting method according to the present invention;
FIG. 6 is a third schematic flow chart of the wideband piecewise linear fitting method provided by the present invention;
FIG. 7 is a schematic diagram of a wideband piecewise linear fitting method provided by the present invention;
FIG. 8 is a fifth flow chart of a wideband piecewise linear fitting method provided by the present invention;
FIG. 9 is a flowchart of a wideband piecewise linear fitting method provided by the present invention;
FIG. 10 is a schematic diagram of a wideband piecewise linear fitting method provided by the present invention;
FIG. 11 is a schematic diagram of a wideband piecewise linear fitting apparatus provided by the present invention;
fig. 12 is an internal structural diagram of a computer device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The wideband piecewise linear fitting method, apparatus, electronic device and storage medium of the present invention are described below in conjunction with fig. 1-12.
As shown in fig. 1, in one embodiment, a wideband piecewise linear fitting method includes the steps of:
Step S110, a plurality of CDN nodes and a plurality of corresponding broadband values of each CDN node in a broadband charging period are obtained according to a broadband sampling period, wherein each CDN node has a corresponding broadband value at each time point in the broadband charging period.
Specifically, the server performs data summarization based on the big data platform, and obtains historical bandwidth sampling values of the CDN nodes according to the bandwidth sampling period, namely, a plurality of CDN nodes and a plurality of broadband values corresponding to each CDN node in the broadband charging period, and broadband data can be obtained in a dynamic increment mode.
Wherein each CDN node has a corresponding broadband value at each point in time in a broadband billing period, one broadband billing period has a plurality of points in time, and each CDN node has a corresponding broadband value at each point in time in the broadband billing period, i.e., each CDN node has a plurality of broadband values in the broadband billing period.
Step S120, based on a plurality of broadband values of each CDN node in the broadband charging period, constructing a small root heap data structure of a first capacity to dynamically acquire the broadband values of each CDN node ordered from big to small, wherein the first capacity is free sampling points in the broadband charging period.
Specifically, with reference to fig. 2, the server constructs a root heap data structure with capacity K for each CDN node. And dynamically acquiring historical bandwidth sampling points (broadband values) of k (total sampling number of charging period multiplied by 5% +1) before sequencing from large to small through an efficient algorithm, namely, important bandwidth histories affecting node charging results. By constructing the small root heap data structure, the historical bandwidth sampling point (broadband value) of k (the total sampling number of charging periods is 5% +1) before ordering from large to small of each CDN node can be obtained more efficiently and dynamically.
The k (total sampling number of charging period is multiplied by 5% +1) before ranking is the first threshold, and the established small root heap data structure can perform efficient dynamic ordering processing operation on the data in dynamic increment without re-ordering the total data every time there is new data. The small root heap data structure adopts a complete binary tree structure, and the father node is smaller than or equal to the value of the child node. According to the characteristics of the small root heap data structure, the root of the small root heap is the smallest value in the small root heap data structure, namely the smallest broadband value of the CDN node. By this feature, every time there is new data, if the number of nodes in the root heap is smaller than K, the new data is directly inserted. If the node number in the small root pile is K, comparing the node number with the existing root of the small root pile, if the new value is larger than the root, replacing and adjusting the pile, otherwise, directly discarding the new value. Therefore, the purpose of dynamically acquiring k historical bandwidth sampling points before sorting from large to small is achieved.
Step S130, selecting broadband values with the rank not lower than a first threshold value according to the broadband values of the rank of each CDN node from big to small, and calling a piecewise linear fitting algorithm to linearly fit the broadband values with the rank not lower than the first threshold value of each CDN node.
Specifically, on the basis of a classical piecewise linear fitting algorithm, a self-adaptive piecewise number method is designed, and the fitting piecewise number meeting the service requirement and information such as the cutting-off point, the slope and the like of each piece are determined by self for the important bandwidth history of k (first threshold) before the dynamic ranking of each CDN node. As shown in fig. 3 and fig. 4, parameters such as fitting accuracy (maxFitErr, maximum allowable error of fitting accuracy), maximum number of segments (maxSeg) and the like are determined according to the service requirements. And then, directly performing pure linear regression (linear regression), and counting the root mean square deviation of the fitted linear function and the original data, and if the fitted square deviation meets the fitting precision requirement, directly returning a fitting result. Then, dividing the CDN node into a maximum segmentation number (MaxSeg) segment, performing piecewise linear fitting by using a piecewise linear fitting algorithm (pwlf. PieceWisLinFit), obtaining a root mean square error of a fitting result, returning the fitting result if the error is higher than a maximum allowable error (maxFitErr) of the fitting precision, and marking the CDN node which does not meet the calculation precision requirement under the limitation of the maximum segmentation number.
According to the broadband piecewise linear fitting method, the plurality of CDN nodes and the plurality of broadband values corresponding to each CDN node in the broadband charging period are obtained according to the broadband sampling period, and the small root heap data structure of free sampling point capacity in the broadband charging period is constructed based on the plurality of CDN nodes and the broadband values corresponding to each CDN node in the broadband charging period, so that the broadband values of each CDN node in order from large to small are dynamically obtained. And then, selecting the broadband value of each CDN node with the ranking not lower than the set threshold according to the broadband values of each CDN node ordered from large to small, and calling a piecewise linear fitting algorithm to linearly fit the broadband value with the ranking not lower than the set threshold of each CDN node, so that the fitting piecewise number of the broadband value of each CDN node and the cutting-off points and slopes of the pieces can be determined. The method can splice and fit discrete charging sampling points by using a plurality of simple linear functions, thereby facilitating the efficient decision processing of the related program under the condition that the related program keeps certain precision on the original data. For example, for a 95 billing node with a billing period of one month (the month is counted by 30 days), and a sampling period of 5 minutes, the threshold is set to 433 (30×24×12×5++1), and when the billing cost plan of the node needs to be adjusted at the end of the month, 432 decision branches will be generated directly according to the ranking history bandwidth curve processing. After the piecewise linear function fitting divided into 3 sections is used, calculation can be performed by only 3 decision branches, so that the complexity of program calculation is greatly reduced.
As shown in fig. 5, in one embodiment, the wideband piecewise linear fitting method provided by the present invention obtains a plurality of CDN nodes and a plurality of wideband values corresponding to each CDN node in a wideband charging period according to a wideband sampling period, and specifically includes the following steps:
step S112, summarizing historical broadband values of physical broadband of CDN nodes of the large data platform where the CDN nodes are located, and determining a broadband sampling period and a broadband charging period.
Specifically, in the process of performing sampling points of the CDN broadband nodes, the server needs to summarize historical broadband values of physical broadband of the CDN nodes of the big data platform where the CDN nodes are located, so as to determine a broadband sampling period and a broadband charging period.
Step S114, obtaining the broadband value corresponding to each time point of each CDN node in the broadband charging period according to the broadband sampling period dynamic increment.
Specifically, the server obtains the broadband value corresponding to each time point in the broadband billing period of each CDN node according to the broadband sampling period dynamic increment determined in step S112.
As shown in fig. 6, in one embodiment, the wideband piecewise linear fitting method provided by the present invention constructs a small root heap data structure of a first capacity based on a plurality of wideband values of each CDN node in a wideband charging period, so as to dynamically obtain wideband values of each CDN node ordered from large to small, and specifically includes the following steps:
Step S122, based on the plurality of CDN nodes, free sampling points in the broadband billing period are obtained to determine the first capacity.
Specifically, the server obtains the number of free sampling points in a preset broadband charging period based on the sampling points obtained by previous sampling, and further determines the first capacity K.
Step S124, constructing a small root heap data structure of the first capacity, sorting the broadband values of each CDN node in the broadband charging period according to the sequence from big to small of the broadband values through the small root heap data structure, and dynamically sorting the broadband values of the dynamic increment according to the size of the broadband values.
Specifically, the server constructs a root heap data structure with the capacity K according to the first capacity K determined in step S122, sorts the broadband values of each CDN node in the broadband charging period according to the sequence from the big broadband value to the small broadband value through the root heap data structure, and dynamically sorts the dynamically increased broadband values (i.e., newly-entered data) according to the size of the broadband values.
The root heap data structure adopts a complete binary tree structure, and the broadband value corresponding to each father node is smaller than or equal to the broadband value corresponding to the child node.
As shown in fig. 7, in one embodiment, the wideband piecewise linear fitting method provided by the present invention constructs a small root heap data structure of a first capacity based on a plurality of wideband values of each CDN node in a wideband charging period, so as to dynamically obtain wideband values of each CDN node ordered from large to small, and specifically further includes the following steps:
step S121, determining whether the number of nodes in the root heap data structure reaches the first capacity.
Specifically, the server determines whether the number of nodes in the root heap data structure reaches a first capacity K.
Step S123, comparing the broadband value of the dynamic increment with the broadband value corresponding to the original node in the small root heap data structure, and replacing the broadband value corresponding to the original node when the broadband value of the dynamic increment is larger than the broadband value corresponding to the original node.
Specifically, when the determination result in step S121 is that the number of nodes in the small root heap reaches the first capacity K, the server compares the broadband value of the dynamic increment with the broadband value corresponding to the original node in the small root heap data structure, and replaces the broadband value corresponding to the original node when the broadband value of the dynamic increment is greater than the broadband value corresponding to the original node.
In step S125, when the broadband value of the dynamic increment is smaller than the broadband value corresponding to the original node, the broadband value of the dynamic increment is discarded.
Specifically, when the broadband value of the dynamic increment is smaller than the broadband value corresponding to the original node in the small root heap, the server discards the broadband value of the dynamic increment.
Step S127, directly inserting the dynamically incremented wideband value into the small root heap data structure.
Specifically, when the determination result in step S121 is that the number of nodes in the root heap is smaller than the first capacity K, the server directly inserts the new broadband value into the root heap.
As shown in fig. 8, in one embodiment, the wideband piecewise linear fitting method provided by the present invention selects wideband values with a ranking not lower than a first threshold value from wideband values ordered from big to small according to each CDN node, and invokes a piecewise linear fitting algorithm to perform linear fitting on the wideband values with a ranking not lower than the first threshold value of each CDN node, and specifically includes the following steps:
step S132, calculating a first threshold according to the first capacity, wherein the first threshold is a product of the total CDN node number and the first percentage in the broadband billing period plus a first value.
Specifically, the server calculates a first threshold according to the first capacity K, where the first threshold is a product of the total CDN node number and a first percentage in the broadband billing period plus a first numerical value, that is, a total sampling number of the billing period×5++1, where 5% is the first percentage and 1 is the first numerical value.
Step S134, obtaining fitting parameters, and carrying out pure linear regression on broadband values of CDN nodes with the ranking not lower than a first threshold value in a broadband charging period based on the fitting parameters so as to count root mean square deviation of the fitting linear function and the broadband values of the original CDN nodes.
Specifically, the server obtains fitting parameters, namely parameters such as fitting precision (maxFitErr), maximum segmentation number (maxSeg), and the like, and performs pure linear regression on broadband values of CDN nodes with the ranking not lower than a first threshold (top k) in a broadband charging period based on the fitting parameters so as to count root mean square deviation of a fitting linear function and broadband values of original CDN nodes.
Wherein the fitting parameters include, but are not limited to, fitting accuracy and maximum number of segments.
As shown in fig. 9, in one embodiment, the wideband piecewise linear fitting method provided by the present invention selects wideband values with a ranking not lower than a first threshold value according to wideband values ordered from big to small for each CDN node, and invokes a piecewise linear fitting algorithm to perform linear fitting on the wideband values with a ranking not lower than the first threshold value for each CDN node, and specifically further includes the following steps:
step S136, dividing the broadband value of the CDN node with the ranking not lower than the first threshold value in the broadband charging period into a plurality of sections according to the maximum number of sections, and calling a piecewise linear fitting algorithm to perform piecewise linear fitting to obtain root mean square error.
Specifically, the server divides the broadband value of each CDN node, the ranking of which is not lower than a first threshold (top k) in the broadband charging period, into a plurality of sections according to the maximum segmentation number, and calls a piecewise linear fitting algorithm to perform piecewise linear fitting on the plurality of sections of broadband values to obtain root mean square errors.
And S138, determining the maximum allowable error of the root mean square error according to the fitting precision, and marking CDN nodes corresponding to broadband values with the ranking not lower than a first threshold value when the root mean square error exceeds the maximum allowable error.
Specifically, the server determines a maximum allowable error (maxFitErr) of the root mean square error according to the previously determined fitting precision, and determines that the corresponding CDN node (i.e., the CDN node corresponding to the broadband value ranked not lower than the first threshold) does not meet the fitting precision when the root mean square error exceeds the maximum allowable error, and then marks the CDN node that does not meet the fitting precision.
As shown in fig. 10, in one embodiment, the wideband piecewise linear fitting method provided by the present invention further includes the following steps:
step S1010, obtaining the minimum segmentation number meeting the fitting precision through a dichotomy, dividing the broadband value of the CDN node meeting the fitting precision in the broadband charging period into a plurality of sections according to the minimum segmentation number, and performing piecewise linear fitting to obtain a fitting result.
Specifically, the server determines the minimum segmentation number meeting the fitting precision through a dichotomy, divides the broadband value of the CDN node meeting the fitting precision in the broadband charging period into a plurality of sections according to the minimum segmentation number, and performs piecewise linear fitting again to obtain a final fitting result. As shown in fig. 3, first, l=1, r=maxseg is set, and second, m= (l+r)/2 is set, piecewise linear fitting is performed in m segments, and a fitting result and an error are obtained. If the fitting result is equal to the fitting precision (maxFitErr), the segmentation number m and the corresponding fitting result are directly returned. If the fitting result is greater than the fitting accuracy (maxFitErr), then l is increased by 1, and if the fitting result is less than the fitting accuracy (maxFitErr), then r is decreased by 1. The above steps may be repeated until l=r and the number of segments is returned as l and the corresponding fitting result.
In step S1020, when the fitting result is equal to the maximum allowable error, the minimum number of segments and the fitting result are fed back, and when the fitting result is greater than or less than the maximum allowable error, the minimum number of segments is adjusted.
Specifically, the server feeds back the minimum number of segments and the fitting result when the fitting result is equal to the maximum allowable error (maxFitErr), and adjusts the minimum number of segments when the fitting result is greater than or less than the maximum allowable error (maxFitErr). The method is based on a classical piecewise linear fitting algorithm, a self-adaptive piecewise number method is designed, and the fitting piecewise number meeting the service requirement and information such as the cutting-off points, the slopes and the like of each piece are determined by a dichotomy for the dynamic front k important bandwidth history of each CDN node.
The wideband piecewise linear fitting device provided by the invention is described below, and the wideband piecewise linear fitting device described below and the wideband piecewise linear fitting method described above can be referred to correspondingly with each other.
As shown in FIG. 11, in one embodiment, a wideband piecewise linear fitting apparatus includes a data sampling module 1110, a data structuring module 1120, and a linear fitting module 1130.
The data sampling module 1110 is configured to obtain a plurality of CDN nodes and a plurality of broadband values corresponding to each CDN node in a broadband billing period according to a broadband sampling period, where each CDN node has a corresponding broadband value at each time point in the broadband billing period.
The data structuring module 1120 is configured to construct a small root heap data structure of a first capacity based on a plurality of broadband values of each CDN node in the broadband charging period, so as to dynamically obtain the broadband values of each CDN node ordered from large to small, where the first capacity is free sampling points in the broadband charging period.
The linear fitting module 1130 is configured to select, according to the bandwidth values of each CDN node ordered from large to small, a bandwidth value with a rank not lower than a first threshold, and invoke a piecewise linear fitting algorithm to perform linear fitting on the bandwidth value with a rank not lower than the first threshold for each CDN node.
In this embodiment, the wideband piecewise linear fitting device provided by the present invention, the data sampling module is specifically configured to:
and summarizing historical broadband values of physical broadband of the CDN node of the large data platform where the CDN node is located, and determining a broadband sampling period and a broadband charging period.
And dynamically and incrementally acquiring a broadband value corresponding to each time point of each CDN node in the broadband charging period according to the broadband sampling period.
The historical broadband values comprise broadband values corresponding to each CDN node.
In this embodiment, the wideband piecewise linear fitting device provided by the present invention, the data structuring module is specifically configured to:
based on a plurality of CDN nodes, free sampling points in a broadband charging period are obtained to determine a first capacity.
Constructing a small root heap data structure of the first capacity, sequencing the broadband values of each CDN node in a broadband charging period according to the sequence from big to small of the broadband values through the small root heap data structure, and dynamically sequencing the broadband values of the dynamic increment according to the sizes of the broadband values.
The root heap data structure adopts a complete binary tree structure, and the broadband value corresponding to each father node is smaller than or equal to the broadband value corresponding to the child node.
In this embodiment, the wideband piecewise linear fitting device provided by the present invention, the data structuring module is specifically further configured to:
and judging whether the number of nodes in the small root heap data structure reaches a first capacity. If yes, then
And comparing the broadband value of the dynamic increment with the broadband value corresponding to the original node in the small root heap data structure, and replacing the broadband value corresponding to the original node when the broadband value of the dynamic increment is larger than the broadband value corresponding to the original node. And
And discarding the broadband value of the dynamic increment when the broadband value of the dynamic increment is smaller than the broadband value corresponding to the original node. If not, then
The dynamically incremented wideband value is inserted directly into the small root heap data structure.
In this embodiment, the wideband piecewise linear fitting device provided by the present invention, the linear fitting module is specifically configured to:
and calculating a first threshold according to the first capacity, wherein the first threshold is the product of the total CDN node number and the first percentage in the broadband charging period plus a first numerical value.
And acquiring fitting parameters, and carrying out pure linear regression on the broadband values of CDN nodes with the ranking not lower than a first threshold value in the broadband charging period based on the fitting parameters so as to count the root mean square deviation of the fitting linear function and the broadband values of the original CDN nodes.
Wherein the fitting parameters at least comprise fitting precision and maximum segmentation number.
In this embodiment, the wideband piecewise linear fitting device provided by the present invention, the linear fitting module is specifically further configured to:
dividing the broadband value with the ranking not lower than the first threshold value into a plurality of sections according to the maximum number of sections, and calling a piecewise linear fitting algorithm to perform piecewise linear fitting to obtain root mean square error.
And determining the maximum allowable error of the root mean square error according to the fitting precision, and marking CDN nodes corresponding to the broadband values with the ranking not lower than the first threshold value when the root mean square error exceeds the maximum allowable error.
In this embodiment, the wideband piecewise linear fitting device provided by the present invention further includes a linear fitting sub-module, configured to:
obtaining the minimum segmentation number meeting the fitting precision through a dichotomy, dividing the broadband value of the CDN node meeting the fitting precision in the broadband charging period into a plurality of sections according to the minimum segmentation number, and performing piecewise linear fitting to obtain a fitting result.
And feeding back the minimum segmentation number and the fitting result when the fitting result is equal to the maximum allowable error, and adjusting the minimum segmentation number when the fitting result is greater than or less than the maximum allowable error.
Fig. 12 illustrates a physical structure diagram of an electronic device, which may be an intelligent terminal, and an internal structure diagram thereof may be as shown in fig. 12. The electronic device includes a processor, an internal memory, and a network interface connected by a system bus. Wherein the processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic device includes a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the electronic device is used for communicating with an external terminal through a network connection. The computer program, when executed by a processor, implements a wideband piecewise linear fitting method comprising:
acquiring a plurality of CDN nodes and a plurality of corresponding broadband values of each CDN node in a broadband charging period according to a broadband sampling period, wherein each CDN node has a corresponding broadband value at each time point in the broadband charging period;
constructing a small root heap data structure of a first capacity based on a plurality of broadband values of each CDN node in a broadband charging period so as to dynamically acquire the broadband values of each CDN node ordered from big to small, wherein the first capacity is free sampling points in the broadband charging period;
And selecting the broadband value of which the rank is not lower than a first threshold value according to the broadband values of which the ranks are ordered from large to small of each CDN node, and calling a piecewise linear fitting algorithm to linearly fit the broadband value of which the rank is not lower than the first threshold value.
It will be appreciated by those skilled in the art that the structure shown in fig. 12 is merely a block diagram of a portion of the structure associated with the present inventive arrangements and is not limiting of the electronic device to which the present inventive arrangements are applied, and that a particular electronic device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In another aspect, the present invention also provides a computer storage medium storing a computer program which when executed by a processor implements a wideband piecewise linear fitting method, the method comprising:
acquiring a plurality of CDN nodes and a plurality of corresponding broadband values of each CDN node in a broadband charging period according to a broadband sampling period, wherein each CDN node has a corresponding broadband value at each time point in the broadband charging period;
constructing a small root heap data structure of a first capacity based on a plurality of broadband values of each CDN node in a broadband charging period so as to dynamically acquire the broadband values of each CDN node ordered from big to small, wherein the first capacity is free sampling points in the broadband charging period;
And selecting the broadband value of which the rank is not lower than a first threshold value according to the broadband values of which the ranks are ordered from large to small of each CDN node, and calling a piecewise linear fitting algorithm to linearly fit the broadband value of which the rank is not lower than the first threshold value.
In yet another aspect, a computer program product or computer program is provided, the computer program product or computer program comprising computer instructions stored in a computer readable storage medium. A processor of an electronic device reads the computer instructions from a computer readable storage medium, the processor executing the computer instructions to implement a wideband piecewise linear fitting method, the method comprising:
acquiring a plurality of CDN nodes and a plurality of corresponding broadband values of each CDN node in a broadband charging period according to a broadband sampling period, wherein each CDN node has a corresponding broadband value at each time point in the broadband charging period;
constructing a small root heap data structure of a first capacity based on a plurality of broadband values of each CDN node in a broadband charging period so as to dynamically acquire the broadband values of each CDN node ordered from big to small, wherein the first capacity is free sampling points in the broadband charging period;
And selecting the broadband value of which the rank is not lower than a first threshold value according to the broadband values of which the ranks are ordered from large to small of each CDN node, and calling a piecewise linear fitting algorithm to linearly fit the broadband value of which the rank is not lower than the first threshold value.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory.
By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples illustrate only a few embodiments of the invention and are described in detail herein without thereby limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.
Claims (7)
1. A wideband piecewise linear fitting method, the method comprising:
acquiring a plurality of CDN nodes and a plurality of corresponding broadband values of each CDN node in a broadband charging period according to a broadband sampling period, wherein each CDN node has a corresponding broadband value at each time point in the broadband charging period;
constructing a small root heap data structure of a first capacity based on a plurality of broadband values of each CDN node in the broadband charging period so as to dynamically acquire the broadband values of each CDN node ordered from big to small, wherein the first capacity is free sampling points in the broadband charging period;
Selecting a broadband value of which the ranking of each CDN node is not lower than a first threshold value according to the broadband values of which the ranking of each CDN node is not lower than the first threshold value from big to small, and calling a piecewise linear fitting algorithm to perform linear fitting on the broadband value of which the ranking of each CDN node is not lower than the first threshold value;
the constructing a small root heap data structure of a first capacity based on a plurality of broadband values of each CDN node in the broadband charging period to dynamically obtain broadband values of each CDN node ordered from large to small includes:
based on the CDN nodes, free sampling points in a broadband charging period are obtained to determine the first capacity;
constructing the small root heap data structure of the first capacity, sorting the broadband values of each CDN node in the broadband charging period according to the sequence from big to small of the broadband values through the small root heap data structure, and dynamically sorting the broadband values of the dynamic increment according to the size of the broadband values;
the root heap data structure adopts a complete binary tree structure, and the broadband value corresponding to each father node is smaller than or equal to the broadband value corresponding to the child node;
selecting the broadband value of each CDN node with the ranking not lower than the first threshold value from the broadband values of the sequences from big to small according to each CDN node, and calling a piecewise linear fitting algorithm to linearly fit the broadband value of each CDN node with the ranking not lower than the first threshold value, wherein the method comprises the following steps:
Calculating the first threshold according to the first capacity, wherein the first threshold is the product of the total CDN node number and the first percentage in the broadband charging period plus a first numerical value;
acquiring fitting parameters, and carrying out pure linear regression on the broadband values with the ranks not lower than a first threshold value based on the fitting parameters so as to calculate the root mean square deviation of the fitting linear function and the broadband values of the original CDN nodes;
wherein the fitting parameters at least comprise fitting precision and maximum segmentation number;
selecting a broadband value of which the rank is not lower than a first threshold value according to the broadband values of which the ranks are not lower than the first threshold value, and calling a piecewise linear fitting algorithm to linearly fit the broadband value of which the rank is not lower than the first threshold value, wherein the method further comprises the steps of:
dividing the broadband value with the rank not lower than a first threshold value into a plurality of sections according to the maximum number of sections, and calling the piecewise linear fitting algorithm to perform piecewise linear fitting to obtain root mean square error;
and determining the maximum allowable error of the root mean square error according to the fitting precision, and marking CDN nodes corresponding to the broadband values with the rank not lower than a first threshold value when the root mean square error exceeds the maximum allowable error.
2. The wideband piecewise linear fitting method of claim 1, wherein the obtaining a plurality of CDN nodes according to a wideband sampling period and a plurality of wideband values corresponding to each CDN node in a wideband charging period comprises:
summarizing historical broadband values of physical broadband of CDN nodes of the large data platform where the CDN nodes are located, and determining the broadband sampling period and the broadband charging period;
acquiring a broadband value corresponding to each time point of each CDN node in the broadband charging period according to the broadband sampling period dynamic increment;
the historical broadband values comprise a plurality of broadband values corresponding to each CDN node.
3. The broadband piecewise linear fitting method of claim 1, wherein the constructing a first capacity small root heap data structure based on a plurality of broadband values of each CDN node in the broadband billing period to dynamically obtain the broadband values of each CDN node ordered from large to small further comprises:
judging whether the number of nodes in the small root pile data structure reaches the first capacity or not; if yes, then
Comparing the broadband value of the dynamic increment with the broadband value corresponding to the original node in the small root heap data structure, and replacing the broadband value corresponding to the original node when the broadband value of the dynamic increment is larger than the broadband value corresponding to the original node; and
Discarding the broadband value of the dynamic increment when the broadband value of the dynamic increment is smaller than the broadband value corresponding to the original node; if not, then
The dynamically incremented wideband value is inserted directly into the small root heap data structure.
4. The wideband piecewise linear fitting method of claim 1, wherein the method further comprises:
obtaining the minimum segmentation number meeting the fitting precision through a dichotomy, dividing the broadband value of the CDN node meeting the fitting precision in the broadband charging period into a plurality of sections according to the minimum segmentation number, and performing piecewise linear fitting to obtain a fitting result;
and feeding back the minimum segmentation number and the fitting result when the fitting result is equal to the maximum allowable error, and adjusting the minimum segmentation number when the fitting result is greater than or less than the maximum allowable error.
5. A wideband piecewise linear fitting apparatus, the apparatus comprising:
the data sampling module is used for acquiring a plurality of CDN nodes and a plurality of corresponding broadband values of each CDN node in a broadband charging period according to a broadband sampling period, wherein each CDN node has a corresponding broadband value at each time point in the broadband charging period;
The data structuring module is used for constructing a small root heap data structure of a first capacity based on a plurality of broadband values of each CDN node in the broadband charging period so as to dynamically acquire the broadband values of each CDN node in a sequence from large to small, wherein the first capacity is free sampling points in the broadband charging period;
the linear fitting module is used for selecting the broadband value of each CDN node, the ranking of which is not lower than a first threshold value, according to the broadband value of each CDN node which is sequenced from big to small, and calling a piecewise linear fitting algorithm to perform linear fitting on the broadband value of each CDN node, the ranking of which is not lower than the first threshold value;
the constructing a small root heap data structure of a first capacity based on a plurality of broadband values of each CDN node in the broadband charging period to dynamically obtain broadband values of each CDN node ordered from large to small includes:
based on the CDN nodes, free sampling points in a broadband charging period are obtained to determine the first capacity;
constructing the small root heap data structure of the first capacity, sorting the broadband values of each CDN node in the broadband charging period according to the sequence from big to small of the broadband values through the small root heap data structure, and dynamically sorting the broadband values of the dynamic increment according to the size of the broadband values;
The root heap data structure adopts a complete binary tree structure, and the broadband value corresponding to each father node is smaller than or equal to the broadband value corresponding to the child node;
selecting the broadband value of each CDN node with the ranking not lower than the first threshold value from the broadband values of the sequences from big to small according to each CDN node, and calling a piecewise linear fitting algorithm to linearly fit the broadband value of each CDN node with the ranking not lower than the first threshold value, wherein the method comprises the following steps:
calculating the first threshold according to the first capacity, wherein the first threshold is the product of the total CDN node number and the first percentage in the broadband charging period plus a first numerical value;
acquiring fitting parameters, and carrying out pure linear regression on the broadband values with the ranks not lower than a first threshold value based on the fitting parameters so as to calculate the root mean square deviation of the fitting linear function and the broadband values of the original CDN nodes;
wherein the fitting parameters at least comprise fitting precision and maximum segmentation number;
selecting a broadband value of which the rank is not lower than a first threshold value according to the broadband values of which the ranks are not lower than the first threshold value, and calling a piecewise linear fitting algorithm to linearly fit the broadband value of which the rank is not lower than the first threshold value, wherein the method further comprises the steps of:
Dividing the broadband value with the rank not lower than a first threshold value into a plurality of sections according to the maximum number of sections, and calling the piecewise linear fitting algorithm to perform piecewise linear fitting to obtain root mean square error;
and determining the maximum allowable error of the root mean square error according to the fitting precision, and marking CDN nodes corresponding to the broadband values with the rank not lower than a first threshold value when the root mean square error exceeds the maximum allowable error.
6. An electronic device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 4 when the computer program is executed.
7. A computer storage medium storing a computer program, characterized in that the computer program when executed by a processor implements the steps of the method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311452079.7A CN117176489B (en) | 2023-11-03 | 2023-11-03 | Broadband piecewise linear fitting method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311452079.7A CN117176489B (en) | 2023-11-03 | 2023-11-03 | Broadband piecewise linear fitting method, device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117176489A CN117176489A (en) | 2023-12-05 |
| CN117176489B true CN117176489B (en) | 2024-02-27 |
Family
ID=88932144
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311452079.7A Active CN117176489B (en) | 2023-11-03 | 2023-11-03 | Broadband piecewise linear fitting method, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117176489B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112671612A (en) * | 2020-12-24 | 2021-04-16 | 深圳市高德信通信股份有限公司 | Network bandwidth detection method based on CDN network technology |
| CN113949741A (en) * | 2021-10-14 | 2022-01-18 | 北京奇艺世纪科技有限公司 | Scheduling method, scheduling device, electronic equipment and storage medium |
| CN115396244A (en) * | 2021-05-08 | 2022-11-25 | 北京金山云网络技术有限公司 | Bandwidth scheduling management method and device, electronic equipment and storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080259867A1 (en) * | 2007-04-22 | 2008-10-23 | Chuanxiong Guo | Method and system for scheduling packets from different flows to provide fair bandwidth sharing |
| US20100094962A1 (en) * | 2008-10-15 | 2010-04-15 | Patentvc Ltd. | Internet backbone servers with edge compensation |
| US9467387B2 (en) * | 2015-02-10 | 2016-10-11 | Ericsson Ab | System and method for managing bandwidth responsive to the duty cycle of an ABR client |
-
2023
- 2023-11-03 CN CN202311452079.7A patent/CN117176489B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112671612A (en) * | 2020-12-24 | 2021-04-16 | 深圳市高德信通信股份有限公司 | Network bandwidth detection method based on CDN network technology |
| CN115396244A (en) * | 2021-05-08 | 2022-11-25 | 北京金山云网络技术有限公司 | Bandwidth scheduling management method and device, electronic equipment and storage medium |
| CN113949741A (en) * | 2021-10-14 | 2022-01-18 | 北京奇艺世纪科技有限公司 | Scheduling method, scheduling device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117176489A (en) | 2023-12-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110880014B (en) | Data processing method, device, computer equipment and storage medium | |
| CN112287968A (en) | Image model training method, image processing method, chip, device and medium | |
| CN112306027A (en) | Equipment production line scheduling method and system applied to intelligent park | |
| CN111080477A (en) | Household power load prediction method and system | |
| CN111177217A (en) | Data preprocessing method and device, computer equipment and storage medium | |
| CN111209336B (en) | Data distribution method and device based on block chain and server | |
| CN111259137A (en) | Method and system for generating knowledge graph abstract | |
| CN111881477A (en) | Indexing method and device of data content, computer equipment and storage medium | |
| CN117176489B (en) | Broadband piecewise linear fitting method, device, electronic equipment and storage medium | |
| US11881873B1 (en) | Huffman correction encoding method and system, and relevant components | |
| CN110929994A (en) | Enterprise associated risk analysis method, device, equipment and storage medium | |
| CN112148942B (en) | Business index data classification method and device based on data clustering | |
| CN115080156B (en) | Flow-batch-integration-based optimized calculation method and device for big data batch calculation | |
| US11797480B2 (en) | Storage of order books with persistent data structures | |
| CN115718691A (en) | Fault root cause positioning method, device and equipment of micro-service system | |
| CN115693684A (en) | Power grid transient voltage stabilizing method, device, equipment and storage medium | |
| CN109284260B (en) | Big data file reading method and device, computer equipment and storage medium | |
| CN111737263A (en) | Method and device for screening information and computer equipment | |
| CN115017215B (en) | Data processing method and computing device | |
| CN117725540B (en) | Method, system, equipment and medium for calculating dynamic baseband | |
| WO2024001867A1 (en) | Communication load prediction method, and computer-readable storage medium | |
| CN111124491B (en) | Batch processing method, device, equipment and storage medium | |
| CN111340683B (en) | Image data processing method, image data processing device, image processing system and server | |
| CN117632927A (en) | Data processing method, device, equipment and storage medium | |
| CN115904725A (en) | Fusing parameter generation method, device, terminal and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |