CN113778972B

CN113778972B - Method for estimating log increment in Content Delivery Network (CDN) system

Info

Publication number: CN113778972B
Application number: CN202111081511.7A
Authority: CN
Inventors: 刘静; 耿芳歌
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2021-09-15
Filing date: 2021-09-15
Publication date: 2022-05-17
Anticipated expiration: 2041-09-15
Also published as: CN113778972A

Abstract

The invention provides a method for pre-estimating log increment in a Content Delivery Network (CDN) system, relates to the technical field of cloud platforms and artificial intelligence, and particularly relates to log data processing. The implementation scheme is as follows: determining a node group request number increment corresponding to at least one node group according to a request number increment of the CDN system and a first request number sharing proportion between the at least one node group; determining the corresponding node request number increment of each node in at least one node group according to the node group request number increment and the corresponding second request number sharing proportion between the nodes in at least one node group; and determining at least one log increment corresponding to at least one log of each node in at least one node group according to the node request number increment and the proportional coefficient relationship between the node request number increment and the log increment.

Description

Method for estimating log increment in Content Delivery Network (CDN) system

Technical Field

The present disclosure relates to the field of cloud platforms and artificial intelligence technologies, and in particular, to a method and an apparatus for estimating a log increment in a content delivery network CDN system, an electronic device, a computer-readable storage medium, and a computer program product.

Background

Content Delivery Networks (CDNs) can direct a user's access request to the appropriate node server by placing the node servers throughout the network. Meanwhile, each node server is provided with a log system, the log system comprises a plurality of logs, each log corresponds to one type of application or service, and information of the type of application or service is recorded.

When the amount of access requests of the user increases, the log amount of each log also increases. When the log quantity is predicted to exceed the range which can be carried by the log system after being increased, corresponding measures need to be taken in advance, so that the log system can carry the increase of the log quantity caused by the increase of the user request quantity. Therefore, it is very important to correctly estimate the log increment according to the request number increment of the user to evaluate the bearing capacity of the log system.

The approaches described in this section are not necessarily approaches that have been previously conceived or pursued. Unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section. Similarly, unless otherwise indicated, the problems mentioned in this section should not be considered as having been acknowledged in any prior art.

Disclosure of Invention

The present disclosure provides a method, an apparatus, an electronic device, a computer-readable storage medium, and a computer program product for predicting a log increment in a Content Delivery Network (CDN) system.

According to an aspect of the present disclosure, a method for predicting log increment in a Content Delivery Network (CDN) system is provided, where the CDN system includes at least one node group, each node group includes at least one node, and each node includes at least one log, the method including: determining a node group request number increment corresponding to at least one node group according to a request number increment of the CDN system and a first request number sharing proportion between the at least one node group; determining the corresponding node request number increment of each node in at least one node group according to the node group request number increment and the corresponding second request number sharing proportion between the nodes in at least one node group; and determining at least one log increment corresponding to at least one log of each node in at least one node group according to the node request number increment and the proportional coefficient relationship between the node request number increment and the log increment.

According to another aspect of the present disclosure, an apparatus for predicting log increment in a content delivery network CDN system is provided, where the CDN system includes at least one node group, each node group includes at least one node, each node includes at least one log, and the apparatus includes: the first determining module is configured to determine a node group request number increment corresponding to at least one node group according to a request number increment of the CDN system and a first request number sharing ratio between the at least one node group; a second determining module configured to determine a corresponding node request number increment for each node in the at least one node group according to the node group request number increment and a corresponding second request number apportionment ratio between nodes in the at least one node group; and the third determining module is configured to determine the corresponding log increment of at least one log of each node in at least one node group according to the node request number increment and the proportional coefficient relation between the node request number increment and the log increment.

According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the above-described method.

According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the above method.

According to another aspect of the disclosure, a computer program product is provided, comprising a computer program, wherein the computer program realizes the above method when executed by a processor.

According to one or more embodiments of the disclosure, the log increment can be correctly estimated according to the request number increment of the user, so as to evaluate the bearing capacity of the log system, and provide a basis for subsequently improving the log system and guaranteeing the service quality.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the embodiments and, together with the description, serve to explain the exemplary implementations of the embodiments. The illustrated embodiments are for purposes of illustration only and do not limit the scope of the claims. Throughout the drawings, identical reference numbers designate similar, but not necessarily identical, elements.

FIG. 1 illustrates a schematic diagram of an exemplary system in which various methods described herein may be implemented, according to an embodiment of the present disclosure;

FIG. 2 shows a flow diagram of a method of pre-estimating log deltas in a Content Delivery Network (CDN) system according to an embodiment of the present disclosure;

FIG. 3 illustrates a block diagram of an exemplary system that can be used to implement embodiments of the present disclosure;

FIG. 4 shows a block diagram of an apparatus for predicting log deltas in a CDN system of a content delivery network according to an embodiment of the present disclosure;

FIG. 5 illustrates a block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

In the present disclosure, unless otherwise specified, the use of the terms "first", "second", etc. to describe various elements is not intended to limit the positional relationship, the timing relationship, or the importance relationship of the elements, and such terms are used only to distinguish one element from another. In some examples, a first element and a second element may refer to the same instance of the element, and in some cases, based on the context, they may also refer to different instances.

The terminology used in the description of the various described examples in this disclosure is for the purpose of describing particular examples only and is not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the elements may be one or more. Furthermore, the term "and/or" as used in this disclosure is intended to encompass any and all possible combinations of the listed items.

Embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

Fig. 1 illustrates a schematic diagram of an exemplary system 100 in which various methods and apparatus described herein may be implemented in accordance with embodiments of the present disclosure. Referring to fig. 1, the system 100 includes one or

more client devices

101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120.

Client devices

101, 102, 103, 104, 105, and 106 may be configured to execute one or more applications.

In embodiments of the present disclosure, server 120 may run one or more services or software applications that enable the performance of the method of pre-estimating log increments in a content delivery network CDN system.

In some embodiments, the server 120 may also provide other services or software applications that may include non-virtual environments and virtual environments. In certain embodiments, these services may be provided as web-based services or cloud services, for example, provided to users of

client devices

101, 102, 103, 104, 105, and/or 106 under a software as a service (SaaS) model.

In the configuration shown in fig. 1, server 120 may include one or more components that implement the functions performed by server 120. These components may include software components, hardware components, or a combination thereof, which may be executed by one or more processors. A user operating a

client device

101, 102, 103, 104, 105, and/or 106 may, in turn, utilize one or more client applications to interact with the server 120 to take advantage of the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from system 100. Accordingly, fig. 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

The user may send the request using

client devices

101, 102, 103, 104, 105, and/or 106. The client device may provide an interface that enables a user of the client device to interact with the client device. The client device may also output information to the user via the interface. Although fig. 1 depicts only six client devices, those skilled in the art will appreciate that any number of client devices may be supported by the present disclosure.

Client devices

101, 102, 103, 104, 105, and/or 106 may include various types of computer devices, such as portable handheld devices, general purpose computers (such as personal computers and laptops), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, gaming systems, thin clients, various messaging devices, sensors or other sensing devices, and so forth. These computer devices may run various types and versions of software applications and operating systems, such as MICROSOFT Windows, APPLE iOS, UNIX-like operating systems, Linux, or Linux-like operating systems (e.g., GOOGLE Chrome OS); or include various Mobile operating systems such as MICROSOFT Windows Mobile OS, iOS, Windows Phone, Android. Portable handheld devices may include cellular telephones, smart phones, tablets, Personal Digital Assistants (PDAs), and the like. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. The gaming system may include a variety of handheld gaming devices, internet-enabled gaming devices, and the like. The client device is capable of executing a variety of different applications, such as various Internet-related applications, communication applications (e.g., email applications), Short Message Service (SMS) applications, and may use a variety of communication protocols.

Network 110 may be any type of network known to those skilled in the art that may support data communications using any of a variety of available protocols, including but not limited to TCP/IP, SNA, IPX, etc. By way of example only, one or more networks 110 may be a Local Area Network (LAN), an ethernet-based network, a token ring, a Wide Area Network (WAN), the internet, a virtual network, a Virtual Private Network (VPN), an intranet, an extranet, a Public Switched Telephone Network (PSTN), an infrared network, a wireless network (e.g., bluetooth, WIFI), and/or any combination of these and/or other networks.

The server 120 may include one or more general purpose computers, special purpose server computers (e.g., PC (personal computer) servers, UNIX servers, mid-end servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and/or combination. The server 120 may include one or more virtual machines running a virtual operating system, or other computing architecture involving virtualization (e.g., one or more flexible pools of logical storage that may be virtualized to maintain virtual storage for the server). In various embodiments, the server 120 may run one or more services or software applications that provide the functionality described below.

The computing units in server 120 may run one or more operating systems including any of the operating systems described above, as well as any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and/or middle tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, and the like.

In some implementations, the server 120 may include one or more applications to analyze and consolidate data feeds and/or event updates received from users of the

client devices

101, 102, 103, 104, 105, and 106. Server 120 may also include one or more applications to display data feeds and/or real-time events via one or more display devices of

client devices

101, 102, 103, 104, 105, and 106.

In some embodiments, the server 120 may be a server of a distributed system, or a server incorporating a blockchain. The server 120 may also be a cloud server, or a smart cloud computing server or a smart cloud host with artificial intelligence technology. The cloud Server is a host product in a cloud computing service system, and is used for solving the defects of high management difficulty and weak service expansibility in the traditional physical host and Virtual Private Server (VPS) service.

The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. The database 130 may reside in various locations. For example, the database used by the server 120 may be local to the server 120, or may be remote from the server 120 and may communicate with the server 120 via a network-based or dedicated connection. The database 130 may be of different types. In certain embodiments, the database used by the server 120 may be, for example, a relational database. One or more of these databases may store, update, and retrieve data to and from the database in response to the command.

In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the application may be different types of databases, such as key-value stores, object stores, or conventional stores supported by a file system.

The system 100 of fig. 1 may be configured and operated in various ways to enable application of the various methods and apparatus described in accordance with the present disclosure.

Fig. 2 is a flow diagram illustrating a method 200 of predicting log deltas in a content delivery network CDN system in accordance with an embodiment of the present disclosure. The CDN system comprises at least one node group, wherein the at least one node group comprises at least one node, and the at least one node comprises at least one log.

For example, a customer of a hundredth CDN system has a manufacturer i, a manufacturer ii, etc., and a plurality of users use a website or application software of the manufacturer i. During certain periods, such as during a promotion, the number of requests for application software access by users of customer vendor i to vendor i may increase, and the CDN system may incrementally distribute the number of requests to each node of the CDN system, resulting in an increased amount of logs in each node. At this time, the log increment corresponding to the log of the CDN system may be determined by the method 200 of predicting the log increment in the content delivery network CDN system.

As shown, the method 200 may include:

step S201, determining a node group request number increment corresponding to at least one node group according to the request number increment of the CDN system and the first request number sharing ratio between the at least one node group.

For example, according to some embodiments, the CDN system is composed of node group a, node group b, node group c, and node group d. The node group a consists of a node a1, a node a2, a node a3 and a node a 4; the node group b consists of a node b1, a node b2 and a node b 3; the node group c consists of a node c1, a node c2, a node c3 and a node c 4; the node group d includes a node d1 and a node d 2. Wherein, the first request number apportionment ratio is as follows: node group a, node group b, node group c and node groupD the ratio of the request number to be shared is A: B: C: D. When the increment of the number of requests of the customer manufacturer i carried by the CDN system is X, the increment of the number of requests shared by the node group a, the node group b, the node group c, and the node group d is respectively:

step S203, determining the corresponding node request number increment of each node in at least one node group according to the node group request number increment and the corresponding second request number sharing proportion between the nodes in at least one node group.

In one example, the second request number apportionment ratio corresponding to the node group is: for example, the second split ratio of the number of requests among the nodes within the node group A is the ratio of the number of requests split by the node a1, the node a2, the node A3 and the node a4, i.e., A1: A2: A3: A4. Since the request number of the node group A is increased to

The request number increment allocated by the node a1, the node a2, the node a3 and the node a4 is respectively as follows:

in the same way, the respective second request number apportionment proportions of the node group B, the node group C and the node group D can be obtained, and the request number increment corresponding to the node in each node group is determined according to the second request number apportionment proportions of each node group and the request number increment of each node group.

Step S205, determining at least one log increment corresponding to at least one log of each node in at least one node group according to the node request number increment and the proportional coefficient relation between the node request number increment and the log increment.

In one example, there are 6 kinds of logs in node a1 of node group A. As used hereinAs used herein, the term scaling factor relationship refers to the ratio of the request number increment to the log increment for a node. For example, the proportional coefficient relationships between the node request number increment corresponding to the node a1 and the log increments corresponding to the 6 types of logs are Y1, Y2, Y3, Y4, Y5, and Y6, respectively. Node a1 has a request number increment of

Therefore, according to the proportional coefficient relationship between the request number increment of the node a1 and the log increments corresponding to the 6 kinds of logs, the log increments corresponding to the 6 kinds of logs can be determined to be respectively

According to some embodiments, all nodes of the same one of the at least one node group belong to the same administrative division and belong to the same operator. For example, node a1, node a2, node a3 and node a4 in node group a all belong to cantonese; node b1, node b2 and node b3 in node group b belong to Guangdong telecom; the node c1, the node c2, the node c3 and the node c4 in the node group c all belong to Zhejiang mobile; both node d1 and node d2 in node group d belong to the Shanghai Mobile. The distribution of users to different customers of the CDN system varies. For example, users of the customer manufacturer i are mainly concentrated in the south china, and when the number of user access requests of the customer manufacturer i increases, nodes of each province in the south china carry more increments than nodes of other provinces; the users of the customer manufacturer II are mainly concentrated in the east China area, and when the number of the user access requests of the customer manufacturer II is increased, the nodes of each province in the east China area bear more increment than the nodes of other provinces. Thus, the node groups are divided according to the administrative divisions and the operators, and the allocation ratio of each node group is calculated, so that the influence of user distribution of different customers can be taken into consideration in calculation.

According to some embodiments, the method 200 may further comprise: and acquiring a request number apportionment proportion corresponding to the type of the estimated time period from the first request number apportionment proportion warehouse as a first request number apportionment proportion.

According to some embodiments, the types of forecast periods include a first period type, a second period type, and a third period type, and wherein, in the first request share proportion warehouse, the request share proportion corresponding to the first period type, the request share proportion corresponding to the second period type, and the request share proportion corresponding to the third period type are calculated by operations including:

-clustering at least one first request share proportion history data of a first period type, at least one first request share proportion history data of a second period type, and at least one first request share proportion history data of a third period type, respectively, using a clustering model.

-taking the cluster center of one of the most concentrated of the first request share ratio history data of the first period type as the request share ratio corresponding to the first period type.

-taking the cluster center of the one of the most concentrated first request share ratios historical data of the second period type as the request share ratio corresponding to the second period type.

-taking the cluster center of the one of the most concentrated first request share ratios historical data of the third period type as the request share ratio corresponding to the third period type.

The first time period type, the second time period type and the third time period type can be set according to the requirements of different customers, for example, in some examples, the first time period type can be a time period manually set by the customer during an e-commerce promotion event, a live broadcast event, a red envelope rush event and the like; the second period type may be a legal holiday period or a weekend period other than the first period type; the third period type may be a weekday late peak period in addition to the first period type and the second period type. In the time periods corresponding to the first time period type, the second time period type and the third time period type, the access requests of the users are increased, but the increase amplitude is different. Moreover, the users in each administrative district have different response degrees to activities, holidays or late peaks, so that the log increment in different periods can be estimated more accurately by acquiring the historical data of the first request number allocation proportion of the first period type, the second period type and the third period type respectively to acquire the request number allocation proportion.

In the clustering model, a value of one dimension of each piece of history data represents an apportionment ratio of one node group in the CDN system. In some examples, in the first request share proportion warehouse, clustering computation is performed on first request share historical data of a plurality of activity periods to obtain a plurality of classes, one class in the most concentrated historical data is taken, and a clustering center of the class is used as a request share proportion corresponding to the activity periods. And performing cluster calculation on the first request number apportionment historical data of a plurality of legal holidays and weekend periods to obtain a plurality of classes, taking one class in the most concentrated historical data, and taking the cluster center of the class as the apportionment proportion of the request numbers corresponding to the legal holidays and the weekend periods. And performing clustering calculation on the first request number apportionment historical data of a plurality of weekday late peak periods to obtain a plurality of classes, taking one class in the most concentrated historical data, and taking the clustering center of the class as the request number apportionment proportion corresponding to the weekday late peak period.

According to some embodiments, the method 200 may further comprise: and acquiring a request number share ratio corresponding to the sampling time interval closest to the estimated time period in time from the second request number share ratio warehouse as a second request number share ratio.

According to some embodiments, in the second request number share proportion warehouse, for any sampling time interval, the request number share proportion corresponding to the sampling time interval is calculated by operations including:

-clustering the at least one second request share proportion history data within the sampling time interval using a clustering model.

-taking the cluster center of one of the most concentrated classes of the second request share ratio history data as the request share ratio corresponding to the sampling time interval.

The value of one dimension of each piece of history data in the clustering model represents the apportionment ratio of one node in a node group. In one example, the cluster calculation is performed on the plurality of pieces of second request share ratio historical data to obtain a plurality of classes, one class in the most concentrated second request share ratio historical data is taken, and the cluster center of the class is used as the request share ratio.

In some examples, in the second request share proportion warehouse, a plurality of pieces of history data of the second request share proportion collected every 1 hour before 3 hours may be acquired, the history data may be clustered, a cluster center of one class in which the history data is most concentrated may be calculated, and the cluster center may be used as the request share proportion. Since the bearing capacity of the node is dynamically updated, if the time distance of the historical data is estimated too long, the referential performance of the node is lost. Therefore, the request number apportionment proportion corresponding to the sampling time interval closest to the estimated time interval is obtained as the second request number apportionment proportion, and the estimated result is more referential.

According to some embodiments, the method 200 may further comprise: and acquiring a coefficient relation corresponding to the sampling time interval closest to the estimated time interval in time from the proportional coefficient relation warehouse as a proportional coefficient relation.

According to some embodiments, in the scale factor relationship repository, for any sampling time interval, the factor relationship corresponding to that sampling time interval is calculated by operations comprising:

-obtaining at least one piece of scaling factor relationship history data within the sampling time interval.

-taking a statistic of at least one piece of scaling factor relationship history data as the factor relationship corresponding to the sampling time interval.

According to some embodiments, the statistical values comprise: median, mode, mean.

In some examples, in the proportional coefficient relationship warehouse, 7 pieces of historical data of proportional coefficient relationships of late peak periods of each day of the previous week may be taken, and for each log, the median of the 7 pieces of historical data corresponding to the log may be taken as the coefficient relationship corresponding to the log.

According to some embodiments, the at least one log comprises: one or more of nginx access log, nginx error log, nginx charging log, cache access log, cache charging log and cache source return log. Wherein the nginx access log records the request information of the user; the nginx error log records 4XX, 5XX and other error reporting information; the nginx charging log records the bandwidth information requested by the user; the cache accesses the log record to access the cached information; the cache charging log records the bandwidth information of the access cache; and the cache records the information of the data requested by the source station back to the source log.

FIG. 3 illustrates a block diagram of an exemplary system 300 that can be used to implement embodiments of the present disclosure. Referring to fig. 3, the system 300 includes an online service system 310, a warehouse system 320, and an offline computing system 330. The offline calculation system 330 includes a first request share proportion calculation system 331, configured to calculate a corresponding request share proportion by inputting the plurality of pieces of first request share proportion history data of each client into the clustering model, and store the trained request share proportion to the first request share proportion warehouse 321. The offline calculation system 330 further includes a second request share proportion calculation system 332, which is configured to train a request share proportion by inputting a plurality of pieces of second request share proportion history data into the clustering model, and store the trained request share proportion in the second request share proportion warehouse 322. The offline computing system 330 further comprises a scaling factor relationship computing system 333, which is configured to compute a scaling factor relationship according to the historical data of the scaling factor relationship, and store the computed scaling factor relationship in the scaling factor relationship repository 323.

The online service system 310 includes an API server 311, a log increment pre-estimation system 312, a log increment table 313, and a reporting system 314. And submitting a calculation request through the API server 311, where the calculation request includes information about selecting the type of the forecast period as a first period type, a second period type, or a third period type, selecting a client, and inputting a request number increment of a CDN system of the client. After receiving the calculation request, the log increment pre-estimation system 312 queries and acquires the specified first request number apportionment proportion, second request number apportionment proportion and scale coefficient relationship from the first request number apportionment proportion warehouse 321, the second request number apportionment proportion warehouse 322 and the scale coefficient relationship warehouse 323, calculates to obtain log increment data, and persists the log increment data in the log increment table 313. Reporting system 314 exposes the data in log delta table 313.

According to another aspect of the present disclosure, as shown in fig. 4, an apparatus 400 for estimating a log increment in a CDN system of a content delivery network is also provided. The CDN system comprises at least one node group, wherein the at least one node group comprises at least one node, and the at least one node comprises at least one log. The apparatus 400 includes a first determination module 402, a second determination module 404, and a third determination module 406.

The first determining module 402 is configured to determine a node group request number increment corresponding to at least one node group according to a request number increment of the CDN system and a first request number split ratio between the at least one node group.

The second determining module 404 is configured to determine a corresponding node request number increment for each node in the at least one node group based on the node group request number increment and a corresponding second request number split ratio between nodes in the at least one node group.

The third determining module 406 is configured to determine a log increment corresponding to at least one log of each node in the at least one node group according to the node request number increment and a proportionality coefficient relationship between the node request number increment and the log increment.

According to some embodiments, all nodes of the same one of the at least one node group belong to the same administrative division and belong to the same operator.

According to some embodiments, the apparatus 400 further comprises: a first obtaining module 401 configured to obtain, as a first request number apportionment ratio, a request number apportionment ratio corresponding to the type of the forecast time period from the first request number apportionment ratio warehouse.

According to some embodiments, the apparatus 400 further comprises a second obtaining module 403 configured to obtain, from the second request share proportion repository, the request share proportion corresponding to the temporally closest sampling time interval of the forecast period as the second request share proportion.

According to some embodiments, in the second request share proportion warehouse, for any sampling time interval, the request share proportion corresponding to the sampling time interval is calculated by an operation including:

According to some embodiments, the apparatus 400 further comprises a third obtaining module 405 configured to obtain, as the scaling factor relationship, a coefficient relationship corresponding to a temporally closest sampling time interval of the pre-estimation period from a scaling factor relationship repository.

It should be understood that the various modules of the apparatus 400 shown in fig. 4 may correspond to the various steps in the method 200 described with reference to fig. 2. Thus, the operations, features and advantages described above with respect to the method 200 are equally applicable to the apparatus 400 and the modules comprised thereby. Certain operations, features and advantages may not be described in detail herein for the sake of brevity.

Although specific functionality is discussed above with reference to particular modules, it should be noted that the functionality of the various modules discussed herein may be divided into multiple modules and/or at least some of the functionality of multiple modules may be combined into a single module. Performing an action by a particular module discussed herein includes the particular module itself performing the action, or alternatively the particular module invoking or otherwise accessing another component or module that performs the action (or performs the action in conjunction with the particular module). Thus, a particular module that performs an action can include the particular module that performs the action itself and/or another module that the particular module invokes or otherwise accesses that performs the action.

It should also be appreciated that various techniques may be described herein in the general context of software, hardware elements, or program modules. The various modules described above with respect to fig. 4 may be implemented in hardware or in hardware in combination with software and/or firmware. For example, the modules may be implemented as computer program code/instructions configured to be executed in one or more processors and stored in a computer-readable storage medium. Alternatively, the modules may be implemented as hardware logic/circuitry.

According to another aspect of the present disclosure, as shown in fig. 5, there is also provided an electronic device including: at least one processor; a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the above-described method.

According to another aspect of the present disclosure, there is also provided a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the above method.

According to another aspect of the present disclosure, there is also provided a computer program product comprising a computer program, wherein the computer program realizes the above method when executed by a processor.

Referring to fig. 5, a block diagram of a structure of an electronic device 500, which may be a server or a client of the present disclosure, which is an example of a hardware device that may be applied to aspects of the present disclosure, will now be described. In one example, log delta prediction system 312 of FIG. 3 can take the form of electronic device 500. Electronic device is intended to represent various forms of digital electronic computer devices, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device 500 may also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 5, the device 500 comprises a computing unit 501 which may perform various suitable actions and processes according to a computer program stored in a Read Only Memory (ROM)502 or a computer program loaded from a storage unit 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The calculation unit 501, the ROM 502, and the RAM 503 are connected to each other by a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.

A number of components in the device 500 are connected to the I/O interface 505, including: an input unit 506, an output unit 507, a storage unit 508, and a communication unit 509. The input unit 506 may be any type of device capable of inputting information to the device 500, and the input unit 506 may receive input numeric or character information and generate key signal inputs related to user settings and/or function controls of the electronic device, and may include, but is not limited to, a mouse, a keyboard, a touch screen, a track pad, a track ball, a joystick, a microphone, and/or a remote controller. Output unit 507 may be any type of device capable of presenting information and may include, but is not limited to, a display, speakers, a video/audio output terminal, a vibrator, and/or a printer. The storage unit 508 may include, but is not limited to, a magnetic disk, an optical disk. The communication unit 509 allows the device 500 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers and/or chipsets, such as bluetooth (TM) devices, 1302.11 devices, WiFi devices, WiMax devices, cellular communication devices, and/or the like.

The computing unit 501 may be a variety of general-purpose and/or special-purpose processing components having processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and so forth. The computing unit 501 performs the various methods and processes described above, such as the method 200. For example, in some embodiments, the method 200 may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 500 via the ROM 502 and/or the communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the method 200 described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the method 200 by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be performed in parallel, sequentially or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.

Although embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it is to be understood that the above-described methods, systems and apparatus are merely exemplary embodiments or examples and that the scope of the present invention is not limited by these embodiments or examples, but only by the claims as issued and their equivalents. Various elements in the embodiments or examples may be omitted or may be replaced with equivalents thereof. Further, the steps may be performed in an order different from that described in the present disclosure. Further, various elements in the embodiments or examples may be combined in various ways. It is important that as technology evolves, many of the elements described herein may be replaced with equivalent elements that appear after the present disclosure.

Claims

1. A method for predicting log increment in a Content Delivery Network (CDN) system, wherein the CDN system comprises at least one node group, each node group comprises at least one node, each node comprises at least one log, and the method comprises the following steps:

determining a node group request number increment corresponding to the at least one node group according to the request number increment of the CDN system and a first request number sharing proportion between the at least one node group;

determining the corresponding node request number increment of each node in the at least one node group according to the node group request number increment and the corresponding second request number sharing proportion between the nodes in the at least one node group; and

and determining the log increment corresponding to the at least one log of each node in the at least one node group according to the node request number increment and the proportional coefficient relationship between the node request number increment and the log increment.

2. The method of claim 1, wherein all nodes of a same one of the at least one node group belong to a same administrative division and belong to a same operator.

3. The method of claim 1, further comprising:

and acquiring a request number apportionment proportion corresponding to the type of the estimated time period from a first request number apportionment proportion warehouse as the first request number apportionment proportion.

4. The method of claim 3, wherein the types of forecast periods include a first period type, a second period type, and a third period type, and wherein, in the first request share proportion warehouse, a request share proportion corresponding to the first period type, a request share proportion corresponding to the second period type, and a request share proportion corresponding to the third period type are calculated by operations comprising:

clustering at least one piece of first request number apportionment proportion historical data of the first time period type, at least one piece of first request number apportionment proportion historical data of the second time period type and at least one piece of first request number apportionment proportion historical data of the third time period type by utilizing a clustering model;

taking a cluster center of a class in the most concentrated first request number allocation proportion historical data of the first time period type as a request number allocation proportion corresponding to the first time period type;

taking the cluster center of one class in the most concentrated first request number apportionment ratio historical data of the second time period type as the request number apportionment ratio corresponding to the second time period type; and

and taking the cluster center of one class in the most concentrated first request number allocation proportion historical data of the third time interval type as the request number allocation proportion corresponding to the third time interval type.

5. The method of claim 1, further comprising:

and acquiring a request number share ratio corresponding to the sampling time interval closest to the estimated time period in time from a second request number share ratio warehouse as the second request number share ratio.

6. The method of claim 5, wherein in the second request share proportion store, for any sampling time interval, the request share proportion corresponding to that sampling time interval is calculated by operations comprising:

clustering at least one second request number apportionment proportion historical data in the sampling time interval by using a clustering model; and

and taking the cluster center of one class in the most concentrated historical data of the second request number apportionment ratio as the request number apportionment ratio corresponding to the sampling time interval.

7. The method of claim 1, further comprising:

and acquiring a coefficient relation corresponding to the sampling time interval closest to the estimated time interval in time from a proportional coefficient relation warehouse as the proportional coefficient relation.

8. The method of claim 7, wherein, in the scaling factor relationship repository, for any sampling time interval, the factor relationship corresponding to that sampling time interval is calculated by an operation comprising:

acquiring at least one piece of proportionality coefficient relation historical data in the sampling time interval; and

and taking the statistic value of the at least one piece of scale coefficient relation historical data as the coefficient relation corresponding to the sampling time interval.

9. The method of claim 8, wherein the statistical values comprise: median, mode, mean.

10. The method of any of claims 1-9, wherein the at least one log comprises: one or more of nginx access log, nginx error log, nginx charging log, cache access log, cache charging log and cache source return log.

11. An apparatus for predicting log increment in a Content Delivery Network (CDN) system, wherein the CDN system comprises at least one node group, each node group comprises at least one node, each node comprises at least one log, and the apparatus comprises:

a first determining module, configured to determine a node group request number increment corresponding to the at least one node group according to a request number increment of the CDN system and a first request number sharing ratio between the at least one node group;

a second determining module configured to determine a corresponding node request number increment for each node in the at least one node group according to the node group request number increment and a corresponding second request number apportionment ratio between the nodes in the at least one node group; and

and the third determining module is configured to determine the corresponding log increment of the at least one log of each node in the at least one node group according to the node request number increment and the proportional coefficient relation between the node request number increment and the log increment.

12. The apparatus of claim 11, wherein all nodes of a same one of the at least one node group belong to a same administrative division and belong to a same operator.

13. The apparatus of claim 11, further comprising:

a first obtaining module configured to obtain a request number share ratio corresponding to a type of a forecast time period from a first request number share ratio warehouse as the first request number share ratio.

14. The apparatus of claim 13, wherein the types of forecast periods comprise a first period type, a second period type, and a third period type, and wherein, in the first request share proportion warehouse, a request share proportion corresponding to the first period type, a request share proportion corresponding to the second period type, and a request share proportion corresponding to the third period type are calculated by operations comprising:

15. The apparatus of claim 11, further comprising:

a second obtaining module configured to obtain, as the second request number apportionment ratio, a request number apportionment ratio corresponding to a sampling time interval closest in time to the forecast time period from a second request number apportionment ratio warehouse.

16. The apparatus of claim 15, wherein in the second request share proportion store, for any sampling time interval, the request share proportion corresponding to that sampling time interval is calculated by operations comprising:

clustering at least one piece of second request number apportionment ratio historical data in the sampling time interval by using a clustering model; and

17. The apparatus of claim 11, further comprising:

a third obtaining module configured to obtain, from the proportional coefficient relationship repository, a coefficient relationship corresponding to a sampling time interval temporally closest to the estimated time period as the proportional coefficient relationship.

18. The apparatus of claim 17, wherein, in the scaling factor relationship repository, for any sampling time interval, the factor relationship corresponding to that sampling time interval is calculated by an operation comprising:

19. The apparatus of claim 18, wherein the statistical value comprises: median, mode, mean.

20. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-10.

21. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-10.

22. A computer program product comprising a computer program, wherein the computer program realizes the method of any one of claims 1-10 when executed by a processor.