CN111935306B - Node scheduling method and device - Google Patents

Node scheduling method and device Download PDF

Info

Publication number
CN111935306B
CN111935306B CN202010950849.0A CN202010950849A CN111935306B CN 111935306 B CN111935306 B CN 111935306B CN 202010950849 A CN202010950849 A CN 202010950849A CN 111935306 B CN111935306 B CN 111935306B
Authority
CN
China
Prior art keywords
node
hash
url
nodes
value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010950849.0A
Other languages
Chinese (zh)
Other versions
CN111935306A (en
Inventor
吴枭
黄宏文
胡文宇
李童星
陈晓杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202010950849.0A priority Critical patent/CN111935306B/en
Publication of CN111935306A publication Critical patent/CN111935306A/en
Application granted granted Critical
Publication of CN111935306B publication Critical patent/CN111935306B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1001Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
    • H04L67/1004Server selection for load balancing
    • H04L67/1008Server selection for load balancing based on parameters of servers, e.g. available memory or workload
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/955Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061Partitioning or combining of resources
    • G06F9/5072Grid computing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061Partitioning or combining of resources
    • G06F9/5077Logical partitioning of resources; Management or configuration of virtualized resources
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00Indexing scheme relating to G06F9/00
    • G06F2209/50Indexing scheme relating to G06F9/50
    • G06F2209/5011Pool

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Computer Hardware Design (AREA)
  • Signal Processing (AREA)
  • Mathematical Physics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Data Mining & Analysis (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The invention discloses a node scheduling method and a node scheduling device, which can be applied to network scheduling in the technical field of cloud. Wherein, the method comprises the following steps: determining the number of first nodes corresponding to a first Uniform Resource Locator (URL) according to the access times of the first URL of the URL to be accessed in a first time period; searching a first target node in the first node subset according to the node arrangement sequence in the first hash chain; and under the condition that the first target node is found in the first node subset, sending the node information of the first target node to the first client.

Description

Node scheduling method and device
Technical Field
The invention relates to the field of computers, in particular to a node scheduling method and device
Background
Domain Name Service (DNS) makes access to the internet more ergodic by mapping Domain names and IP addresses to each other. DNS is an indispensable service for the development of the internet.
Most applications and services use domain names as service portals. In the prior art, various scheduling modes exist, for example, regional scheduling is realized by using DNS. In the case of massive node scheduling, the scheduling that the prior art can have has the following problems: (1) the number of nodes is too large, the number of Internet protocols (IP for short) limited by DNS for single resolution is small, and the resource of a single node is too small to bear a large number of requests of users; (2) for nodes with poor stability, the rejection delay is too high to be acceptable, and the nodes cannot be rejected in real time; (3) the system is easy to hijack by operators, so that the dispatching does not follow the dispatching service command, the single machine load is too high, and the dispatching result is not in accordance with the expectation; (4) the node scheduling resource utilization rate is low; (5) the node cache hit rate is low.
No effective solution exists to the above problems.
Disclosure of Invention
The embodiment of the invention provides a node scheduling method and a node scheduling device, which are used for at least solving the technical problem of low utilization rate of node scheduling resources.
According to an aspect of the embodiments of the present invention, a node scheduling method is provided, including: determining a first node quantity corresponding to a first Uniform Resource Locator (URL) according to the access times of the first URL to be accessed in a first time length, wherein the first node quantity is the node quantity in a first node subset which is allowed to be scheduled for the first URL in a target node set; searching a first target node in the first node subset according to a node arrangement sequence in a first hash chain, wherein the first hash chain comprises the first node subset arranged according to the node arrangement sequence, the node arrangement sequence is determined according to a hash value related to the first URL, and the remaining scheduling times of the first target node are greater than 0; and sending the node information of the first target node to the first client under the condition that the first target node is found in the first node subset.
According to another aspect of the embodiments of the present invention, there is also provided a node scheduling apparatus, including: the determining module is used for determining a first node number corresponding to a first Uniform Resource Locator (URL) according to the access times of the first URL to be accessed in a first time length, wherein the first node number is the number of nodes in a first node subset which is allowed to be scheduled for the first URL in a target node set; a searching module, configured to search a first target node in the first node subset according to a node arrangement order in a first hash chain, where the first hash chain includes the first node subset arranged according to the node arrangement order, the node arrangement order is determined according to a hash value related to the first URL, and a remaining scheduling number of the first target node is greater than 0; a sending module, configured to send node information of the first target node to the first client when the first target node is found in the first node subset.
According to another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, in which a computer program is stored, where the computer program is configured to execute the above node scheduling method when running.
According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including a memory and a processor, where the memory stores a computer program, and the processor is configured to execute the node scheduling method through the computer program.
In the embodiment of the invention, the number of first nodes corresponding to a first Uniform Resource Locator (URL) to be accessed is determined according to the access times of the first URL in a first time length; searching a first target node in the first node subset according to the node arrangement sequence in the first hash chain; and under the condition that the first target node is found in the first node subset, sending the node information of the first target node to the first client. Therefore, the technical effect of improving the resource utilization rate is achieved, and the technical problem of low resource utilization rate of node scheduling is solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:
fig. 1 is a schematic diagram of an application environment of an alternative node scheduling method according to an embodiment of the present invention;
FIG. 2 is a flow chart of an alternative node scheduling method according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an alternative node scheduling according to an embodiment of the present invention;
FIG. 4 is an alternative system framework diagram according to an embodiment of the invention;
figure 5 is a schematic diagram of an alternative hash chain according to an embodiment of the present invention;
FIG. 6 is a diagram of an alternative hash chain according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of an alternative conventional consistent hash according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of an alternative virtual node according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of an alternative node storage sharing schedule according to an embodiment of the present invention;
FIG. 10 is a schematic diagram of an alternative bloom filter in accordance with an embodiment of the present invention;
FIG. 11 is a schematic diagram of an alternative low-utilization hash chain in accordance with an embodiment of the present invention;
FIG. 12 is a schematic diagram of an alternative quota obtaining logic, according to an embodiment of the invention;
fig. 13 is a block diagram of a node scheduling apparatus according to an embodiment of the present invention;
fig. 14 is a schematic structural diagram of an alternative electronic device according to an embodiment of the invention.
Detailed Description
Cloud computing (cloud computing) is a computing model that distributes computing tasks over a pool of resources formed by a large number of computers, enabling various application systems to obtain computing power, storage space, and information services as needed. The network that provides the resources is referred to as the "cloud". Resources in the "cloud" appear to the user as being infinitely expandable and available at any time, available on demand, expandable at any time, and paid for on-demand.
As a basic capability provider of cloud computing, a cloud computing resource pool (called as an ifas (Infrastructure as a Service) platform for short is established, and multiple types of virtual resources are deployed in the resource pool and are selectively used by external clients.
According to the logic function division, a PaaS (Platform as a Service) layer can be deployed on an IaaS (Infrastructure as a Service) layer, a SaaS (Software as a Service) layer is deployed on the PaaS layer, and the SaaS can be directly deployed on the IaaS. PaaS is a platform on which software runs, such as a database, a web container, etc. SaaS is a variety of business software, such as web portal, sms, and mass texting. Generally speaking, SaaS and PaaS are upper layers relative to IaaS.
With the development of diversification of internet, real-time data stream and connecting equipment and the promotion of demands of search service, social network, mobile commerce, open collaboration and the like, cloud computing is rapidly developed. Different from the prior parallel distributed computing, the generation of cloud computing can promote the revolutionary change of the whole internet mode and the enterprise management mode in concept.
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
According to an aspect of the embodiments of the present invention, a node scheduling method is provided, and optionally, as an optional implementation manner, the node scheduling method may be applied, but not limited, to the environment shown in fig. 1. As shown in fig. 1, the server may be a Content Delivery Network (CDN), which may be referred to as a CDN server, and is connected to the user equipment through a Network, where the Network includes but is not limited to: the user equipment is not limited to a PC, a mobile phone, a tablet computer, etc., and may be a mobile terminal such as a mobile phone, a tablet computer, etc., and the user equipment is provided with a client, which may be an instant messaging client, a video client, an education client, etc. The node scheduling method of the embodiment of the invention can be executed by a CDN server.
Optionally, as an optional implementation manner, as shown in fig. 2, the node scheduling method includes the following steps:
step S202, determining a first node number corresponding to a first Uniform Resource Locator (URL) according to the access times of the first URL to be accessed in a first time length, wherein the first node number is the number of nodes in a first node subset which is allowed to be scheduled for the first URL in a target node set;
step S204, searching a first target node in the first node subset according to a node arrangement sequence in a first hash chain, wherein the first hash chain comprises the first node subset arranged according to the node arrangement sequence, the node arrangement sequence is determined according to the hash value related to the first URL, and the remaining scheduling frequency of the first target node is greater than 0;
step S206, sending the node information of the first target node to the first client when the first target node is found in the first node subset.
As an optional implementation manner, a Content Delivery Network (CDN) may be referred to as a CDN server, and the present embodiment may be executed on the CDN server. Fig. 3 is a schematic diagram of an alternative node scheduling method according to an embodiment of the present invention. When a user accesses a website, a Uniform Resource Locator (URL) is sent to a CDN server through a client, the CDN server selects a target node from a large number of nodes, and returns node information of the target node to the client, where the node information includes, but is not limited to, an address of the node on the network and a parameter for subsequent access. The client-side carries out information interaction with the target node, and resources are provided for the client-side through the target node. A node may be a computer device with computing, storage, and network capabilities. The client in this embodiment may be a game client, a video client, an education client, or the like.
As an alternative embodiment, the user may input a uniform resource locator URL at the client, and call the resource in the node, which may be the server, through the URL. By accessing the URL, the server may return a network resource, such as a web page, video, audio, etc., corresponding to the URL to the client.
As an optional implementation manner, as shown in fig. 4, which is an optional system framework schematic diagram according to an embodiment of the present invention, in this embodiment, the system may include a heat statistics module, a hash chain acquisition module, and a quota allocation module. The popularity statistic module is configured to determine an access popularity final _ k of the URL, and specifically, the number of times of accessing the URL in a preset time period may be used as the access popularity of the URL. The hash chain obtaining module may obtain the hash chain corresponding to the URL with final _ k through a network Protocol (Internet Protocol, abbreviated as IP). The quota allocation module can acquire the nodes with the scheduling quota in the hash chain, and send the node information with the scheduling quota to the client, so as to perform information interaction with the client and allocate resources to the client. In this embodiment, the input is a URL and a user IP that a user accesses, the access heat is acquired through the heat statistics module, a corresponding hash chain is acquired through the URL, the IP and the access heat, a node with quota is acquired through the quota allocation module, and finally information of a certain node is selected and returned. Therefore, the technical effects of ensuring the system recovery rate and fully utilizing the node resources to avoid resource waste are achieved.
Optionally, the searching for the first target node in the first node subset according to the node arrangement order in the first hash chain includes: repeatedly executing the following steps until the first target node is found, or traversing the first node subset, wherein an initial value of a current preset value is a first value, and each node in the target node set is provided with a different hash value corresponding to a first hash operation: executing the first hash operation on the first URL and the current preset value to obtain a current hash value; searching a node with a hash value being the current hash value in the target node set; determining the node with the hash value as the current hash value as the first target node under the condition that the node with the hash value as the current hash value is found and the remaining scheduling times of the node with the hash value as the current hash value are greater than 0; and under the condition that the node with the hash value as the current hash value is found and the residual scheduling times of the node with the hash value as the current hash value are equal to 0, increasing the current preset value by a preset value.
As an optional implementation manner, after the access heat is obtained through the heat statistics, the node set meeting the scheduling quality may be obtained by querying the address library through the IP. In this embodiment, the corresponding node may be identified by calculating a hash value of the URL and the sense. The value of the nonsense may be a preset value, and may be an integer starting from 1, for example, the nonsense =10001, 10002, and the like. The hot value final _ k of the URL is related to the length of the chain, which may be equal. The quota on the node A can be used for a plurality of URLs, chains corresponding to the URLs are different, but the nodes on the chains are from partial nodes in the same node set, and a fixed-length (the length is determined by the access heat) hash chain can be obtained by using a consistent hash after key splicing.
As an optional embodiment, the corresponding chain of each URL is different, and a unique hash scheduling chain of the URL may be obtained for each URL, as shown in fig. 5, which is an optional hash chain, and as shown in fig. 6, which is another optional hash chain. Assuming FIG. 5 is the hash chain corresponding to URL1, a hash schedule chain of A- > C- > B- > F- > D- > E is available for URL 1. FIG. 6 is a hash chain corresponding to URL2, a hash schedule chain of C- > B- > D- > E- > A- > F is available for URL 2. And each node included in each hash chain is configured with a scheduling quota, and the quota is reduced once each time the node is scheduled. The quota is inquired in sequence through the hash chain, the request can be converged in the first hop nodes as far as possible, in addition, as the heat degree is higher, the hash chain is longer, the hot URL can be diffused into more nodes, and the cold URL can be prevented from being diffused in large quantity to cause the flow to return to the source.
As an alternative embodiment, the example is given by the first subset of nodes including node A, B, C, D, E, F, hash chain A- > C- > B- > F- > D- > E. Assuming that the current preset value noonsense =10001, performing Hash operation on the first URL and the noonsense to obtain a current Hash value Hash (URL + 10001). After the Hash value of the node A is determined to be Hash (URL +10001) in the Hash chain A- > C- > B- > F- > D- > E, the remaining schedulable times of the node A are inquired, if the remaining schedulable times of the node A is larger than 0, the node A is taken as a target node, and node information of the node A is sent to a client. If the remaining scheduling times of the node a is equal to 0, it indicates that the node a cannot be scheduled. Increasing a predetermined value for the nonsense =10001, assuming that the predetermined value is 1, changing the value of the nonsense to the value of the nonsense =10002, continuing to perform Hash operation on the first URL and the nonsense =10002 to obtain Hash (URL +10002), after determining that the Hash value of the node B is Hash (URL +10002) in a Hash chain a- > C- > B- > F- > D- > E, querying the remaining schedulable times of the node B, and if the remaining schedulable times of the node B is greater than 0, taking the node B as a target node. And repeating the steps until the target node is found in the hash chain or traversing the hash chain. In this embodiment, by searching for a node with a remaining scheduling frequency greater than 0 in the hash chain corresponding to the URL as the target node, the efficiency of node scheduling can be improved, and the utilization rate of node resources can be improved.
Optionally, the performing the first hash operation on the first URL and the current preset value to obtain a current hash value includes: executing the first hash operation on the sum of the first URL and the current preset value to obtain the current hash value; or splicing the first URL and the current preset value to obtain a first spliced character string; and executing the first hash operation on the first splicing character string to obtain the current hash value.
As an alternative, a hash operation may be performed on the sum of the URL and the nonsense, for example, the URL is "www.abc.com", and the "nonsense" is 1001, a binary value corresponding to "www.abc.com" may be obtained, and the binary value and 1001 are added to perform the hash operation. Alternatively, the binary value of "www.abc.com, nonsense" may be concatenated with "1001" to calculate the hash value of the concatenated value.
Optionally, the method further comprises: repeatedly executing the following steps until i = the first node number, wherein the initial value of the current preset value is a first value, and the initial value of i is 1: executing the first hash operation on the first URL and the current preset value to obtain a current hash value; searching a node with a hash value being the current hash value in the target node set; setting the node with the hash value as the current hash value as the ith node in the first hash chain under the condition that the node with the hash value as the current hash value is found; and under the condition that i is less than the first node number, i = i +1, and the current preset value is increased by a preset value.
As an optional implementation manner, the position of each node in the target node set in the hash chain may be determined according to the size of the hash value between each node in the target node set and the preset value nsense, and further, the arrangement order of each node in the hash chain may be determined. In this embodiment, the preset value noonsense may be variable, the preset value noonsense may be set to an initial value, the value of the noonsense is increased by a predetermined value every time the position of a node in the hash chain is determined, and the size of the predetermined value may be determined according to actual situations, and is not limited herein, and may be, for example, 1, 2, 10, 100, and the like, until all nodes in the hash chain are determined.
As an optional implementation manner, taking an example that the target node set includes the node A, B, C, D, E, F, and taking an example that the current preset value is noonsense =10001, the description is made. And performing Hash operation on the URL and 10001 to obtain a current Hash value Hash (URL + 10001). Assuming that the Hash value corresponding to the node a in the target node set is equal to the Hash (URL +10001), the node a is determined to be the first node in the first Hash chain. Assuming that the predetermined value is 1, adding the predetermined value to the nonsense to obtain nonsense =10002, performing Hash operation on the URL and 10002 to obtain a current Hash value Hash (URL +10002), and assuming that the Hash value corresponding to the node C in the target node set is equal to the Hash (URL +10002), determining that the node C is a second node in the first Hash chain. By analogy, until all nodes in the target node set are traversed, the arrangement sequence of the nodes in the target node set in the hash chain can be determined according to the hash values of the URL and the nonsense, and the hash chain A- > C- > B- > F- > D- > E is obtained. In this embodiment, the arrangement order of the nodes in the hash chain corresponding to the URL is determined by the hash value, and the target node can be determined in the target node set corresponding to the URL according to the size order of the hash value, so that the search efficiency of the target node can be improved.
Optionally, the method further comprises: and under the condition that the hash result of the first hash operation comprises N different hash values and the target node set comprises N nodes, setting one hash value of the N different hash values for each node of the N nodes, wherein the N different hash values have a one-to-one correspondence relationship with the N nodes, and N is a natural number greater than 1.
As an alternative implementation, fig. 7 is a schematic diagram of a conventional consistent hash according to an alternative embodiment of the present invention, where the conventional consistent hash requires a large amount of node states to be stored and a large amount of node partitioning operations to be performed. This results in high maintenance costs and computational inefficiency. In the embodiment, jump consistency hash (jump consistency hash) is selected and used, so that the discrete type and the uniformity are ensured through large number operation, and the memory use and node change cost is greatly reduced.
As an alternative, one real node (e.g., one server) may correspond to one virtual node in the hash chain, with each real node being assigned a hash value. In this embodiment, for nodes with different resource holding amounts, the nodes may be divided into multiple virtualized nodes according to the resource holding amounts, each node serves as a node in a hash chain, as shown in fig. 8, which is a schematic diagram of virtual nodes according to an optional embodiment of the present invention, ABCDEF are all virtual nodes, and a real node AR may be obtained by querying after selecting a virtual node a, so that it is ensured that the nodes may be weight-divided according to different resource holding amounts, and that resources are used equally.
Optionally, the method further comprises: combining part of the M nodes into a virtual node under the condition that a hash result of the first hash operation comprises N different hash values, the target node set comprises M nodes and M is greater than N, so as to obtain N nodes in total, and setting one hash value of the N different hash values for each node of the N nodes, wherein the N different hash values have a one-to-one correspondence relationship with the N nodes, N is a natural number greater than 1, and the N nodes comprise one or more virtual nodes; or under the condition that the hash result of the first hash operation includes N different hash values, the target node set includes M nodes, and M is smaller than N, splitting some of the M nodes into a plurality of shared nodes to obtain N nodes in total, and setting one of the N different hash values for each of the N nodes, where the N different hash values have a one-to-one correspondence with the N nodes, N is a natural number greater than 1, and the N nodes include a plurality of the shared nodes.
As an optional implementation manner, fig. 9 is a schematic diagram illustrating node storage sharing scheduling according to an optional embodiment of the present invention, where a batch of scheduling nodes AR1/AR2/AR3 may have shared storage, and these nodes may be abstracted to be a large scheduling node AF in scheduling, and participate in scheduling with this node as a real node, and after obtaining the AF node by scheduling, the AF node is subdivided into a specific single scheduling node AR1/AR2/AR3 according to a quota, so that resource utilization is improved and return cost is reduced without changing scheduling logic. In this embodiment, a plurality of real nodes may be merged into one virtual node, and a hash value is assigned to the virtual node, that is, a plurality of real nodes may be merged into one schedulable virtual node. For example, the real node A, B, C may be A, B, C three server nodes, and A, B, C three server nodes may be merged into one virtual node D, and a hash value is assigned to the virtual node D. In this embodiment, a real node may also be split into multiple nodes as shared nodes, and a hash value may be allocated to each split shared node. For example, the real node server a may be split into shared nodes a1, a2, A3, each assigned a hash value. In this embodiment, a plurality of real nodes with a small remaining scheduling frequency may be merged into one virtual node, or one real node with a large remaining scheduling frequency may be split into a plurality of shared nodes. By combining multiple nodes into one node, or splitting one node into multiple shared nodes. The resource allocation of the nodes can be balanced, the problem of low resource utilization rate caused by unbalanced node scheduling is solved, and the technical effect of improving the resource scheduling utilization rate of the nodes can be achieved.
Optionally, the determining, according to the number of times of access of a first URL of a uniform resource locator to be accessed in a first duration, the number of first nodes corresponding to the first URL includes: determining a first hot value of the first URL according to the access times of the first URL in the first time length; and determining the first node quantity corresponding to the first URL according to the first heat value, wherein the first node quantity is positively correlated with the first heat value.
As an optional implementation manner, the number of nodes included in the hash chain may be obtained according to the access heat of the URL, and the higher the access heat of the URL is, the greater the number of nodes included in the hash chain corresponding to the URL is. In this embodiment, the URL access heat may be determined according to the access frequency of the URL within a certain time period, for example, within 5 minutes, the access frequency of the URL is 5 times, the number of nodes included in the hash chain corresponding to the URL may be 5 or may be a multiple of 5, and the number of nodes included in the hash chain corresponding to the URL is positively correlated with the access heat of the URL.
Optionally, the determining a first popularity value of the first URL according to the number of times of access of the first URL in the first duration includes: setting the first heat value final _ k to be: final _ k = k (2 × interval-now _ interval)/interval; and k is the number of times of accessing the first URL in the first time length, interval is the time length of a preset statistical period, and now _ interval is the time length between the current time and the starting time of the last completed statistical period, and the first time length is now _ interval.
As an optional implementation mode, the visit times can reflect the visit heat of the URL, and the visit times of the URL are counted through the heat counting module. The hot statistics module may use a Multiple Bloom Filter (Multiple Bloom Filter) to return a hot value final _ k of each URL, which may be the number of accesses of the URL, of this URL.
As an alternative implementation, fig. 10 is a schematic diagram of a bloom filter, which may be a Bitmap or Bitmap/bitraray, for retrieving whether an element is in a set, according to an alternative embodiment of the present invention. Assuming that the number of input objects is n, the size of the bloom filter is m, and the tolerable misjudgment rate p and the number k of the hash functions are n. The calculation formula is as follows:
Figure DEST_PATH_IMAGE001
Figure 39990DEST_PATH_IMAGE002
Figure DEST_PATH_IMAGE003
the decimal is rounded up, the URL with the size of 64b can be input in 10 hundred million levels, and only 2.5G of memory is needed under the condition that the misjudgment rate is 0.01 percent.
In this embodiment, if the bloom filter continues to operate, the heat level will eventually expand due to the lack of the record cleaning mechanism, and the heat level statistics will eventually fail. Because the space overhead brought by recording the access time of all the URLs is not acceptable and cannot be eliminated according to the access time of the URLs, a heat statistics module cleaning mechanism is introduced in the embodiment: the method comprises the steps of simultaneously maintaining two bloom filters, a main filter master and a standby filter slave in a memory, inserting the two bloom filters each time, inquiring only the main filter master, cleaning after interval, deleting the master, replacing the master with the slave, and initializing one slave again. This ensures that the statistics of the master are always the number of accesses of [ interval, 2 × interval ] time length.
Since the mean value of the statistical data jumps after cleaning is performed, which may cause inaccuracy of heat statistics, a new heat statistics calculation formula is introduced to solve the problem: assuming that the current time from the last cleaning is now _ interval, the result of the number of times of URL visit queried by the master is K (if currently on the 3 rd cycle, K = the part of the 2 nd cycle + the 3 rd cycle), the formula of the return result final _ K is calculated as:
final_k = k * (2 * interval - now_interval) / interval
for the now _ interval in any value range of (0, interval), final _ k mean value mathematic expectation is unchanged, namely, the heat statistic mean value is expected to be unchanged. This ensures that the results returned to final _ k are not affected when clean-up and bloom filter replacement are performed. The hot degree counting module returns the hot degree value final _ k of the URL for the input URL. Final _ k is calculated one at a time per access. The number of nodes included in the first hash chain is determined according to final _ k, the set of nodes included in the first hash chain is a first node subset, and final _ k may be equal to the number of nodes included in the first hash chain.
Optionally, the determining, according to the first heat value, the first number of nodes corresponding to the first URL includes: setting the first number of nodes equal to the first heat value; or setting the first node number to be equal to the product of the first heat value and M, wherein M is a natural number greater than 1.
As an optional implementation manner, the number of nodes included in the hash chain corresponding to the URL is positively correlated with the access heat of the URL, and the higher the access of the URL, the more the number of nodes. The number of URL access times in the preset time length can be used as the number of nodes in the hash chain, and the multiple of the number of URL access times in the preset time length can also be used as the number of the hash chain summary nodes.
Optionally, the method further comprises: determining a second node subset in the target node set, wherein the remaining scheduling times of each node in the second node subset at the end of a second duration is greater than a predetermined time threshold; forming nodes in the second node subset into a second hash chain; acquiring a second URL to be accessed, which is sent by a second client; determining a second node quantity corresponding to the second URL according to the access times of the second URL in a third time length, wherein the second node quantity is the node quantity in a third node subset which is allowed to be scheduled for the second URL in a target node set; searching a second target node in the third node subset according to the sequence of a third hash chain, wherein the nodes in the third node subset form the third hash chain according to the hash value related to the second URL, and the remaining scheduling times of the second target node is greater than 0; and under the condition that the second target node cannot be found in the third node subset, finding a third target node in the second node subset according to the sequence of the second hash chain, wherein the residual scheduling times of the third target node are more than 0.
As an optional implementation manner, a node whose remaining scheduling number is greater than a predetermined number threshold may be selected as a node in the second node subset, where the predetermined number threshold may be determined according to an actual situation, for example, 10 times, 20 times, 50 times, and the like, and the nodes with a large remaining number are individually taken as a node set, and form the second hash chain. And under the condition of acquiring the URL to be accessed sent by the client, if the target node cannot be searched in the third hash chain corresponding to the URL to be accessed, searching the target node in the second hash chain.
As an optional implementation manner, a node with a small number of called times (a large number of remaining scheduling times) is used as a node set, and when a URL to be accessed is obtained, if a target node cannot be found in a hash chain corresponding to the URL to be accessed, the target node is found in the node set with the large number of remaining scheduling times, so that the problem of low node resource utilization rate caused by unbalanced node calling can be avoided. In an embodiment, assuming that the remaining scheduling times of the node A, B, C, D, E, F, G, H are 0, 1, 2, 3, 4, 5, 6, and 7, respectively, and the predetermined time threshold is 3, the node E, F, G, H with the remaining scheduling times greater than 3 is determined as the node with the smaller scheduling times, the node E, F, G, H is the second node subset, and the hash chain formed by the nodes E, F, G, H is used as the second hash chain. When a second URL to be accessed, which is sent by a client, is obtained, a third hash chain corresponding to the second URL is determined, and if the remaining scheduling times of the nodes in the third hash chain are all 0, a target node cannot be searched in the third hash chain. In this case, the destination node may be looked up in E, F, G, H to form a second hash chain.
As an alternative implementation, the node with the larger remaining scheduling times may be used as the low utilization node. Assuming that, in the ABCDEF node, if the BDF is a low-utilization node, hash chain generation is performed on the batch of low-utilization nodes once, and a low-utilization hash chain B- > F- > D can be obtained for the URL, as shown in fig. 11, which is a schematic diagram of a low-utilization hash chain according to an alternative embodiment of the present invention. Similarly, the under-utilized hash chains are also balanced and consistent for the under-utilized nodes. In this embodiment, since schedulable nodes are massive, there are some nodes that are repeatedly called and some nodes that are rarely called, so that the problems of unbalanced node calling and low node resource utilization rate are presented.
Optionally, the method further comprises: and when the current scheduling period is finished, setting the residual scheduling times of each node in the target node set as corresponding quota values, wherein the configuration values corresponding to different nodes in the target node set are the same, or the configuration values corresponding to at least 2 nodes in the target node set are different, and each node in the target node set is set to schedule for different URLs in the next scheduling period.
As an optional embodiment, the node may be scheduled in the scheduling period, and a scheduling quota may be allocated to the node outside the scheduling period of the node, where the scheduling quota is used to indicate the number of times that the node may be scheduled. The scheduling quotas allocated by different nodes may be the same, for example, the scheduling quotas of node a, node B, and node C are configured 10 times. Different nodes may also be allocated with different scheduling quotas, for example, the scheduling quotas of node a, node B, and node C are configured to 2, 4, 6, and 8 times, respectively. A node can be scheduled by different URLs, and the remaining scheduling times are reduced once each time the node is scheduled, until the remaining scheduling times are 0, the node cannot be scheduled.
Optionally, after the sending of the node information of the first target node to the first client, the method includes: and under the condition that the first client sends a first scheduling request to the first target node according to the node information, acquiring the resource which is sent by the first target node and is related to the first URL on the first client.
As an optional implementation manner, after the client acquires the node information of the target node, the user may acquire the relevant resource by sending a scheduling request to the target node, where the relevant resource may be a web page corresponding to the URL. For example, the user may send a URL of "www.abc.com" through the client, the server determines that the target node corresponding to the URL is node a, and sends node information of node a, which may be an identifier or an address of the node, to the client. The user can send a scheduling request to the node through the client, and the node A returns a webpage corresponding to 'www.abc.comde' to the client.
As an optional implementation manner, for the whole quota allocation, the quota obtaining logic diagram shown in fig. 12 may be used, and the hash chain itself is scheduled in the highest order, so that it is ensured that the quota allocation is always the first node to use the optimal one. And secondly, allotting quotas of the shared storage nodes, because the shared storage nodes can be regarded as different exit information of the same node. And finally, using a low-utilization-rate hash chain, namely scheduling the request which cannot be met by the current quota to the node with low resource utilization rate, and simultaneously ensuring the resource utilization rate and the flow recovery rate.
For any node, the node may be given a quota attribute that determines how many requests are allowed to be scheduled per scheduling period. Since the URL scheduled to a single node tends to be folded, the change of the average resource usage amount caused by the request scheduled to the node cannot be too severe for the node in a small time range, so that the load of the single node can be well controlled through quota control of the scheduling times, and the node resource is not wasted.
For the node quota, two states of a quick recovery mode and a maintenance scheduling mode are set, wherein for the quick recovery mode, when a new node is put on shelf and the node is forbidden to recover and use, estimation is carried out according to the resource holding capacity, a larger quota change step length is set, and personalized adjustment of the step length can be carried out according to the time for determining that the node is expected to use the complete resources. For the maintenance scheduling mode, the state occurs when the node runs stably, and at this time, the quota is slightly decreased under the condition of exceeding the expected resource utilization and slightly increased under the condition of being less than the expected resource utilization according to the resource utilization conditions of a plurality of previous scheduling periods. Therefore, the quota value is always maintained at a value which does not exceed the resource limit and is fully utilized in the maintenance scheduling mode. The mechanism ensures that the resource utilization rate is maintained at an expected level, meanwhile, an abnormal bottom prediction is set, and nodes with large fluctuation in resource utilization and quota increase and decrease are found and removed, so that the access quality is fully ensured under the condition of ensuring the resource utilization rate.
Optionally, in this embodiment, the terminal device may be a terminal device configured with a target client, and may include, but is not limited to, at least one of the following: mobile phones (such as Android phones, iOS phones, etc.), notebook computers, tablet computers, palm computers, MID (Mobile Internet Devices), PAD, desktop computers, smart televisions, etc. The target client may be a video client, an instant messaging client, a browser client, an educational client, etc. Such networks may include, but are not limited to: a wired network, a wireless network, wherein the wired network comprises: a local area network, a metropolitan area network, and a wide area network, the wireless network comprising: bluetooth, WIFI, and other networks that enable wireless communication. The server may be a single server, a server cluster composed of a plurality of servers, or a cloud server. The above is merely an example, and this is not limited in this embodiment.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.
According to another aspect of the embodiments of the present invention, there is also provided a node scheduling apparatus for implementing the above node scheduling method. As shown in fig. 13, the apparatus includes: a determining module 1302, configured to determine, according to access times of a first URL of a uniform resource locator to be accessed in a first duration, a first node number corresponding to the first URL, where the first node number is a number of nodes in a first node subset allowed in a target node set and scheduled for the first URL; a searching module 1304, configured to search a first target node in the first node subset according to a node arrangement order in a first hash chain, where the first hash chain includes the first node subset arranged according to the node arrangement order, the node arrangement order is determined according to a hash value related to the first URL, and a remaining scheduling number of the first target node is greater than 0; a sending module 1306, configured to send node information of the first target node to the first client if the first target node is found in the first node subset.
Optionally, the apparatus is further configured to search the first node subset for the first target node according to the node ranking order in the first hash chain by: repeatedly executing the following steps until the first target node is found, or traversing the first node subset, wherein an initial value of a current preset value is a first value, and each node in the target node set is provided with a different hash value corresponding to a first hash operation: executing the first hash operation on the first URL and the current preset value to obtain a current hash value; searching a node with a hash value being the current hash value in the target node set; determining the node with the hash value as the current hash value as the first target node under the condition that the node with the hash value as the current hash value is found and the remaining scheduling times of the node with the hash value as the current hash value are greater than 0; and under the condition that the node with the hash value as the current hash value is found and the residual scheduling times of the node with the hash value as the current hash value are equal to 0, increasing the current preset value by a preset value.
Optionally, the apparatus is further configured to perform the first hash operation on the first URL and the current preset value to obtain a current hash value in the following manner: executing the first hash operation on the sum of the first URL and the current preset value to obtain the current hash value; or splicing the first URL and the current preset value to obtain a first spliced character string; and executing the first hash operation on the first splicing character string to obtain the current hash value.
Optionally, the apparatus is further configured to repeatedly execute the following steps until i = the first number of nodes, where an initial value of the current preset value is a first value, and an initial value of i is 1: executing the first hash operation on the first URL and the current preset value to obtain a current hash value; searching a node with a hash value being the current hash value in the target node set; setting the node with the hash value as the current hash value as the ith node in the first hash chain under the condition that the node with the hash value as the current hash value is found; and under the condition that i is less than the first node number, i = i +1, and the current preset value is increased by a preset value.
Optionally, the apparatus is further configured to set one hash value of the N different hash values for each node of the N nodes when a hash result of the first hash operation includes N different hash values and the target node set includes N nodes, where the N different hash values have a one-to-one correspondence with the N nodes, and N is a natural number greater than 1.
Optionally, the apparatus is further configured to, when a hash result of the first hash operation includes N different hash values, the target node set includes M nodes, and M is greater than N, merge part of the M nodes into one virtual node to obtain N nodes in total, and set one hash value of the N different hash values for each node of the N nodes, where the N different hash values have a one-to-one correspondence relationship with the N nodes, N is a natural number greater than 1, and the N nodes include one or more virtual nodes; or under the condition that the hash result of the first hash operation includes N different hash values, the target node set includes M nodes, and M is smaller than N, splitting some of the M nodes into a plurality of shared nodes to obtain N nodes in total, and setting one of the N different hash values for each of the N nodes, where the N different hash values have a one-to-one correspondence with the N nodes, N is a natural number greater than 1, and the N nodes include a plurality of the shared nodes.
Optionally, the apparatus is further configured to determine, according to the number of times of access of a first URL of a uniform resource locator to be accessed in a first duration, a first node number corresponding to the first URL by: determining a first hot value of the first URL according to the access times of the first URL in the first time length; and determining the first node quantity corresponding to the first URL according to the first heat value, wherein the first node quantity is positively correlated with the first heat value.
Optionally, the determining a first popularity value of the first URL according to the number of times of access of the first URL in the first duration includes: setting the first heat value final _ k to be: final _ k = k (2 × interval-now _ interval)/interval; and k is the number of times of accessing the first URL in the first time length, interval is the time length of a preset statistical period, and now _ interval is the time length between the current time and the starting time of the last completed statistical period, and the first time length is now _ interval.
Optionally, the apparatus is further configured to determine, according to the first heat value, the first number of nodes corresponding to the first URL by: setting the first number of nodes equal to the first heat value; or setting the first node number to be equal to the product of the first heat value and M, wherein M is a natural number greater than 1.
Optionally, the apparatus is further configured to determine a second node subset in the target node set, where a remaining scheduling time of each node in the second node subset at the end of the second duration is greater than a predetermined time threshold; forming nodes in the second node subset into a second hash chain; acquiring a second URL to be accessed, which is sent by a second client; determining a second node quantity corresponding to the second URL according to the access times of the second URL in a third time length, wherein the second node quantity is the node quantity in a third node subset which is allowed to be scheduled for the second URL in a target node set; searching a second target node in the third node subset according to the sequence of a third hash chain, wherein the nodes in the third node subset form the third hash chain according to the hash value related to the second URL, and the remaining scheduling times of the second target node is greater than 0; and under the condition that the second target node cannot be found in the third node subset, finding a third target node in the second node subset according to the sequence of the second hash chain, wherein the residual scheduling times of the third target node are more than 0.
Optionally, the apparatus is further configured to set a corresponding quota value for the remaining scheduling frequency of each node in the target node set when the current scheduling period ends, where configuration values corresponding to different nodes in the target node set are all the same, or configuration values corresponding to at least 2 nodes in the target node set are different, and each node in the target node set is set to perform scheduling for a different URL in a next scheduling period.
Optionally, the apparatus is further configured to, after the sending of the node information of the first target node to the first client: and under the condition that the first client sends a first scheduling request to the first target node according to the node information, acquiring the resource which is sent by the first target node and is related to the first URL on the first client.
According to another aspect of the embodiment of the present invention, there is also provided an electronic device for implementing the node scheduling method, where the electronic device may be a terminal device or a server shown in fig. 1. The present embodiment takes the electronic device as an example of a node scheduling method. As shown in fig. 14, the electronic device comprises a memory 1402 and a processor 1404, the memory 1402 having stored therein a computer program, the processor 1404 being arranged to execute the steps of any of the method embodiments described above by means of the computer program.
Optionally, in this embodiment, the electronic device may be located in at least one network device of a plurality of network devices of a computer network.
Optionally, in this embodiment, the processor may be configured to execute the following steps by a computer program:
s1, determining a first node number corresponding to a first Uniform Resource Locator (URL) according to the access times of the first URL to be accessed in a first time length, wherein the first node number is the number of nodes in a first node subset which is allowed to be scheduled for the first URL in a target node set;
s2, searching for a first target node in the first node subset according to a node arrangement order in a first hash chain, where the first hash chain includes the first node subset arranged according to the node arrangement order, the node arrangement order is determined according to the hash value associated with the first URL, and the remaining scheduling number of the first target node is greater than 0;
s3, sending the node information of the first target node to the first client when the first target node is found in the first node subset.
Alternatively, it can be understood by those skilled in the art that the structure shown in fig. 14 is only an illustration, and the electronic device may also be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palmtop computer, a Mobile Internet Device (MID), a PAD, and the like. Fig. 14 is a diagram illustrating a structure of the electronic device. For example, the electronics may also include more or fewer components (e.g., network interfaces, etc.) than shown in FIG. 14, or have a different configuration than shown in FIG. 14.
The memory 1402 may be configured to store software programs and modules, such as program instructions/modules corresponding to the node scheduling method and apparatus in the embodiment of the present invention, and the processor 1404 executes various functional applications and data processing by running the software programs and modules stored in the memory 1402, that is, implementing the node scheduling method. Memory 1402 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 1402 may further include memory located remotely from the processor 1404, which may be connected to a terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof. The storage 1402 may be specifically, but not limited to, used for storing information such as sample characteristics of an item and a target virtual resource account number. As an example, as shown in fig. 14, the memory 1402 may include, but is not limited to, a determining module 1302, a searching module 1304, and a sending module 1306 in the node scheduling apparatus. In addition, the node scheduling apparatus may further include, but is not limited to, other module units in the node scheduling apparatus, which is not described in this example again.
Optionally, the transmitting device 1406 is used for receiving or sending data via a network. Examples of the network may include a wired network and a wireless network. In one example, the transmission device 1406 includes a Network adapter (NIC) that can be connected to a router via a Network cable and other Network devices to communicate with the internet or a local area Network. In one example, the transmitting device 1406 is a Radio Frequency (RF) module, which is used to communicate with the internet by wireless means.
In addition, the electronic device further includes: a connection bus 1408 for connecting the respective module components in the electronic apparatus is provided.
In other embodiments, the terminal device or the server may be a node in a distributed system, where the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting a plurality of nodes through a network communication. Nodes can form a Peer-To-Peer (P2P, Peer To Peer) network, and any type of computing device, such as a server, a terminal, and other electronic devices, can become a node in the blockchain system by joining the Peer-To-Peer network.
According to an aspect of the application, a computer program product or computer program is provided, comprising computer instructions, the computer instructions being stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to execute the node scheduling method provided in the above-mentioned various alternative implementations. Wherein the computer program is arranged to perform the steps of any of the above method embodiments when executed.
Alternatively, in the present embodiment, the above-mentioned computer-readable storage medium may be configured to store a computer program for executing the steps of:
s1, determining a first node number corresponding to a first Uniform Resource Locator (URL) according to the access times of the first URL to be accessed in a first time length, wherein the first node number is the number of nodes in a first node subset which is allowed to be scheduled for the first URL in a target node set;
s2, searching for a first target node in the first node subset according to a node arrangement order in a first hash chain, where the first hash chain includes the first node subset arranged according to the node arrangement order, the node arrangement order is determined according to the hash value associated with the first URL, and the remaining scheduling number of the first target node is greater than 0;
s3, sending the node information of the first target node to the first client when the first target node is found in the first node subset.
Alternatively, in this embodiment, a person skilled in the art may understand that all or part of the steps in the methods of the foregoing embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.
The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.
The integrated unit in the above embodiments, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in the above computer-readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing one or more computer devices (which may be personal computers, servers, network devices, etc.) to execute all or part of the steps of the method according to the embodiments of the present invention.
In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed client may be implemented in other manners. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one type of division of logical functions, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims (14)

1. A node scheduling method, comprising:
determining a first node quantity corresponding to a first Uniform Resource Locator (URL) according to the access times of the first URL to be accessed in a first time length, wherein the first node quantity is the node quantity in a first node subset which is allowed to be scheduled for the first URL in a target node set;
searching a first target node in the first node subset according to a node arrangement sequence in a first hash chain, wherein the first hash chain comprises the first node subset arranged according to the node arrangement sequence, the node arrangement sequence is determined according to a hash value related to the first URL, and the remaining scheduling times of the first target node are greater than 0;
under the condition that the first target node is found in the first node subset, sending node information of the first target node to a first client;
determining a second node subset in the target node set, wherein the remaining scheduling times of each node in the second node subset at the end of a second duration is greater than a predetermined time threshold;
forming nodes in the second node subset into a second hash chain;
acquiring a second URL to be accessed, which is sent by a second client;
determining a second node quantity corresponding to the second URL according to the access times of the second URL in a third time length, wherein the second node quantity is the node quantity in a third node subset which is allowed to be scheduled for the second URL in a target node set;
searching a second target node in the third node subset according to the sequence of a third hash chain, wherein the nodes in the third node subset form the third hash chain according to the hash value related to the second URL, and the remaining scheduling times of the second target node is greater than 0;
and under the condition that the second target node cannot be found in the third node subset, finding a third target node in the second node subset according to the sequence of the second hash chain, wherein the residual scheduling times of the third target node are more than 0.
2. The method of claim 1, wherein searching for the first destination node in the first subset of nodes in the order of arrangement of nodes in the first hash chain comprises:
repeatedly executing the following steps until the first target node is found, or traversing the first node subset, wherein an initial value of a current preset value is a first value, and each node in the target node set is provided with a different hash value corresponding to a first hash operation:
executing the first hash operation on the first URL and the current preset value to obtain a current hash value;
searching a node with a hash value being the current hash value in the target node set;
determining the node with the hash value as the current hash value as the first target node under the condition that the node with the hash value as the current hash value is found and the remaining scheduling times of the node with the hash value as the current hash value are greater than 0;
and under the condition that the node with the hash value as the current hash value is found and the residual scheduling times of the node with the hash value as the current hash value are equal to 0, increasing the current preset value by a preset value.
3. The method of claim 2, wherein the performing the first hash operation on the first URL and the current preset value to obtain a current hash value comprises:
executing the first hash operation on the sum of the first URL and the current preset value to obtain the current hash value; or
Splicing the first URL and the current preset value to obtain a first spliced character string; and executing the first hash operation on the first splicing character string to obtain the current hash value.
4. The method of claim 2, further comprising:
repeatedly executing the following steps until i = the first node number, wherein the initial value of the current preset value is a first value, and the initial value of i is 1:
executing the first hash operation on the first URL and the current preset value to obtain a current hash value;
searching a node with a hash value being the current hash value in the target node set;
setting the node with the hash value as the current hash value as the ith node in the first hash chain under the condition that the node with the hash value as the current hash value is found;
and under the condition that i is less than the first node number, i = i +1, and the current preset value is increased by a preset value.
5. The method of claim 2, further comprising:
and under the condition that the hash result of the first hash operation comprises N different hash values and the target node set comprises N nodes, setting one hash value of the N different hash values for each node of the N nodes, wherein the N different hash values have a one-to-one correspondence relationship with the N nodes, and N is a natural number greater than 1.
6. The method of claim 2, further comprising:
combining part of the M nodes into a virtual node under the condition that a hash result of the first hash operation comprises N different hash values, the target node set comprises M nodes and M is greater than N, so as to obtain N nodes in total, and setting one hash value of the N different hash values for each node of the N nodes, wherein the N different hash values have a one-to-one correspondence relationship with the N nodes, N is a natural number greater than 1, and the N nodes comprise one or more virtual nodes; or
When the hash result of the first hash operation includes N different hash values, the target node set includes M nodes, and M is smaller than N, splitting some of the M nodes into a plurality of shared nodes to obtain N nodes in total, and setting one of the N different hash values for each of the N nodes, where the N different hash values have a one-to-one correspondence with the N nodes, N is a natural number greater than 1, and the N nodes include a plurality of the shared nodes.
7. The method of claim 1, wherein the determining, according to the number of times of access of a Uniform Resource Locator (URL) to be accessed in a first time period, the number of first nodes corresponding to the first URL comprises:
determining a first hot value of the first URL according to the access times of the first URL in the first time length;
and determining the first node quantity corresponding to the first URL according to the first heat value, wherein the first node quantity is positively correlated with the first heat value.
8. The method of claim 7, wherein determining the first popularity value for the first URL based on the number of visits the first URL made over the first duration comprises:
setting the first heat value final _ k to be:
final_k = k * (2 * interval - now_interval) / interval;
and k is the number of times of accessing the first URL in the first time length, interval is the time length of a preset statistical period, and now _ interval is the time length between the current time and the starting time of the last completed statistical period, and the first time length is now _ interval.
9. The method of claim 7, wherein determining the first number of nodes corresponding to the first URL according to the first popularity value comprises:
setting the first number of nodes equal to the first heat value; or
Setting the first number of nodes equal to a product of the first heat value and M, wherein M is a natural number greater than 1.
10. The method according to any one of claims 1 to 9, further comprising:
and when the current scheduling period is finished, setting the residual scheduling times of each node in the target node set as corresponding quota values, wherein the configuration values corresponding to different nodes in the target node set are the same, or the configuration values corresponding to at least 2 nodes in the target node set are different, and each node in the target node set is set to schedule for different URLs in the next scheduling period.
11. The method according to any of claims 1 to 9, wherein after said sending node information of the first target node to the first client, the method comprises:
and under the condition that the first client sends a first scheduling request to the first target node according to the node information, acquiring the resource which is sent by the first target node and is related to the first URL on the first client.
12. A node scheduling apparatus, comprising:
the determining module is used for determining a first node number corresponding to a first Uniform Resource Locator (URL) according to the access times of the first URL to be accessed in a first time length, wherein the first node number is the number of nodes in a first node subset which is allowed to be scheduled for the first URL in a target node set;
a searching module, configured to search a first target node in the first node subset according to a node arrangement order in a first hash chain, where the first hash chain includes the first node subset arranged according to the node arrangement order, the node arrangement order is determined according to a hash value related to the first URL, and a remaining scheduling number of the first target node is greater than 0;
a sending module, configured to send node information of the first target node to a first client when the first target node is found in the first node subset;
the apparatus is further configured to determine a second subset of nodes in the target set of nodes, wherein each node in the second subset of nodes has a remaining number of schedules at the end of a second duration that is greater than a predetermined threshold number of times; forming nodes in the second node subset into a second hash chain; acquiring a second URL to be accessed, which is sent by a second client; determining a second node quantity corresponding to the second URL according to the access times of the second URL in a third time length, wherein the second node quantity is the node quantity in a third node subset which is allowed to be scheduled for the second URL in a target node set; searching a second target node in the third node subset according to the sequence of a third hash chain, wherein the nodes in the third node subset form the third hash chain according to the hash value related to the second URL, and the remaining scheduling times of the second target node is greater than 0; and under the condition that the second target node cannot be found in the third node subset, finding a third target node in the second node subset according to the sequence of the second hash chain, wherein the residual scheduling times of the third target node are more than 0.
13. A computer-readable storage medium comprising a stored program, wherein the program when executed performs the method of any of claims 1 to 11.
14. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to execute the method of any of claims 1 to 11 by means of the computer program.
CN202010950849.0A 2020-09-11 2020-09-11 Node scheduling method and device Active CN111935306B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010950849.0A CN111935306B (en) 2020-09-11 2020-09-11 Node scheduling method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010950849.0A CN111935306B (en) 2020-09-11 2020-09-11 Node scheduling method and device

Publications (2)

Publication Number Publication Date
CN111935306A CN111935306A (en) 2020-11-13
CN111935306B true CN111935306B (en) 2020-12-29

Family

ID=73310100

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010950849.0A Active CN111935306B (en) 2020-09-11 2020-09-11 Node scheduling method and device

Country Status (1)

Country Link
CN (1) CN111935306B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112732201A (en) * 2021-02-26 2021-04-30 邹威 Block chain based data storage configuration method and block chain cloud platform
CN114356587B (en) * 2022-03-17 2022-07-05 梯度云科技(北京)有限公司 Calculation power task cross-region scheduling method, system and equipment
CN114579582B (en) * 2022-04-29 2022-08-23 武汉北大高科软件股份有限公司 Resource processing method and device based on block chain

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103428008A (en) * 2013-08-28 2013-12-04 浙江大学 Big data distribution strategy oriented to multiple user groups
CN105844121A (en) * 2016-03-31 2016-08-10 乐视控股(北京)有限公司 Method and system for applying digital watermark to content delivery network (CDN)

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102263828B (en) * 2011-08-24 2013-08-07 北京蓝汛通信技术有限责任公司 Load balanced sharing method and equipment
CN104852934A (en) * 2014-02-13 2015-08-19 阿里巴巴集团控股有限公司 Method for realizing flow distribution based on front-end scheduling, device and system thereof
CN107801086B (en) * 2017-10-20 2019-01-04 广东省南方数字电视无线传播有限公司 The dispatching method and system of more cache servers
CN109951543A (en) * 2019-03-14 2019-06-28 网宿科技股份有限公司 A kind of data search method of CDN node, device and the network equipment
CN110290196B (en) * 2019-06-20 2021-08-27 网宿科技股份有限公司 Content distribution method and device

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103428008A (en) * 2013-08-28 2013-12-04 浙江大学 Big data distribution strategy oriented to multiple user groups
CN105844121A (en) * 2016-03-31 2016-08-10 乐视控股(北京)有限公司 Method and system for applying digital watermark to content delivery network (CDN)

Also Published As

Publication number Publication date
CN111935306A (en) 2020-11-13

Similar Documents

Publication Publication Date Title
CN111935306B (en) Node scheduling method and device
CN111522641B (en) Task scheduling method, device, computer equipment and storage medium
US11102290B2 (en) Peer-to-peer network prioritizing propagation of objects through the network
CN102882985A (en) File sharing method based on cloud storage
CN110113399A (en) Load balancing management method and relevant apparatus
CN105608117A (en) Information recommendation method and apparatus
CN103581207A (en) Cloud terminal data storage system and data storing and sharing method based on cloud terminal data storage system
CN101840418A (en) User word library synchronous update method, update server and input method system
CN106790552B (en) A kind of content providing system based on content distributing network
CN110413845B (en) Resource storage method and device based on Internet of things operating system
CN110532493B (en) Data processing method and device, storage medium and electronic device
CN104410660A (en) Method and device of downloading multimedia resources as well as router and server
CN101068173B (en) Resource sharing method and system, and server
Setty et al. Cost-effective resource allocation for deploying pub/sub on cloud
CN110035306A (en) Dispositions method and device, the dispatching method and device of file
CN111556135A (en) Request scheduling method, system and device and electronic equipment
CN112468589A (en) Data distribution method and device, computer equipment and storage medium
CN102882960A (en) Method and device for transmitting resource files
CN111400301A (en) Data query method, device and equipment
CN113382065A (en) File downloading method, device, system, equipment and storage medium
KR100876105B1 (en) How to download multimedia contents to wired and wireless terminals
CN112307058A (en) Short link processing method and device, storage medium and computer equipment
CN106850707A (en) Method, apparatus and system that material batch is uploaded
CN115086194A (en) Data transmission method for cloud application, computing equipment and computer storage medium
Haridas et al. Cloudy knapsack algorithm for offloading tasks from large scale distributed applications

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant