CN113689103A - Adaptive load balancing employing flow distribution intelligent scheduling management method, device and system - Google Patents
Adaptive load balancing employing flow distribution intelligent scheduling management method, device and system Download PDFInfo
- Publication number
- CN113689103A CN113689103A CN202110949110.2A CN202110949110A CN113689103A CN 113689103 A CN113689103 A CN 113689103A CN 202110949110 A CN202110949110 A CN 202110949110A CN 113689103 A CN113689103 A CN 113689103A
- Authority
- CN
- China
- Prior art keywords
- server
- load balancing
- buyer
- seller
- segment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000009826 distribution Methods 0.000 title claims abstract description 28
- 230000003044 adaptive effect Effects 0.000 title claims abstract description 15
- 238000007726 management method Methods 0.000 title claims description 22
- 238000000034 method Methods 0.000 claims abstract description 35
- 238000013507 mapping Methods 0.000 claims abstract description 17
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 14
- 238000003860 storage Methods 0.000 claims description 14
- 230000011218 segmentation Effects 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 9
- 230000009471 action Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 8
- 238000004590 computer program Methods 0.000 description 7
- 230000005611 electricity Effects 0.000 description 5
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q10/06312—Adjustment or analysis of established resource schedule, e.g. resource or task levelling, or dynamic rescheduling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5083—Techniques for rebalancing the load in a distributed system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q10/06313—Resource planning in a project environment
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Human Resources & Organizations (AREA)
- Economics (AREA)
- Theoretical Computer Science (AREA)
- Strategic Management (AREA)
- General Physics & Mathematics (AREA)
- Entrepreneurship & Innovation (AREA)
- Physics & Mathematics (AREA)
- Tourism & Hospitality (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- Game Theory and Decision Science (AREA)
- Health & Medical Sciences (AREA)
- Operations Research (AREA)
- Educational Administration (AREA)
- Development Economics (AREA)
- Software Systems (AREA)
- Quality & Reliability (AREA)
- Biodiversity & Conservation Biology (AREA)
- Public Health (AREA)
- Water Supply & Treatment (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a method, a device and a system for intelligent scheduling management of adaptive load balancing by adopting shunting, wherein the method comprises the steps of obtaining a consistent hash ring, wherein the consistent hash ring is positioned between a user and a server and comprises a plurality of virtual nodes, and mappings are respectively arranged between the virtual nodes and the user as well as between the virtual nodes and the server; based on a self-adaptive load balancing principle, segmenting a consistent hash ring, and distributing and modeling all virtual nodes into a buyer-seller game model; and solving the buyer-seller game model through a greedy algorithm to determine a final server distribution mode. The invention can obtain a relatively excellent server distribution scheme and provides a certain reference for the power acquisition scheme of a power grid company.
Description
Technical Field
The invention belongs to the crossing field of power transmission and distribution technologies and information science, and particularly relates to a method, a device and a system for intelligent scheduling management of flow distribution for self-adaptive load balancing.
Background
In recent years, the progress and the internal demand of science and technology are increased, the living standard of people is steadily improved, and the electricity consumption of residents is increased day by day. The increasing expansion of power data puts tremendous operational pressure on power data acquisition systems. With the development of new technologies such as big data and cloud computing, new changes are brought to the power system. How to build a sampling and shunting system which can stably operate under the conditions of high load and large amount of data is particularly important. The adoption and distribution refers to the collection of the power utilization information of the user and the distributed processing of the collected data on a plurality of computing nodes.
At present, an electricity utilization information acquisition system becomes an important component of an intelligent power grid, supports important data requirements of a plurality of professional business applications such as power transaction, electricity charge recovery, electricity utilization verification, line loss lean analysis, power grid operation monitoring, power supply quality monitoring and fault first-aid repair and becomes an important basis and core data source for power company production and operation decision capability analysis. The existing power utilization information acquisition system has various bottlenecks of poor expansibility, poor flexibility, poor compatibility and the like on the basis of construction and a traditional IOE architecture, and cannot adapt to data processing in a big data environment. In the face of ever-increasing data acquisition and data analysis demands, existing architectures have been unable to support the development of future power consumption information acquisition systems.
Disclosure of Invention
In order to solve the problems, the invention provides a method, a device and a system for intelligent scheduling management of flow distribution for self-adaptive load balancing, which can realize stable operation of power data acquisition under high load.
In order to achieve the technical purpose and achieve the technical effects, the invention is realized by the following technical scheme:
in a first aspect, the present invention provides a method for intelligent scheduling and management of sampling and offloading for adaptive load balancing, including:
the method comprises the steps of obtaining a consistent hash ring, wherein the consistent hash ring is positioned between a user and a server and comprises a plurality of virtual nodes, and mappings are respectively arranged among the virtual nodes, the user and the server;
based on a self-adaptive load balancing principle, segmenting a consistent hash ring, and distributing and modeling all virtual nodes into a buyer-seller game model;
and solving the buyer-seller game model through a greedy algorithm to determine a final server distribution mode.
Optionally, a mapping created based on the consistent hash is provided between the virtual node and the user.
Optionally, a mapping created by a modulo operation is provided between the virtual node and the server.
Optionally, the segmenting the consistent hash ring specifically includes:
based on the proportion of the section i in the total load of the system, the upper limit threshold value T is setupAnd a lower threshold value TlowDefined as a function of the total load L;
if fi≤TlowxXL, combining the section i with the section with lower load in the left section or the right section;
if fi>TupxL, dividing the segment i into two equal halves, and distributing a father node for the two halves;
upper limit threshold T of operationupAnd a lower threshold value TlowAnd when the threshold value between the two meets the set requirement, the merging or the segmentation action does not occur any more, and the optimal segmentation point is obtained.
Optionally, the buyer-seller gaming model is constructed by the following method:
defining the server j to be distributed to the segment i, and the obtained profit is pij；
Each segment having a data volume wiEach server has a maximum capacity cjAnd with the goal of maximizing profit, obtaining a buyer-seller game model:
xij∈0,1
wherein x isijIndicating whether server j is assigned to segment i.
Alternatively, said pijThe calculation formula of (2) is as follows:
pij＝fi×Tj
wherein f isiRepresents the proportion of segment i in the total system load, TjIndicating that server j maintains an average throughput for one query.
Optionally, the buyer-seller game model is solved through a greedy algorithm to determine a final server allocation mode, specifically:
solving the buyer-seller game model by a greedy algorithm, and calculating profit P obtained by using each section of unit data amount for each bidij，
Will calculate the profit PijSorted in descending order and server allocation is performed according to this order.
Optionally, the allocating the servers according to the sequence specifically includes:
each server is traversed and a corresponding segment is assigned to the server as long as the server has space and the segment is not assigned to any other server.
In a second aspect, the present invention provides a device for intelligent scheduling and managing distribution for adaptive load balancing, including:
the system comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a consistent hash ring, the consistent hash ring is positioned between a user and a server and comprises a plurality of virtual nodes, and mappings are respectively arranged among the virtual nodes, the user and the server;
the modeling module is used for segmenting the consistent hash ring based on a self-adaptive load balancing principle and modeling all virtual node distribution as a buyer-seller game model;
and the management module is used for solving the buyer-seller game model through a greedy algorithm to determine a final server distribution mode.
In a third aspect, the present invention provides a self-adaptive load balancing employing offloading intelligent scheduling management system, including a storage medium and a processor;
the storage medium is used for storing instructions;
the processor is configured to operate in accordance with the instructions to perform the method according to any one of the first aspects.
Compared with the prior art, the invention has the beneficial effects that:
the invention can realize the stable operation of power data acquisition under high load. Firstly, a server with low cost is used for building a user electricity utilization information acquisition system of a million level through a consistent hash; secondly, a new virtual node partitioning method based on a virtual layer is provided; secondly, the whole virtual node distribution is modeled into a multi-knapsack problem of a buyer-seller game by determining the server bid amount and dividing the hash sections; and finally, solving the constraint problem by using a greedy algorithm to obtain a relatively excellent server distribution scheme, wherein the obtained result can provide a certain reference for the power acquisition scheme of the power grid company.
Drawings
In order that the present disclosure may be more readily and clearly understood, reference is now made to the following detailed description of the present disclosure taken in conjunction with the accompanying drawings, in which:
fig. 1 is a schematic flowchart of a method for intelligent scheduling management by offloading for adaptive load balancing according to an embodiment of the present invention. (ii) a
Fig. 2 is a diagram of a consistent hash ring and virtual nodes according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of the invention.
The following detailed description of the principles of the invention is provided in connection with the accompanying drawings.
Example 1
It is very important to achieve load balancing of the microservice architecture because it is not known how many users will be accessing. Assuming that one service simultaneously handles a user request with an upper limit of 1000, 20 services are allocated to 10000 users to provide the service. When there are 20000 users making requests, new services, some of which are overloaded and some of which are relatively idle, will have to be opened to handle the newly added requests.
The present invention selects consistent hashes as the hash function. Consistent hash functions, however, have some drawbacks. Consistent hashing works best in an environment of over 1000 virtual nodes, while many applications typically have too few slice keys to use consistent hashing to solve the load balancing problem, and improvements are needed to compensate for this deficiency.
The virtual node is a concept widely applied to engineering scenes, and a plurality of network connections are established on one physical host in a network mapping mode. By using the virtual nodes, the hash length traversed by searching the target node after the request of the user is mapped on the hash ring is shortened, so that the distribution efficiency is improved.
The basic principle of the consistent hashing method is to abstract a storage space into 232And the circle of the nodes distributes the storage nodes to the circle, and the nodes on the ring all have a hash value. The hash value of the key storing the data is found in the same way and is mapped to the circle in the same way. And then looking up clockwise from the position to which the data is mapped, and storing the data on the 1 st found server.
In the distributed architecture scenario of the present invention, the key step of consistent hashing is to map each server (node) to a point on the edge of a circle, but if the number of nodes is not enough, the distribution will be uneven, which may cause a problem of uneven load. The method for solving the problem is to add a virtual layer between the user and the server, wherein the hash ring of the virtual layer is shown in fig. 2, and the virtual layer is composed of a large number of virtual nodes. The mapping between the user and the virtual node is created by using the consistent hash, the mapping between the virtual node and the server is created by simple modulo operation, and the consistent hash operation is performed by enough virtual nodes through the method. The number of the virtual nodes is also important, and if the number of the virtual nodes is too small, the load balancing effect of the original consistent hash method is not obvious; otherwise, the time required for finding a virtual node corresponding to the user is too long.
The consistent hash ring is divided into a plurality of segments, each segment being assigned to one virtual node, and wherein there is a mapping from the virtual node to the physical node. In practical applications, multiple virtual nodes may be assigned to a single physical node. Different physical nodes have different specifications and therefore different capabilities. Obviously, it is the most beneficial allocation method to map the most loaded consistency hash segments to the most powerful servers in sequence, but the load of any consistency hash segment in a dynamic system is constantly changing. In such a case, reassigning the entire segment to a new physical server can be a costly operation. But under certain conditions it may be beneficial to split the hash segments and provide them with a new virtual server. If a segment is heavily loaded from a small range, meaning that there is a small set of keys that are repeatedly queried or used, dividing the segment into smaller segments and allocating the smaller segments that will be queried multiple times to a powerful server, while ensuring minimal data movement, will improve the overall performance of the system. But in the process, the number of segments is increased in the consistency hash ring, and how to track all the segments in the distributed environment becomes a new problem. It is therefore desirable to find an optimal way to partition it, which should ensure that the size of the partitioned segments is small enough so that there is not too much overhead in transmitting the data of the segments, while keeping the number of segments to a minimum. The entire virtual node allocation is modeled as a buyer-seller game. We have a buyer, i.e. a segment of the consistency hash ring, and a seller, i.e. a server. The buyer needs to purchase a server so that it can allocate the server for each segment's load. The buyer pays the cost of the server as a price. The buyer quotes all the servers according to the performance of the servers, and the servers have a plurality of quotes to choose from under the condition of meeting the space requirement.
Specifically, as shown in fig. 1, an embodiment of the present invention provides a method for intelligent scheduling management of adaptive load balancing employing offloading, which specifically includes the following steps:
the method comprises the following steps that (1) a consistent hash ring is obtained, wherein the consistent hash ring is located between a user and a server and comprises a plurality of virtual nodes, and mappings are respectively arranged among the virtual nodes, the user and the server; specifically, a mapping created based on consistent hashing is set between the virtual node and the user; a mapping created through a modulus taking operation is arranged between the virtual node and the server;
based on a self-adaptive load balancing principle, segmenting a consistent hash ring, and modeling all virtual node distribution into a buyer-seller game model;
and (3) solving the buyer-seller game model through a greedy algorithm to determine a final server distribution mode.
In one embodiment of the present invention, it is desirable that a partitioned segment be small enough that the overhead of moving segment data is minimal. But at the same time it is necessary to ensure that a segment is not too small because too small a segment will serve too little traffic, thereby increasing the burden of tracking this segment. For this purpose, an optimal segmentation point needs to be defined, and the segmenting the consistent hash ring comprises the following steps:
based on the proportion of the section i in the total load of the system, the upper limit threshold value T is setupAnd a lower threshold value TlowDefined as a function of the total load L;
if fi≤TlowXl, which means the load is too low to merit tracking this part, so it needs to be merged, segment i is merged with the lower loaded one of the left or right segments (i.e. it is chosen to be merged with the left or right segment, both as alternatives, and whichever segment is lower, it is merged, if it is merged with the segment with higher traffic, the merged segment will be split in the next iteration);
if fi>TupXl, which means that the load is too high, such a segment is at a higher risk and expensive to migrate, segment i is split into two equal halves, and a parent node is assigned to both halves;
upper limit threshold T of operationupAnd a lower threshold value TlowThe number of segments in the consistent hash ring can be managed to obtain the optimal segmentation point. Utensil for cleaning buttockBody ground, lowering TupWhile maintaining TlowInvariance leads to frequent segmentation, which means that the number of segments increases until the system remains stable. Similarly, T is increasedupFewer segmentation operations result, which means that the number of segments is reduced until the system stabilizes. Similarly, if T is increased or decreasedlowThe opposite effect will result. Difference between two thresholds Tup-TlowDetermining the stability of the system. If this difference is small, merging and splitting operations occur frequently, wasting system resources), merging or splitting actions no longer occur, and the best segmentation point is obtained.
In the course of the specific implementation,
in a specific implementation manner of the embodiment of the present invention, the buyer-seller game model is constructed by the following method:
defining the server j to be distributed to the segment i, and the obtained profit is pij；
Each segment having a data volume wiEach server has a maximum capacity cjAnd with the goal of maximizing profit, obtaining a buyer-seller game model:
xij∈0,1
wherein x isijIndicating whether server j is assigned to segment i.
pijIn fact, the price paid for server j by segment i is based on:
1) if the ith segment is assigned to a serverAmount of data L that the server must loadi；
2) Server j may maintain an average throughput T of one queryj；
Other factors such as latency, power utilization, etc. may also be considered, but are limited in these two respects in the present invention for testing purposes. If absolute loads are used, the defined constants and thresholds must be changed to compensate for variations in the value range. Thus, in order to make it easier to manage, a normalized representation of the load is required, such as the proportion of the load that the system serves. The lowest value that a segment can bid on is 0 and is the default value for all bids. Corresponding mathematical models are established according to the two standards. Each segment i gets a fraction of the total load of all systems, using fiServer quote p indicating fraction of total system load for segment i, and therefore segment i is willing to payijThe calculation formula of (2) is as follows:
pij＝fi×Tj
wherein f isiRepresents the proportion of segment i in the total system load, TjIndicating that server j maintains an average throughput for one query.
In a specific implementation manner of the embodiment of the present invention, the solving the buyer-seller game model through a greedy algorithm to determine a final server allocation manner specifically includes:
solving the buyer-seller game model by a greedy algorithm, and calculating profit P obtained by using each section of unit data amount for each bidij，
Will calculate the profit PijSorting in descending order and distributing servers according to the order; the server allocation according to this sequence specifically includes: each server is traversed and a corresponding segment is assigned to the server as long as the server has space and the segment is not assigned to any other server.
Example 2
Based on the same inventive concept as embodiment 1, the embodiment of the present invention provides a flow distribution intelligent scheduling management device for adaptive load balancing, including:
the system comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a consistent hash ring, the consistent hash ring is positioned between a user and a server and comprises a plurality of virtual nodes, and mappings are respectively arranged among the virtual nodes, the user and the server;
the modeling module is used for segmenting the consistent hash ring based on a self-adaptive load balancing principle and modeling all virtual node distribution as a buyer-seller game model;
and the management module is used for solving the buyer-seller game model through a greedy algorithm to determine a final server distribution mode.
The rest of the process was the same as in example 1.
Example 3
Based on the same inventive concept as the embodiment 1, the embodiment of the invention provides a self-adaptive load balancing flow distribution intelligent scheduling management system, which comprises a storage medium and a processor;
the storage medium is used for storing instructions;
the processor is configured to operate in accordance with the instructions to perform the method of any of embodiment 1.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting the same, and although the present invention is described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications and equivalents may be made to the embodiments of the invention without departing from the spirit and scope of the invention, which is to be covered by the claims.
Claims (10)
1. A method for intelligent scheduling management of flow distribution for adaptive load balancing is characterized by comprising the following steps:
the method comprises the steps of obtaining a consistent hash ring, wherein the consistent hash ring is positioned between a user and a server and comprises a plurality of virtual nodes, and mappings are respectively arranged among the virtual nodes, the user and the server;
based on a self-adaptive load balancing principle, segmenting a consistent hash ring, and distributing and modeling all virtual nodes into a buyer-seller game model;
and solving the buyer-seller game model through a greedy algorithm to determine a final server distribution mode.
2. The method for intelligent scheduling management of adoption and offloading for adaptive load balancing according to claim 1, characterized in that: the mapping between the virtual node and the user is created based on the consistent hash.
3. The method for intelligent scheduling management of adoption and offloading for adaptive load balancing according to claim 1, characterized in that: and a mapping created by a modulo operation is arranged between the virtual node and the server.
4. The method for intelligent scheduling management of adoption and offloading for adaptive load balancing according to claim 1, characterized in that: the segmentation of the consistent hash ring specifically comprises the following steps:
based on the proportion of the section i in the total load of the system, the upper limit threshold value T is setupAnd a lower threshold value TlowDefined as a function of the total load L;
if fi≤TlowxXL, combining the section i with the section with lower load in the left section or the right section;
if fi＞TupxL, dividing the segment i into two equal halves, and distributing a father node for the two halves;
upper limit threshold T of operationupAnd a lower threshold value TlowAnd when the threshold value between the two meets the set requirement, the merging or the segmentation action does not occur any more, and the optimal segmentation point is obtained.
5. The method for intelligent scheduling management of adoption and offloading for adaptive load balancing according to claim 4, wherein: the buyer-seller game model is constructed by the following method:
defining the server j to be distributed to the segment i, and the obtained profit is pij；
Each segment having a data volume wiEach server has a maximum capacity cjAnd with the goal of maximizing profit, obtaining a buyer-seller game model:
xij∈0，1
wherein x isijIndicating whether server j is assigned to segment i.
6. The adaptive load balancing intelligent scheduling management method with flow distribution according to claim 5, wherein p isijThe calculation formula of (2) is as follows:
pij＝fi×Tj
wherein f isiRepresents the proportion of segment i in the total system load, TjIndicating that server j maintains an average throughput for one query.
7. The method for intelligent scheduling and management of flow distribution for adaptive load balancing according to claim 5 or 6, wherein the buyer-seller game model is solved through a greedy algorithm to determine a final server allocation mode, specifically:
solving the buyer-seller game model by greedy algorithm, and calculating each section of using list for each bidProfit P obtained by bit data amountij，
Will calculate the profit PijSorted in descending order and server allocation is performed according to this order.
8. The method according to claim 7, wherein the method comprises the following steps: the server allocation according to this sequence specifically includes:
each server is traversed and a corresponding segment is assigned to the server as long as the server has space and the segment is not assigned to any other server.
9. The utility model provides a self-adaptation load balancing is with adopting reposition of redundant personnel intelligent scheduling management device which characterized in that includes:
the system comprises an acquisition module, a storage module and a processing module, wherein the acquisition module is used for acquiring a consistent hash ring, the consistent hash ring is positioned between a user and a server and comprises a plurality of virtual nodes, and mappings are respectively arranged among the virtual nodes, the user and the server;
the modeling module is used for segmenting the consistent hash ring based on a self-adaptive load balancing principle and modeling all virtual node distribution as a buyer-seller game model;
and the management module is used for solving the buyer-seller game model through a greedy algorithm to determine a final server distribution mode.
10. The utility model provides a self-adaptation load balancing is with adopting reposition of redundant personnel intelligent scheduling management system which characterized in that: comprising a storage medium and a processor;
the storage medium is used for storing instructions;
the processor is configured to operate in accordance with the instructions to perform the method of any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110949110.2A CN113689103B (en) | 2021-08-18 | 2021-08-18 | Mining and shunting intelligent scheduling management method, device and system for self-adaptive load balancing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110949110.2A CN113689103B (en) | 2021-08-18 | 2021-08-18 | Mining and shunting intelligent scheduling management method, device and system for self-adaptive load balancing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113689103A true CN113689103A (en) | 2021-11-23 |
| CN113689103B CN113689103B (en) | 2023-11-24 |
Family
ID=78580392
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110949110.2A Active CN113689103B (en) | 2021-08-18 | 2021-08-18 | Mining and shunting intelligent scheduling management method, device and system for self-adaptive load balancing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113689103B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107918559A (en) * | 2016-10-08 | 2018-04-17 | 腾讯科技（深圳）有限公司 | A kind of data processing method and device |
| CN108551474A (en) * | 2018-03-26 | 2018-09-18 | 南京邮电大学 | A kind of load-balancing method of server cluster |
| CN108733475A (en) * | 2018-05-22 | 2018-11-02 | 哈尔滨工业大学（威海） | A kind of dynamical feedback dispatching method |
| CN109472520A (en) * | 2018-12-21 | 2019-03-15 | 南京邮电大学 | Multiple no-manned plane method for allocating tasks based on block chain |
| US20190268405A1 (en) * | 2018-02-28 | 2019-08-29 | International Business Machines Corporation | Load balancing with power of random choices |
| CN110213351A (en) * | 2019-05-17 | 2019-09-06 | 北京航空航天大学 | A kind of dynamic self-adapting I/O load equalization methods towards wide area high-performance computing environment |
| CN111083232A (en) * | 2019-12-27 | 2020-04-28 | 南京邮电大学 | Server-side load balancing method based on improved consistent hash |
| CN111177154A (en) * | 2019-12-27 | 2020-05-19 | 掌迅亿通(北京)信息科技有限公司 | Distributed database caching method and hash ring optimization thereof |
| CN111209120A (en) * | 2020-01-09 | 2020-05-29 | 平安科技（深圳）有限公司 | Data synchronization method and device for microservice and computer readable storage medium |
| WO2021003935A1 (en) * | 2019-07-11 | 2021-01-14 | 平安科技（深圳）有限公司 | Data cluster storage method and apparatus, and computer device |
| CN112487560A (en) * | 2020-12-11 | 2021-03-12 | 国网江苏省电力有限公司南通供电分公司 | Electric power traffic coupling system coordinated game scheduling method based on EV vehicle owner intention |
-
2021
- 2021-08-18 CN CN202110949110.2A patent/CN113689103B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107918559A (en) * | 2016-10-08 | 2018-04-17 | 腾讯科技（深圳）有限公司 | A kind of data processing method and device |
| US20190268405A1 (en) * | 2018-02-28 | 2019-08-29 | International Business Machines Corporation | Load balancing with power of random choices |
| CN108551474A (en) * | 2018-03-26 | 2018-09-18 | 南京邮电大学 | A kind of load-balancing method of server cluster |
| CN108733475A (en) * | 2018-05-22 | 2018-11-02 | 哈尔滨工业大学（威海） | A kind of dynamical feedback dispatching method |
| CN109472520A (en) * | 2018-12-21 | 2019-03-15 | 南京邮电大学 | Multiple no-manned plane method for allocating tasks based on block chain |
| CN110213351A (en) * | 2019-05-17 | 2019-09-06 | 北京航空航天大学 | A kind of dynamic self-adapting I/O load equalization methods towards wide area high-performance computing environment |
| WO2021003935A1 (en) * | 2019-07-11 | 2021-01-14 | 平安科技（深圳）有限公司 | Data cluster storage method and apparatus, and computer device |
| CN111083232A (en) * | 2019-12-27 | 2020-04-28 | 南京邮电大学 | Server-side load balancing method based on improved consistent hash |
| CN111177154A (en) * | 2019-12-27 | 2020-05-19 | 掌迅亿通(北京)信息科技有限公司 | Distributed database caching method and hash ring optimization thereof |
| CN111209120A (en) * | 2020-01-09 | 2020-05-29 | 平安科技（深圳）有限公司 | Data synchronization method and device for microservice and computer readable storage medium |
| CN112487560A (en) * | 2020-12-11 | 2021-03-12 | 国网江苏省电力有限公司南通供电分公司 | Electric power traffic coupling system coordinated game scheduling method based on EV vehicle owner intention |
Non-Patent Citations (4)
| Title |
|---|
| JIADI CHEN等: "Divisible Load Scheduling in Mobile Grid based on Stackelberg Pricing Game", 《VEHICULAR TECHNOLOGY CONFERENCE》, pages 11 - 14 * |
| 张开琦;刘晓燕;王信;吉春山;严馨;: "基于动态权重的一致性哈希微服务负载均衡优化", 计算机工程与科学, no. 08, pages 1339 - 1344 * |
| 李宁;: "基于一致性Hash算法的分布式缓存数据冗余", 软件导刊, no. 01, pages 47 - 50 * |
| 苑迎;王翠荣;王聪;任婷婷;刘冰玉;: "基于非完全信息博弈的云资源分配模型", 计算机研究与发展, no. 06, pages 1342 - 1351 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113689103B (en) | 2023-11-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ismaeel et al. | Proactive dynamic virtual-machine consolidation for energy conservation in cloud data centres | |
| CN103595780B (en) | Cloud computing resource scheduling method based on the weight that disappears | |
| Andronikou et al. | Dynamic QoS-aware data replication in grid environments based on data “importance” | |
| US9250965B2 (en) | Resource allocation for migration within a multi-tiered system | |
| Mokadem et al. | A data replication strategy with tenant performance and provider economic profit guarantees in Cloud data centers | |
| CN103383653B (en) | Cloud resource management and dispatching method and system | |
| CN104731528B (en) | A kind of construction method and system of cloud computing block storage service | |
| Javadpour et al. | Improving load balancing for data-duplication in big data cloud computing networks | |
| Jie et al. | Game-theoretic online resource allocation scheme on fog computing for mobile multimedia users | |
| Limam et al. | Data replication strategy with satisfaction of availability, performance and tenant budget requirements | |
| CN102158533B (en) | Distributed web service selection method based on QoS (Quality of Service) | |
| CN111176784B (en) | Virtual machine integration method based on extreme learning machine and ant colony system | |
| CN115562870B (en) | Task node resource construction method of cluster | |
| Tu et al. | An optimized cluster storage method for real-time big data in Internet of Things | |
| Boukhelef et al. | Optimizing the cost of DBaaS object placement in hybrid storage systems | |
| CN105635285B (en) | A kind of VM migration scheduling method based on state aware | |
| CN102609508B (en) | High-speed access method of files in network storage | |
| Selvi et al. | Popularity (hit rate) based replica creation for enhancing the availability in cloud storage | |
| Chikhaoui et al. | A cost model for hybrid storage systems in a cloud federations | |
| CN116048773B (en) | Distributed collaborative task assignment method and system based on wave function collapse | |
| CN113689103B (en) | Mining and shunting intelligent scheduling management method, device and system for self-adaptive load balancing | |
| Tabet et al. | A data replication strategy for document-oriented NoSQL systems | |
| US20190079987A1 (en) | Distributed data storage | |
| KR102054068B1 (en) | Partitioning method and partitioning device for real-time distributed storage of graph stream | |
| Zhang | A QoS-enhanced data replication service in virtualised cloud environments |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |