CN109032801B - Request scheduling method, system, electronic equipment and storage medium - Google Patents
Request scheduling method, system, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN109032801B CN109032801B CN201810835346.1A CN201810835346A CN109032801B CN 109032801 B CN109032801 B CN 109032801B CN 201810835346 A CN201810835346 A CN 201810835346A CN 109032801 B CN109032801 B CN 109032801B
- Authority
- CN
- China
- Prior art keywords
- request
- load
- node
- total
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 230000004044 response Effects 0.000 claims abstract description 48
- 238000004364 calculation method Methods 0.000 claims description 23
- 238000004590 computer program Methods 0.000 claims description 12
- 230000003068 static effect Effects 0.000 claims description 5
- 230000009286 beneficial effect Effects 0.000 abstract description 4
- 230000006854 communication Effects 0.000 description 11
- 238000004891 communication Methods 0.000 description 10
- 230000008569 process Effects 0.000 description 10
- 238000005516 engineering process Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000009471 action Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/505—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the load
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Computer And Data Communications (AREA)
Abstract
The application discloses a request scheduling method, a request scheduling system, an electronic device and a computer readable storage medium, wherein the method comprises the following steps: when a target request of a client is received, determining the request type of the target request; determining weight values corresponding to all resource index items of the target request according to a preset weight rule and the request type; acquiring current load data of all the resource index items of all the main nodes, and calculating the total load of all the main nodes according to the current load data and the weight values; and determining a response main node of the target request according to the total load. The request scheduling method provided by the application determines the response main nodes by using the total load of each main node, disperses the pressure of a single main node, reduces the cluster pressure, is beneficial to cluster load balancing, and has good adaptivity. Meanwhile, the response speed of the system is improved, and the service risk caused by a single cluster entrance of centralized scheduling is effectively avoided.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a request scheduling method, a request scheduling system, an electronic device, and a computer-readable storage medium.
Background
With the rapid development of computer science and technology, large-scale distributed storage systems have also entered the computing era centered on networks. And the large-scale storage computing node takes network management software as a client, and provides services such as data basic storage, data computing and the like for a user more intuitively. Then, the scheduling of the storage system for the client requests also becomes an important method and function for expanding the performance of the storage system and balancing the load of the distributed cluster nodes.
Currently, the request scheduling of the distributed storage cluster to the client mostly adopts centralized scheduling: the client requests are directly sent and sent out and processed on a single main node deployed in the cluster. Centralized scheduling often causes problems of overlarge cluster master node pressure, overhigh load occupation and the like; meanwhile, the deployment master node is used as a single entry point of the whole cluster, the reliability and stability of the cluster deployment master node are rarely considered, and once a fault occurs, serious loss is caused to corresponding storage services.
Therefore, how to reduce the load pressure of the master node when scheduling is requested is a problem that needs to be solved by those skilled in the art.
Disclosure of Invention
An object of the present application is to provide a request scheduling method, system, an electronic device, and a computer-readable storage medium, which reduce load stress on a master node when scheduling is requested.
In order to achieve the above object, the present application provides a request scheduling method, which is applied to a distributed storage cluster, where the distributed storage cluster includes N master nodes, where N is a positive integer greater than 1, and the method includes:
when a target request of a client is received, determining the request type of the target request;
determining weight values corresponding to all resource index items of the target request according to a preset weight rule and the request type;
acquiring current load data of all the resource index items of all the main nodes, and calculating the total load of all the main nodes according to the current load data and the weight values;
and determining a response main node of the target request according to the total load.
The request types comprise a query type request and a configuration type request, the query type request comprises a dynamic request and a static request, and the configuration type request comprises a dynamic request.
Wherein determining a response master node for the target request according to the total load comprises:
and determining the master node corresponding to the minimum load total value as the response master node of the target request.
After determining the response master node of the target request according to the total load, the method further includes:
updating current load data of all the resource indicator items of all the master nodes.
If the request type of the target request is a query type request, after updating the current load data of all the resource index items of all the master nodes, the method further includes:
and the response main node returns the request data corresponding to the target request to the client.
And the sum of the weighted values corresponding to all the resource index items is 1.
Wherein calculating the total load of all the master nodes according to the current load data and the weight values comprises:
calculating the total load of all the main nodes according to a total load calculation formula; the load total amount calculation formula specifically includes:
wherein tl (j) is a total load amount of the j-th master node, lsc (i) is current load data of the i-th resource index item, weight (i) is a weight value corresponding to the i-th resource index item of the target request; n is the total number of the resource indicator items.
In order to achieve the above object, the present application provides a request scheduling system, which is applied to a distributed storage cluster, where the distributed storage cluster includes N master nodes, where N is a positive integer greater than 1, and the system includes:
the system comprises a receiving module, a sending module and a receiving module, wherein the receiving module is used for determining the request type of a target request when the target request of a client is received;
the determining module is used for determining the weight values corresponding to all resource index items of the target request according to a preset weight rule and the request type;
the computing module is used for acquiring current load data of all the resource index items of all the main nodes and computing the total load of all the main nodes according to the current load data and the weight values;
and the response module is used for determining a response main node of the target request according to the total load.
To achieve the above object, the present application provides an electronic device including:
a memory for storing a computer program;
a processor for implementing the steps of the request scheduling method as described above when executing the computer program.
To achieve the above object, the present application provides a computer-readable storage medium having stored thereon a computer program, which when executed by a processor, performs the steps of the above-mentioned request scheduling method.
According to the above scheme, the request scheduling method provided by the application includes: when a target request of a client is received, determining the request type of the target request; determining weight values corresponding to all resource index items of the target request according to a preset weight rule and the request type; acquiring current load data of all the resource index items of all the main nodes, and calculating the total load of all the main nodes according to the current load data and the weight values; and determining a response main node of the target request according to the total load.
According to the request scheduling method, the client request types are firstly subdivided, a plurality of main nodes are arranged in the cluster, after load data of each main node in the cluster are counted, weighted values are distributed to each resource index item, the total load of each main node is determined according to different client request types and load node states, the main node with the minimum total load can be used as a response main node, and the request response rule is adjusted dynamically and adaptively. Therefore, the request scheduling method provided by the application determines the response main nodes by using the total load of each main node, disperses the pressure of single main node, reduces the cluster pressure, is beneficial to cluster load balancing, and has good adaptivity.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a request scheduling method disclosed in an embodiment of the present application;
FIG. 2 is a flow chart of another request scheduling method disclosed in the embodiments of the present application;
FIG. 3 is a timing diagram illustrating another method for scheduling requests according to an embodiment of the present disclosure;
FIG. 4 is an architecture diagram of a distributed storage cluster disclosed in an embodiment of the present application;
fig. 5 is a structural diagram of a request scheduling system disclosed in an embodiment of the present application;
fig. 6 is a block diagram of an electronic device disclosed in an embodiment of the present application;
fig. 7 is a block diagram of another electronic device disclosed in the embodiments of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The embodiment of the application discloses a request scheduling method, which reduces the load pressure of a main node when scheduling is requested.
Referring to fig. 1, a flowchart of a request scheduling method disclosed in an embodiment of the present application is shown in fig. 1, and includes:
s101: when a target request of a client is received, determining the request type of the target request;
in a specific implementation, the distributed storage cluster includes a plurality of master nodes, each master node has the same function, and when a request of a client is received, a request type of the request is determined first. As a preferred embodiment, the request types of this step may include a query class request and a configuration class request, the query class request may include a dynamic request and a static request, and the configuration class request is typically a dynamic request.
S102: determining weight values corresponding to all resource index items of the target request according to a preset weight rule and the request type;
it should be noted that, a basis is provided for electing a request response node in the subsequent step, and in this step, weight values corresponding to all resource index items of the request need to be determined for the request type determined in the previous step.
In the specific implementation, the weight of each resource index item load is first defined as weight (i), i.e. the variable represents the weight of the ith resource index item load. Wherein i represents specific resource index items, such as hard disk IO, CPU (Central Processing Unit, etc.). As a preferred embodiment, all load weights use normalized values, that is, the sum of the weight values corresponding to all resource indicator items is 1, that is:
according to different client request types, different load weights are distributed, for example, the weight value of a hard disk IO index item for a static request (IO intensive type) is distributed to be 0.8, and the weight values of other resource index items are distributed to be 0.1 or 0; the load weight distribution of the CPU and hard disk IO index items of the dynamic request (CPU intensive type) is 0.4, the weight values of other resource index items are relatively less, the load weight distribution of the network port bandwidth of the network intensive request is higher, and the load weights of other index items are relatively less.
S103: acquiring current load data of all the resource index items of all the main nodes, and calculating the total load of all the main nodes according to the current load data and the weight values;
in a specific implementation, the total load is calculated according to the current load data of each resource index item of the main node and the weight value determined in the previous step. The specific load total calculation formula may be in the form of:
wherein tl (j) is a total load amount of the j-th master node, lsc (i) is current load data of the i-th resource index item, weight (i) is a weight value corresponding to the i-th resource index item of the target request; n is the total number of the resource indicator items.
Wherein, LSCused(i) To be used for the ith resource indicator item, LSCtotal(i) Is the total amount of the ith resource index item. For example, if the resource indicator is embodied as memory, then the LSCused(i) For used amount of memory, LSCtotal(i) Is the total amount of memory.
S104: and determining a response main node of the target request according to the total load.
In a specific implementation, after the total load of each master node is obtained, the master node corresponding to the minimum total load may be determined as the response master node of the target request. The scheduling task responding to the master node load target request. The response main node with the minimum load total amount is adopted to improve the response speed of the system, and the service risk caused by a single cluster entrance of centralized scheduling is effectively avoided. The application also discloses a request scheduling system, an electronic device and a computer readable storage medium, which can also achieve the technical effects.
The request scheduling method provided by the embodiment of the application comprises the steps of firstly subdividing client request types, setting a plurality of main nodes in a cluster, distributing weight values to all resource index items after counting load data of all the main nodes in the cluster, determining the total load of all the main nodes according to different client request types and load node states, and enabling the main node with the minimum total load to serve as a response main node, so that the request response rule is dynamically and adaptively adjusted. Therefore, the request scheduling method provided by the embodiment of the application determines the response main node by using the total load of each main node, disperses the pressure of a single main node, reduces the cluster pressure, is beneficial to cluster load balancing, and has good adaptivity.
The embodiment of the application discloses a request scheduling method, and compared with the previous embodiment, the embodiment further explains and optimizes the technical scheme. Specifically, the method comprises the following steps:
referring to fig. 2, a flowchart of another request scheduling method provided in an embodiment of the present application is shown in fig. 2, and includes:
s201: when an inquiry request of a client is received, determining weight values corresponding to all resource index items of the target request according to a preset weight rule and the inquiry request;
s202: acquiring current load data of all the resource index items of all the main nodes, and calculating the total load of all the main nodes according to the current load data and the weight values;
s203: determining the main node corresponding to the minimum load total value as a response main node of the query request;
in this embodiment, a process of responding to a query request by a distributed storage cluster is described in detail, as shown in fig. 3, a client sends a query request to a client request marking module in a host node through a network, the module marks a weight for each resource index item of the query request according to a preset weight rule, and sends the weight to a node load calculation module, and after the module calculates a total load of the host node, a distributed request scheduling module determines a responding host node.
S204: updating the current load data of all the resource index items of all the main nodes;
s205: and the response main node returns the request data corresponding to the query class request to the client.
In a specific implementation, when the response of the master node to the completion of the scheduling request is returned to the client, the current load data of each resource index item of the master node needs to be updated, so that the total load of each master node is recalculated when other requests of the client are received.
Referring to a specific application embodiment of the request scheduling method disclosed in the present application, specifically as shown in fig. 4, the distributed storage cluster includes a plurality of host nodes, each host node includes a node load acquisition module, a node load calculation module, a client request marking module, a distributed request scheduling module, and a DB database, and each host node and each module perform data communication and interaction through an asynchronous message channel.
An independent acquisition process and an interface for providing data query to the outside exist in the node load acquisition module, and Json is used as an exchange format for returned data. Different timing acquisition threads are established in the acquisition process according to different resource load index items, the timing time can be set to 15 seconds, namely, load data is inquired once every 15 seconds through a bottom layer command, and the load data is persisted to the DB database of the local node. Meanwhile, the module provides a query interface in a Json data exchange format for the external for the upper layer to query and call. The specific flow of the module is as follows:
step 1: the node load collection module creates an independent load collection process on each master node.
Step 2: and the acquisition process starts different timing load acquisition threads according to different system load index data sources and the time granularity of 15 s.
Step 3: and the timing acquisition thread calls different bottom layer commands to acquire load sample data of corresponding types.
Step 4: and the timing thread collects the data of the node and simultaneously persists the collected data to the DB database of the node.
Step 5: and the data query thread in the independent process encapsulates the data in the database into a uniform data query interface for upper-layer query.
Step 6: when the data query request is transmitted to the node load acquisition module, the module encapsulates the message according to the message channel format and then distributes the message to the corresponding module.
The client request marking module marks different weight values for each resource index item in the node load calculation module according to different types of requests transmitted by the client. The specific flow of the module is as follows:
step 1: the client request marking module creates and defines a set of uniform load weight distribution rules on each main node.
Step 2: the module assigns different load weights to requests for different types of requests.
Step 3: the module passes the assigned load weight results to the node load calculation module.
Step 4: the load weights are synchronously assigned to the master nodes using asynchronous message channels.
And the node load calculation module calculates the total load of the main node according to the load data and the load weight sent by the node load acquisition module and the client request marking module. The specific data flow of the module is as follows:
step 1: and the node load calculation module receives the load values of the resource index items transmitted by the node load acquisition module.
Step 2: and the node load calculation module receives the weight value transmitted by the client request marking module.
Step 3: and the node load calculation module calculates the total load of each main node according to the load value and the weight value.
Step 4: and the node load calculation module sends the load total calculation result of each main node to the distributed scheduling module.
And the distributed request scheduling module adds the load data sent by the node load calculation module to a load queue corresponding to each main node. When a user triggers a client request, traversing the load value of the load queue, counting the total load in the load queue, and selecting the node with the minimum total load as a response main node of the request. The load queue only stores a fixed amount of load data according to a first-in-first-out principle (FIFO).
Step 1: and the distributed request scheduling module receives the load data sent by the node load calculation module.
Step 2: and the distributed request scheduling module adds the load data to the load queues of the main nodes.
Step 3: when a client request is received, the distributed request scheduling module traverses the load queue, and elects a node with the minimum load of the current load queue as a request response node.
Step 4: and the distributed request scheduling module issues the request instruction to the request response node and returns a request data result.
The asynchronous message channel is responsible for communication between a main node and the main node, between a node load acquisition module and a node load calculation module, between the node load calculation module and a client request marking module, between a distributed scheduling module and the node load calculation module and the like, and mainly relates to client node data query requests, data synchronization between the main nodes, message distribution and processing between the modules and the like in the communication process, wherein the processing flows of the messages are basically consistent, and specifically:
step 1: the message sending source (client, main node, each module) and the message receiving source generate the message module subscription and subscribed relationship through the asynchronous message channel.
Step 2: the message source sends data to the message receiving source through the asynchronous message channel.
Step 3: the message receiving source decodes the received message, processes the message and filters the message (such as repeated message filtering) according to different message types, and then sends the message to the destination to distribute different messages.
In the following, a request scheduling system provided by an embodiment of the present application is introduced, and a request scheduling system described below and a request scheduling method described above may be referred to each other.
Referring to fig. 5, a structure diagram of a request scheduling system provided in an embodiment of the present application is shown in fig. 5, and includes:
a receiving module 501, configured to determine a request type of a target request when the target request of a client is received;
a determining module 502, configured to determine weight values corresponding to all resource indicator items of the target request according to a preset weight rule and the request type;
a calculating module 503, configured to obtain current load data of all the resource indicator items of all the host nodes, and calculate a total load of all the host nodes according to the current load data and the weight values;
a response module 504, configured to determine a response master node of the target request according to the total load.
The request scheduling system provided by the embodiment of the application firstly subdivides the client request types, a plurality of main nodes are arranged in the cluster, after the load data of each main node in the cluster are counted, the weight value is distributed to each resource index item, the total load of each main node is determined according to different client request types and load node states, the main node with the minimum total load can be used as a response main node, and the request response rule is dynamically and adaptively adjusted. Therefore, the request scheduling system provided by the embodiment of the application determines the response main nodes by using the total load of each main node, disperses the pressure of single main node, reduces the cluster pressure, is beneficial to cluster load balancing, and has good self-adaptability.
On the basis of the above embodiment, as a preferred implementation, the request type includes a query class request and a configuration class request, the query class request includes a dynamic request and a static request, and the configuration class request includes a dynamic request.
On the basis of the foregoing embodiment, as a preferred implementation manner, the response module 504 is specifically a module that determines the master node corresponding to the minimum load total amount as the response master node of the target request.
On the basis of the above embodiment, as a preferred implementation, the method further includes:
and the updating module is used for updating the current load data of all the resource index items of all the main nodes.
On the basis of the foregoing embodiment, as a preferred implementation manner, if the request type of the target request is a query type request, the method further includes:
and the return module is used for returning the request data corresponding to the target request to the client by the response main node.
On the basis of the foregoing embodiment, as a preferred implementation manner, the sum of the weighted values corresponding to all resource index items is 1.
On the basis of the foregoing embodiment, as a preferred implementation, the calculating module 503 includes:
the acquisition unit is used for acquiring the current load data of all the resource index items of all the main nodes;
the computing unit is used for computing the total load of all the main nodes according to a total load computing formula; the load total amount calculation formula specifically includes:
wherein tl (j) is a total load amount of the j-th master node, lsc (i) is current load data of the i-th resource index item, weight (i) is a weight value corresponding to the i-th resource index item of the target request; n is the total number of the resource indicator items.
The present application further provides an electronic device, referring to fig. 6, a structure diagram of an electronic device provided in an embodiment of the present application, as shown in fig. 6, includes:
a memory 100 for storing a computer program;
the processor 200, when executing the computer program, may implement the steps of the request scheduling method provided in any of the above embodiments.
Specifically, the memory 100 includes a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer-readable instructions, and the internal memory provides an environment for the operating system and the computer-readable instructions in the non-volatile storage medium to run. The processor 200 provides computing and control capabilities for the electronic device, and when executing the computer program stored in the memory 100, the steps of the request scheduling method provided by any of the above embodiments can be implemented.
According to the embodiment of the application, the client request types are firstly subdivided, the main nodes are arranged in the cluster, after the load data of each main node in the cluster are counted, the weight value is distributed to each resource index item, the total load of each main node is determined according to different client request types and the load node states, the main node with the minimum total load can be used as a response main node, and the request response rule is adjusted dynamically and adaptively. Therefore, the response main nodes are determined by the total load of the main nodes, the pressure of a single main node is dispersed, the cluster pressure is relieved, the cluster load balance is facilitated, and the self-adaption is good.
On the basis of the above embodiment, as a preferred implementation, referring to fig. 7, the electronic device further includes:
and an input interface 300 connected to the processor 200, for acquiring computer programs, parameters and instructions imported from the outside, and storing the computer programs, parameters and instructions into the memory 100 under the control of the processor 200. The input interface 300 may be connected to an input device for receiving parameters or instructions manually input by a user. The input device may be a touch layer covered on a display screen, or a button, a track ball or a touch pad arranged on a terminal shell, or a keyboard, a touch pad or a mouse, etc. Specifically, in the present embodiment, a preset weight rule and the like are input through the input interface 300.
And a display unit 400 connected to the processor 200 for displaying data transmitted by the processor 200. The display unit 400 may be a display screen on a PC, a liquid crystal display screen, or an electronic ink display screen. Specifically, in the present embodiment, the request data and the like may be displayed through the display unit 400.
And a network port 500 connected to the processor 200 for performing communication connection with each external terminal device. The communication technology adopted by the communication connection can be a wired communication technology or a wireless communication technology, such as a mobile high definition link (MHL) technology, a Universal Serial Bus (USB), a High Definition Multimedia Interface (HDMI), a wireless fidelity (WiFi), a bluetooth communication technology, a low power consumption bluetooth communication technology, an ieee802.11 s-based communication technology, and the like. Specifically, in this embodiment, a preset weighting rule and the like may be imported to the processor 200 through the network port 500.
The present application also provides a computer-readable storage medium, which may include: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk. The storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the request scheduling method provided by any of the above embodiments.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description. It should be noted that, for those skilled in the art, it is possible to make several improvements and modifications to the present application without departing from the principle of the present application, and such improvements and modifications also fall within the scope of the claims of the present application.
It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
Claims (8)
1. A request scheduling method is applied to a distributed storage cluster, wherein the distributed storage cluster comprises N main nodes, N is a positive integer greater than 1, and the method comprises the following steps:
when a target request of a client is received, determining the request type of the target request;
determining weight values corresponding to all resource index items of the target request according to a preset weight rule and the request type;
acquiring current load data of all the resource index items of all the main nodes, and calculating the total load of all the main nodes according to the current load data and the weight values;
determining a response main node of the target request according to the total load;
wherein calculating the total load of all the master nodes according to the current load data and the weight values comprises:
calculating the total load of all the main nodes according to a total load calculation formula; the load total amount calculation formula specifically includes:
wherein tl (j) is a total load amount of the j-th master node, lsc (i) is current load data of the i-th resource index item, weight (i) is a weight value corresponding to the i-th resource index item of the target request; n is the total number of the resource indicator items;
wherein determining a response master node for the target request according to the total load comprises:
and determining the master node corresponding to the minimum load total value as the response master node of the target request.
2. The request scheduling method of claim 1, wherein the request types comprise query class requests and configuration class requests, the query class requests comprise dynamic requests and static requests, and the configuration class requests comprise dynamic requests.
3. The request scheduling method of claim 2, wherein after determining the response master node of the target request according to the total load, further comprising:
updating current load data of all the resource indicator items of all the master nodes.
4. The request scheduling method according to claim 3, wherein if the request type of the target request is a query-class request, after the updating of the current load data of all the resource indicator items of all the master nodes, the method further comprises:
and the response main node returns the request data corresponding to the target request to the client.
5. The request scheduling method according to claim 1, wherein the sum of the weight values corresponding to all resource indicator items is 1.
6. A request scheduling system, applied to a distributed storage cluster, where the distributed storage cluster includes N master nodes, where N is a positive integer greater than 1, and the system includes:
the system comprises a receiving module, a sending module and a receiving module, wherein the receiving module is used for determining the request type of a target request when the target request of a client is received;
the determining module is used for determining the weight values corresponding to all resource index items of the target request according to a preset weight rule and the request type;
the computing module is used for acquiring current load data of all the resource index items of all the main nodes and computing the total load of all the main nodes according to the current load data and the weight values;
the response module is used for determining a response main node of the target request according to the total load;
wherein the calculation module comprises:
the acquisition unit is used for acquiring the current load data of all the resource index items of all the main nodes;
the computing unit is used for computing the total load of all the main nodes according to a total load computing formula; the load total amount calculation formula specifically includes:
wherein tl (j) is a total load amount of the j-th master node, lsc (i) is current load data of the i-th resource index item, weight (i) is a weight value corresponding to the i-th resource index item of the target request; n is the total number of the resource index items
The response module is specifically a module that determines the master node corresponding to the minimum load total amount as the response master node of the target request.
7. An electronic device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the request scheduling method according to any one of claims 1 to 5 when executing the computer program.
8. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps of the request scheduling method according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810835346.1A CN109032801B (en) | 2018-07-26 | 2018-07-26 | Request scheduling method, system, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810835346.1A CN109032801B (en) | 2018-07-26 | 2018-07-26 | Request scheduling method, system, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109032801A CN109032801A (en) | 2018-12-18 |
| CN109032801B true CN109032801B (en) | 2022-02-18 |
Family
ID=64645666
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810835346.1A Active CN109032801B (en) | 2018-07-26 | 2018-07-26 | Request scheduling method, system, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109032801B (en) |
Families Citing this family (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111381928B (en) * | 2018-12-28 | 2021-05-25 | 中兴通讯股份有限公司 | Virtual machine migration method, cloud computing management platform and storage medium |
| CN109769031A (en) * | 2019-02-18 | 2019-05-17 | 珠海天天放送科技有限公司 | A kind of dynamic self-adapting load-balancing method and system |
| CN110209693A (en) * | 2019-05-10 | 2019-09-06 | 苏宁云计算有限公司 | High concurrent data query method, apparatus, system, equipment and readable storage medium storing program for executing |
| CN110399296B (en) * | 2019-06-29 | 2022-12-27 | 苏州浪潮智能科技有限公司 | Method, system and medium for testing interactive interface between client and server |
| CN110795238B (en) * | 2019-10-11 | 2023-09-01 | Oppo广东移动通信有限公司 | Load calculation method and device, storage medium and electronic equipment |
| CN111338575B (en) * | 2020-02-20 | 2021-06-29 | 苏州浪潮智能科技有限公司 | Storage service quality control method, device, equipment and storage medium |
| CN111459677A (en) * | 2020-04-01 | 2020-07-28 | 北京顺达同行科技有限公司 | Request distribution method and device, computer equipment and storage medium |
| CN111597023B (en) * | 2020-05-12 | 2023-07-07 | 武汉美和易思数字科技有限公司 | Cluster intelligent scheduling method and device based on learning state |
| CN112308709B (en) * | 2020-11-24 | 2022-08-12 | 上海浦东发展银行股份有限公司 | Shunting method, system and storage medium for credit card customer-obtaining platform |
| CN112463390A (en) * | 2020-12-11 | 2021-03-09 | 厦门市美亚柏科信息股份有限公司 | Distributed task scheduling method and device, terminal equipment and storage medium |
| CN113010315A (en) * | 2021-03-18 | 2021-06-22 | 中国邮政储蓄银行股份有限公司 | Resource allocation method, resource allocation device and computer-readable storage medium |
| CN114968507B (en) * | 2021-04-27 | 2023-08-15 | 中移互联网有限公司 | Image processing task scheduling method and device |
| CN113419993B (en) * | 2021-05-19 | 2024-06-04 | 北京达佳互联信息技术有限公司 | Read-write request processing method and device, electronic equipment and storage medium |
| CN113612841B (en) * | 2021-08-02 | 2023-05-05 | 北京金山云网络技术有限公司 | Task scheduling method, device, computer equipment and storage medium |
| CN115102958B (en) * | 2022-06-17 | 2024-09-06 | 深圳市网心科技有限公司 | Server node scheduling method, device, equipment and storage medium |
| CN115794837B (en) * | 2023-02-01 | 2023-06-23 | 天翼云科技有限公司 | Data table synchronization method, system, electronic equipment and storage medium |
| CN116719632B (en) * | 2023-08-11 | 2024-03-15 | 腾讯科技（深圳）有限公司 | Task scheduling method, device, equipment and medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104657221A (en) * | 2015-03-12 | 2015-05-27 | 广东石油化工学院 | Multi-queue peak-alternation scheduling model and multi-queue peak-alteration scheduling method based on task classification in cloud computing |
| CN105718479A (en) * | 2014-12-04 | 2016-06-29 | 中国电信股份有限公司 | Execution strategy generation method and device under cross-IDC (Internet Data Center) big data processing architecture |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20140080058A (en) * | 2012-12-20 | 2014-06-30 | 삼성전자주식회사 | Load balancing method for multicore and mobile terminal |
-
2018
- 2018-07-26 CN CN201810835346.1A patent/CN109032801B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105718479A (en) * | 2014-12-04 | 2016-06-29 | 中国电信股份有限公司 | Execution strategy generation method and device under cross-IDC (Internet Data Center) big data processing architecture |
| CN104657221A (en) * | 2015-03-12 | 2015-05-27 | 广东石油化工学院 | Multi-queue peak-alternation scheduling model and multi-queue peak-alteration scheduling method based on task classification in cloud computing |
Non-Patent Citations (2)
| Title |
|---|
| "多核集群中的自适应并行调度算法";赵金雁等;《2010年中国计算机大会论文集》;20110922;第89-100页 * |
| "面向高并发服务请求的负载均衡技术研究";田丰等;《中国民航大学学报》;20151031;第33卷(第5期);第48-50节 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109032801A (en) | 2018-12-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109032801B (en) | Request scheduling method, system, electronic equipment and storage medium | |
| CN112689300B (en) | Method, apparatus, system, and medium for managing application and execution unit | |
| CN108776934B (en) | Distributed data calculation method and device, computer equipment and readable storage medium | |
| CN109495524B (en) | Method, equipment and system for subscribing resources of Internet of things | |
| US20240259327A1 (en) | Software load balancer to maximize utilization | |
| CN108268318A (en) | A kind of method and apparatus of distributed system task distribution | |
| CN112363813A (en) | Resource scheduling method and device, electronic equipment and computer readable medium | |
| CN112860695B (en) | Monitoring data query method, device, equipment, storage medium and program product | |
| US20170126583A1 (en) | Method and electronic device for bandwidth allocation based on online media services | |
| CN103699445A (en) | Task scheduling method, device and system | |
| CN107295090A (en) | A kind of method and apparatus of scheduling of resource | |
| CN111431996B (en) | Method, apparatus, device and medium for resource configuration | |
| CN108809848A (en) | Load-balancing method, device, electronic equipment and storage medium | |
| CN106982245B (en) | Client and server in supervisory control and data acquisition system | |
| CN114205317B (en) | SDN and NFV-based service function chain SFC resource allocation method and electronic equipment | |
| US20210360058A1 (en) | Job allocation support system and method | |
| CN113553160A (en) | Task scheduling method and system for edge computing node of artificial intelligence Internet of things | |
| CN109933610A (en) | Data processing method, device, computer equipment and storage medium | |
| CN114911598A (en) | Task scheduling method, device, equipment and storage medium | |
| JP7313351B2 (en) | Resource processing method and system, storage medium, electronic device | |
| CN112241319A (en) | Method, electronic device and computer program product for balancing load | |
| US9501321B1 (en) | Weighted service requests throttling | |
| CN112561301A (en) | Work order distribution method, device, equipment and computer readable medium | |
| CN111988861A (en) | Wireless communication method, device, system and storage medium | |
| CN115633041A (en) | Multi-cluster management method and device, electronic equipment and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |