CN115599300A - Task allocation method, device, equipment and medium - Google Patents
Task allocation method, device, equipment and medium Download PDFInfo
- Publication number
- CN115599300A CN115599300A CN202211292977.6A CN202211292977A CN115599300A CN 115599300 A CN115599300 A CN 115599300A CN 202211292977 A CN202211292977 A CN 202211292977A CN 115599300 A CN115599300 A CN 115599300A
- Authority
- CN
- China
- Prior art keywords
- task
- node
- nodes
- distribution
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0629—Configuration or reconfiguration of storage systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application discloses a task allocation method, a device, equipment and a medium, and relates to the field of distributed storage. Creating a task recording object in the distributed storage cluster, and creating a task management service corresponding to each node; selecting task distribution nodes from all nodes according to the task recording object; judging whether a task is received or not according to the task recording object and the task distribution node; and if so, distributing the task to the target node through the task distribution node so that the target node processes the task through the corresponding task management service. Therefore, in the scheme, the task recording object recording task is created in the distributed storage cluster, and the task distribution node is selected from all the nodes for task distribution; when the task is received, the task is distributed to other nodes in the storage cluster by the task distribution node for processing, so that the phenomenon that volume management operations are concentrated on management nodes to be accumulated and executed is avoided, node resources are saved, and the overall stability of the distributed storage cluster is improved.
Description
Technical Field
The present application relates to the field of distributed storage, and in particular, to a method, an apparatus, a device, and a medium for task allocation.
Background
For the client, the use of block storage can format and establish an arbitrary file system as if using a physical hard disk, so the block storage is suitable for data storage in most general service scenarios and is increasingly widely used.
In a block storage scenario, the volume assumes the main functional implementation and business output. Although the distributed storage system adopts a partial decentralized thinking design mode, the tasks of management type operation of the volumes still need to be performed at the management nodes, and a large number of tasks of the volume management type operation are concentrated on the management nodes to be executed in a stacking mode, so that the system resources of the management nodes are insufficient, and the overall stability of the distributed storage cluster is influenced.
In view of the above problems, an urgent need exists in the art for a solution to how to avoid accumulation of operation tasks on a volume in a distributed storage system on a node to improve stability of a distributed storage cluster.
Disclosure of Invention
The application aims to provide a task allocation method, a device, equipment and a medium, which are used for realizing allocation of operation tasks of volumes in a distributed storage system so as to improve the stability of a distributed storage cluster.
In order to solve the above technical problem, the present application provides a task allocation method, including:
creating a task recording object in the distributed storage cluster so as to record information of a task representing a volume operation request;
creating task management service corresponding to each node;
selecting task allocation nodes from the nodes according to the task recording object;
judging whether the task is received or not according to the task recording object and the task distribution node;
and if so, distributing the task to a target node through the task distribution node so that the target node processes the task through the corresponding task management service.
Preferably, after the task management service corresponding to each node is created, before a task allocation node is selected from each node according to the task record object, the method further includes:
and registering the node ID of each corresponding node to the task record object through each task management service.
Preferably, the selecting a task allocation node from the nodes according to the task record object includes:
creating a distributed lock based on the task record object;
receiving the application of each node to the distributed lock through the corresponding task management service;
respectively judging whether each node applies for the distributed lock;
and if so, applying to the nodes of the distributed lock to be the task distribution nodes.
Preferably, the determining whether the task is received according to the task recording object and the task allocation node includes:
the task in the task recording object is read at regular time through the task distribution node;
judging whether the read task is the same as the existing task in a task list or not;
if not, confirming that the task is received, and acquiring task information of the task; the task information comprises a task type, source volume information, destination volume information and an executor ID;
setting a task ID of the task and adding the task ID to the task information;
and writing the task information into the task recording object, and entering the step of distributing the task to a target node through the task distribution node.
Preferably, the allocating, by the task allocation node, the task to the target node includes:
calculating the task ID of the task through a consistent hashing algorithm to distribute the task to the target node;
writing the node ID of the target node to a field of the executor ID of the task;
and informing each node of refreshing the corresponding task list through broadcasting.
Preferably, the specific process of the target node processing the task through the corresponding task management service includes:
refreshing the task list according to the broadcast;
reading the executor ID of each task in the task list;
judging whether the executor ID is consistent with the node ID;
if yes, executing the task according to the corresponding task information;
and updating the task execution progress to the progress field of the task through callback, and deleting the corresponding task information after the task execution is finished.
Preferably, the method further comprises the following steps:
and receiving prompt information for updating the task list, which is sent by the target node, so that the prompt information is used for updating the task list through the task distribution node.
In order to solve the above technical problem, the present application further provides a task allocation apparatus, including:
the system comprises a first establishing module, a second establishing module and a third establishing module, wherein the first establishing module is used for establishing a task recording object in the distributed storage cluster so as to record information representing a task of a volume operation request;
the second establishing module is used for establishing task management service corresponding to each node;
the selecting module is used for selecting task distribution nodes from the nodes according to the task recording object;
the judging module is used for judging whether the task is received or not according to the task recording object and the task distribution node; if yes, triggering the distribution module;
the distribution module is configured to distribute the task to a target node through the task distribution node, so that the target node processes the task through the corresponding task management service.
In order to solve the above technical problem, the present application further provides a task allocation apparatus, including:
a memory for storing a computer program;
a processor for implementing the steps of the task allocation method described above when executing the computer program.
In order to solve the above technical problem, the present application further provides a computer-readable storage medium, where a computer program is stored, and the computer program, when executed by a processor, implements the steps of the task allocation method.
According to the task allocation method provided by the application, a task recording object is established in a distributed storage cluster so as to record information of a task representing a volume operation request; creating task management service corresponding to each node; selecting task distribution nodes from all nodes according to the task recording object; judging whether a task is received or not according to the task recording object and the task distribution node; and if so, distributing the task to the target node through the task distribution node so that the target node processes the task through the corresponding task management service. Therefore, in the scheme, the task recording object recording task is created in the distributed storage cluster, and the task distribution node is selected from all the nodes for task distribution; when the task is received, the task is distributed to other nodes in the storage cluster by the task distribution node for processing, so that the phenomenon that volume management operations are concentrated on management nodes to be accumulated and executed is avoided, node resources are saved, and the overall stability of the distributed storage cluster is improved.
In addition, the embodiment of the application also provides a task allocation device, equipment and a medium, and the effect is the same as the above.
Drawings
In order to more clearly illustrate the embodiments of the present application, the drawings needed for the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings can be obtained by those skilled in the art without inventive effort.
Fig. 1 is a flowchart of a task allocation method according to an embodiment of the present application;
fig. 2 is a schematic diagram of a task allocation apparatus according to an embodiment of the present application;
fig. 3 is a schematic diagram of a task allocation apparatus according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without any creative effort belong to the protection scope of the present application.
The core of the application is to provide a task allocation method, a device, equipment and a medium.
In order that those skilled in the art will better understand the disclosure, the following detailed description will be given with reference to the accompanying drawings.
Fig. 1 is a flowchart of a task allocation method provided in an embodiment of the present application. As shown in fig. 1, the method comprises:
s10: a task record object is created in the distributed storage cluster to record information characterizing a task of a volume operation request.
In a specific implementation, a Task record object (Task _ obj) is first created in the distributed storage cluster. It is understood that the object is used to record various items of information of the task, including but not limited to information of task type, operation target (source volume, destination volume), executor ID, etc. Every time the distributed storage cluster receives a volume operation request, the related information of the task should be added to the task record object.
S11: and creating a task management service corresponding to each node.
Further, a task management service corresponding to each node is created. It is understood that there are multiple nodes in the distributed storage cluster, and each node can be applied to data processing in different task scenarios. In the specific implementation, the processing of the tasks by the nodes is mainly realized by the respective task management services.
S12: and selecting a task distribution node from the nodes according to the task recording object.
In order to avoid a large amount of volume management type operations of the distributed storage system being concentrated on the management node accumulation execution, in the present embodiment, a task allocation node (leader) is selected from the nodes according to the task record object.
It should be noted that the task allocation node is one of all nodes of the distributed storage cluster, and its main role is to allocate the task of the volume operation request received by the distributed storage cluster to the rest nodes in the cluster for processing. In this embodiment, the specific selection manner of the task allocation node is not limited, and the task allocation node may be directly selected from each node in the cluster, or may be selected by way of voting or receiving a selection request, depending on the specific implementation situation.
S13: judging whether a task is received or not according to the task recording object and the task distribution node; if yes, the process proceeds to step S14.
S14: and distributing the tasks to the target nodes through the task distribution nodes so that the target nodes process the tasks through corresponding task management services.
And further, judging whether the task is received or not according to the task recording object and the task distribution node. It can be understood that, since the task recording object is used for recording task information and the task allocation node is used for allocating a task, whether the task is received can be determined according to the task recording object and the task allocation node. In this embodiment, the specific process of determining whether to receive the task is not limited, and is determined according to a specific implementation situation.
And after the task is confirmed to be received, distributing the task to the target node through the task distribution node so that the target node can process the task through the corresponding task management service. It should be noted that, in this embodiment, the task allocation node does not limit the specific allocation manner of the task, and may be directly allocated to the designated target node, and may also be allocated according to the relevant information of the task, and may also be allocated according to the current states of the other nodes, depending on the specific implementation situation. And after the target node is distributed to the tasks, processing the tasks through corresponding task management services.
In this embodiment, a task recording object is created in a distributed storage cluster to record information representing a task of a volume operation request; creating task management service corresponding to each node; selecting a task distribution node from each node according to the task recording object; judging whether a task is received or not according to the task recording object and the task distribution node; and if so, distributing the task to the target node through the task distribution node so that the target node processes the task through the corresponding task management service. Therefore, according to the scheme, the task recording object recording task is established in the distributed storage cluster, and the task distribution node is selected from all the nodes for task distribution; when the task is received, the task is distributed to other nodes in the storage cluster by the task distribution node for processing, so that the phenomenon that volume management operations are concentrated on management nodes to be accumulated and executed is avoided, node resources are saved, and the overall stability of the distributed storage cluster is improved.
On the basis of the foregoing embodiment, as a preferred embodiment, after the task management service corresponding to each node is created, before a task allocation node is selected from each node according to a task record object, the method further includes:
and registering the node ID of each corresponding node to the task record object through each task management service.
It can be understood that, in order to enable the task allocation node to allocate the task to the remaining nodes, the task allocation node needs to know information of each remaining node.
Specifically, after the Task management service corresponding to each node is created, the node ID of each corresponding node is registered with the Task recording object Task _ obj through the Task management service of each node, and subsequent Task allocation is facilitated by storing information of each node used for processing the Task.
On the basis of the above embodiment, as a preferred embodiment, selecting a task allocation node from nodes according to a task recording object includes:
creating a distributed lock based on the task record object;
receiving an application of each node to a distributed lock through a corresponding task management service;
respectively judging whether each node applies for the distributed lock;
and if so, applying the nodes of the distributed lock to distribute the nodes for the task.
In order to realize the selection of the Task allocation node, as a preferred embodiment, in this embodiment, a distributed lock is first created on the Task recording object Task _ obj; further all nodes apply for the lock through respective task management service; the task recording object receives the application of each node to the distributed lock and respectively judges whether each node applies to the distributed lock; and the node applying for and holding the distributed lock is the task distribution node leader.
It should be noted that, in this embodiment, the determination method for each node applying for the distributed lock is not limited, and may be determined according to the time sequence of receiving each application, or may be determined by a voting method, depending on the specific implementation.
In the embodiment, a distributed lock is created based on a task record object; receiving an application of each node to a distributed lock through a corresponding task management service; respectively judging whether each node applies for the distributed lock; and if so, applying to the nodes of the distributed lock as task distribution nodes, and realizing the selection of the task distribution nodes.
On the basis of the above embodiment, as a preferred embodiment, the determining whether the task is received according to the task recording object and the task distribution node includes:
reading the tasks in the task recording object at regular time through the task distribution node;
judging whether the read task is the same as the existing task in the task list or not;
if not, confirming the received task and acquiring task information of the task; the task information comprises a task type, source volume information, destination volume information and an executor ID;
setting a task ID of the task and adding the task ID to task information;
and writing the task information into the task recording object, and entering a step of distributing the task to the target node through the task distribution node.
In specific implementation, the task allocation node leader reads the task in the task recording object at regular time, and judges whether the read task is the same as the existing task in the task list. It is understood that the task list in this embodiment is a list for storing existing task information in the distributed storage cluster; and comparing the tasks in the list with the tasks in the task recording objects, and when the read task of the task recording object is different from the existing tasks in the task list, a new task appears.
Further, when the task is confirmed to be received, task information of the task is obtained. In this embodiment, the task information specifically includes a task type, source volume information, destination volume information, and an executor ID. And simultaneously setting task IDs of the tasks, adding the task IDs into the task information, writing all the task information into a task recording object in a structural form, and finally distributing the tasks to target nodes through task distribution nodes. It should be noted that in an implementation, the task ID may be specifically set to a randomly allocated Universal Unique Identifier (UUID) to identify the task.
In addition, in this embodiment, there is no limitation on the timing time for the task distribution node to read the task in the task recording object at regular time, which is determined according to a specific implementation situation.
In the embodiment, the tasks in the task recording objects are read regularly through the task distribution nodes; judging whether the read task is the same as the existing task in the task list or not; if not, confirming the received task and acquiring task information of the task; the task information comprises a task type, source volume information, destination volume information and an executor ID; setting a task ID of a task and adding the task ID to task information; and writing the task information into the task recording object, and entering a step of distributing the task to the target node through the task distribution node, so that the judgment of whether the task is received is realized, and meanwhile, the task information is set so as to facilitate the subsequent task processing.
On the basis of the above embodiment, as a preferred embodiment, allocating a task to a target node through a task allocation node includes:
calculating a task ID of the task through a consistent hash algorithm to distribute the task to the target node;
writing the node ID of the target node into a field of executor ID of the task;
and informing each node to refresh the corresponding task list through broadcasting.
In order to realize task allocation of the task allocation node to the target node, as a preferred embodiment, in this embodiment, a task ID of the task is calculated by using a consistent hash algorithm, so as to allocate the task to the target node.
The Consistent hashing (Consistent hashing) algorithm is a special hashing algorithm and aims to solve the problem of distributed caching. For example, when a server is removed or added, the mapping relationship between the existing service request and the processing request server can be changed as little as possible. Therefore, the task allocation realized by calculating the task ID of the task through the consistent hash algorithm can realize the load balance of the task allocation, avoid allocating a plurality of tasks to the same node, and further improve the task processing efficiency of the whole distributed storage system.
And further writing the node ID of the target node into a field of the executor ID in the task information of the task, and informing each node to refresh a corresponding task list through broadcasting, so that other nodes can know through the task list, and the task is processed by the target node at present.
In the embodiment, the task ID of the task is calculated through a consistent hash algorithm so as to distribute the task to the target node; writing the node ID of the target node into a field of executor ID of the task; and each node is informed to refresh the corresponding task list through broadcasting, so that the task distribution is realized.
On the basis of the above embodiment, as a preferred embodiment, a specific process in which the target node processes the task through the corresponding task management service includes:
refreshing the task list according to the broadcast;
reading the executor ID of each task in the task list;
judging whether the executor ID is consistent with the node ID;
if yes, executing the task according to the corresponding task information;
and updating the task execution progress to the progress field of the task through callback, and deleting corresponding task information after the task execution is finished.
In a preferred embodiment, after the target node receives the broadcast, the target node refreshes the task list according to the broadcast. And reading the executor ID of each task in the list, and comparing the executor ID with the node ID of the executor. And if the executor ID is consistent with the self node ID, executing the corresponding task according to the task information of the task.
In order to realize the visual display of the task execution progress, in the specific implementation, the task execution progress is updated to the progress field of the task in a callback mode, and the task execution condition can be checked through the task list of the target node. In addition, after the task is executed, the corresponding task information is deleted, so that the storage resource of the task list is released.
In the embodiment, the task list is refreshed according to broadcasting; reading the executor ID of each task in the task list; judging whether the executor ID is consistent with the node ID; if yes, executing the task according to the corresponding task information; and updating the task execution progress to the progress field of the task through callback, and deleting corresponding task information after the task execution is finished. The task processing is realized, and the task execution progress is displayed.
On the basis of the above embodiment, as a preferred embodiment, the method further includes:
and receiving prompt information of updating the task list sent by the target node so as to update the task list through the task distribution node.
Finally, after the task processing is completed, in order to update the task processing state by the task allocation node, as a preferred embodiment, the prompt information for updating the task list sent by the target node is received, and the task allocation node further updates the task list according to the prompt information, so that the task information is cleared after the execution is completed, and the storage space is released.
In the foregoing embodiments, detailed descriptions are given to the task allocation method, and the present application also provides embodiments corresponding to the task allocation apparatus.
Fig. 2 is a schematic diagram of a task allocation apparatus according to an embodiment of the present application. As shown in fig. 2, the task assigning apparatus includes:
the first creating module 10 is configured to create a task record object in the distributed storage cluster to record information characterizing a task of the volume operation request.
And a second creating module 11, configured to create a task management service corresponding to each node.
And the selecting module 12 is used for selecting the task distribution node from the nodes according to the task recording object.
The judging module 13 is configured to judge whether the task is received according to the task recording object and the task allocation node; if yes, triggering the distribution module 14;
and the allocating module 14 is configured to allocate the task to the target node through the task allocating node, so that the target node processes the task through the corresponding task management service.
In this embodiment, the task allocation apparatus includes a first creating module, a second creating module, a selecting module, a determining module, and an allocating module. Recording information representing a task of a volume operation request by creating a task record object in a distributed storage cluster; creating task management service corresponding to each node; selecting a task distribution node from each node according to the task recording object; judging whether a task is received or not according to the task recording object and the task distribution node; and if so, distributing the task to the target node through the task distribution node so that the target node processes the task through the corresponding task management service. Therefore, in the scheme, the task recording object recording task is created in the distributed storage cluster, and the task distribution node is selected from all the nodes for task distribution; when the task is received, the task is distributed to other nodes in the storage cluster by the task distribution node for processing, so that the phenomenon that volume management operations are concentrated on management nodes to be accumulated and executed is avoided, node resources are saved, and the overall stability of the distributed storage cluster is improved.
Fig. 3 is a schematic diagram of a task allocation device according to an embodiment of the present application. As shown in fig. 3, the task assigning apparatus includes:
a memory 20 for storing a computer program.
A processor 21 for implementing the steps of the task allocation method as mentioned in the above embodiments when executing the computer program.
The task allocation device provided by the embodiment may include, but is not limited to, a smart phone, a tablet computer, a notebook computer, or a desktop computer.
The processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The Processor 21 may be implemented in hardware using at least one of a Digital Signal Processor (DSP), a Field-Programmable Gate Array (FPGA), and a Programmable Logic Array (PLA). The processor 21 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 21 may be integrated with a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor 21 may further include an Artificial Intelligence (AI) processor for processing computing operations related to machine learning.
The memory 20 may include one or more computer-readable storage media, which may be non-transitory. Memory 20 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used for storing the following computer program 201, wherein after being loaded and executed by the processor 21, the computer program can implement the relevant steps of the task allocation method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202, data 203, and the like, and the storage manner may be a transient storage manner or a permanent storage manner. Operating system 202 may include, among others, windows, unix, linux, and the like. Data 203 may include, but is not limited to, data involved in a task assignment method.
In some embodiments, the task assigning device may further include a display 22, an input/output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.
Those skilled in the art will appreciate that the configuration shown in FIG. 3 is not intended to be limiting and may include more or fewer components than those shown.
In this embodiment, the task assigning apparatus includes a memory and a processor. The memory is used for storing a computer program; the processor is adapted to carry out the steps of the task assigning method as mentioned in the above embodiments when executing the computer program. Recording information representing a task of a volume operation request by creating a task record object in a distributed storage cluster; creating task management service corresponding to each node; selecting task distribution nodes from all nodes according to the task recording object; judging whether a task is received or not according to the task recording object and the task distribution node; and if so, distributing the task to the target node through the task distribution node so that the target node processes the task through the corresponding task management service. Therefore, in the scheme, the task recording object recording task is created in the distributed storage cluster, and the task distribution node is selected from all the nodes for task distribution; when the task is received, the task is distributed to other nodes in the storage cluster by the task distribution node for processing, so that the phenomenon that volume management operations are concentrated on management nodes to be accumulated and executed is avoided, node resources are saved, and the overall stability of the distributed storage cluster is improved.
Finally, the application also provides a corresponding embodiment of the computer readable storage medium. The computer-readable storage medium has stored thereon a computer program which, when being executed by a processor, carries out the steps as set forth in the above-mentioned method embodiments.
It is to be understood that if the method in the above embodiments is implemented in the form of software functional units and sold or used as a stand-alone product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium and executes all or part of the steps of the methods described in the embodiments of the present application, or all or part of the technical solutions. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
In this embodiment, a computer program is stored on a computer-readable storage medium, and when executed by a processor, the computer program implements the steps described in the above method embodiments. Recording information representing a task of a volume operation request by creating a task recording object in a distributed storage cluster; creating task management service corresponding to each node; selecting a task distribution node from each node according to the task recording object; judging whether a task is received or not according to the task recording object and the task distribution node; and if so, distributing the task to the target node through the task distribution node so that the target node processes the task through the corresponding task management service. Therefore, in the scheme, the task recording object recording task is created in the distributed storage cluster, and the task distribution node is selected from all the nodes for task distribution; when the task is received, the task is distributed to other nodes in the storage cluster by the task distribution node for processing, so that the phenomenon that volume management operations are concentrated on management nodes to be accumulated and executed is avoided, node resources are saved, and the overall stability of the distributed storage cluster is improved.
The above detailed description is provided for a task allocation method, apparatus, device and medium provided by the present application. The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description. It should be noted that, for those skilled in the art, without departing from the principle of the present application, the present application can also make several improvements and modifications, and those improvements and modifications also fall into the protection scope of the claims of the present application.
It is further noted that, in the present specification, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
Claims (10)
1. A task allocation method, comprising:
creating a task recording object in the distributed storage cluster so as to record information of a task representing a volume operation request;
creating task management service corresponding to each node;
selecting task allocation nodes from the nodes according to the task recording object;
judging whether the task is received according to the task recording object and the task distribution node;
and if so, distributing the task to a target node through the task distribution node so that the target node processes the task through the corresponding task management service.
2. The task allocation method according to claim 1, wherein after the task management service corresponding to each node is created, before the task allocation node is selected from each node according to the task record object, the method further comprises:
and registering the node ID of each corresponding node to the task record object through each task management service.
3. The task allocation method according to claim 1, wherein the selecting a task allocation node from the nodes according to the task record object comprises:
creating a distributed lock based on the task record object;
receiving the application of each node to the distributed lock through the corresponding task management service;
respectively judging whether each node applies for the distributed lock;
and if so, applying to the nodes of the distributed lock to be the task distribution nodes.
4. The task allocation method according to claim 2, wherein the determining whether the task is received according to the task recording object and the task allocation node comprises:
the task in the task recording object is read at regular time through the task distribution node;
judging whether the read task is the same as the existing task in a task list or not;
if not, confirming that the task is received, and acquiring task information of the task; the task information comprises a task type, source volume information, destination volume information and an executor ID;
setting a task ID of the task and adding the task ID to the task information;
and writing the task information into the task recording object, and entering the step of distributing the task to a target node through the task distribution node.
5. The task allocation method according to claim 4, wherein the allocating the task to a target node by the task allocation node comprises:
calculating the task ID of the task through a consistent hashing algorithm to distribute the task to the target node;
writing the node ID of the target node to a field of the executor ID of the task;
and informing each node of refreshing the corresponding task list through broadcasting.
6. The task allocation method according to claim 5, wherein the specific process of the target node processing the task through the corresponding task management service includes:
refreshing the task list according to the broadcast;
reading the executor ID of each task in the task list;
judging whether the executor ID is consistent with the node ID;
if yes, executing the task according to the corresponding task information;
and updating the task execution progress to the progress field of the task through callback, and deleting the corresponding task information after the task execution is finished.
7. The task allocation method according to any one of claims 4 to 6, further comprising:
and receiving prompt information sent by the target node for updating the task list so as to update the task list through the task distribution node.
8. A task assigning apparatus, comprising:
the system comprises a first establishing module, a second establishing module and a third establishing module, wherein the first establishing module is used for establishing a task recording object in the distributed storage cluster so as to record information representing a task of a volume operation request;
the second establishing module is used for establishing task management service corresponding to each node;
the selecting module is used for selecting task distribution nodes from the nodes according to the task recording object;
the judging module is used for judging whether the task is received or not according to the task recording object and the task distribution node; if yes, triggering the distribution module;
the distribution module is configured to distribute the task to a target node through the task distribution node, so that the target node processes the task through the corresponding task management service.
9. A task assigning apparatus, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the task allocation method according to any one of claims 1 to 7 when executing said computer program.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the task assigning method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211292977.6A CN115599300A (en) | 2022-10-21 | 2022-10-21 | Task allocation method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211292977.6A CN115599300A (en) | 2022-10-21 | 2022-10-21 | Task allocation method, device, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115599300A true CN115599300A (en) | 2023-01-13 |
Family
ID=84848812
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211292977.6A Pending CN115599300A (en) | 2022-10-21 | 2022-10-21 | Task allocation method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115599300A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115934006A (en) * | 2023-03-14 | 2023-04-07 | 北京志凌海纳科技有限公司 | IO access point and data processing task management method, device, equipment and medium |
| CN117591039A (en) * | 2024-01-18 | 2024-02-23 | 济南浪潮数据技术有限公司 | Distributed storage method, system, equipment and medium |
-
2022
- 2022-10-21 CN CN202211292977.6A patent/CN115599300A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115934006A (en) * | 2023-03-14 | 2023-04-07 | 北京志凌海纳科技有限公司 | IO access point and data processing task management method, device, equipment and medium |
| CN117591039A (en) * | 2024-01-18 | 2024-02-23 | 济南浪潮数据技术有限公司 | Distributed storage method, system, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11586673B2 (en) | Data writing and reading method and apparatus, and cloud storage system | |
| CN115599300A (en) | Task allocation method, device, equipment and medium | |
| US9385989B2 (en) | Method and apparatus for managing MAC address generation for virtualized environments | |
| CN110941481A (en) | Resource scheduling method, device and system | |
| CN109408590B (en) | Method, device and equipment for expanding distributed database and storage medium | |
| CN112231108A (en) | Task processing method and device, computer readable storage medium and server | |
| CN111078628A (en) | Multi-disk concurrent data migration method, system, device and readable storage medium | |
| CN107977275B (en) | Task processing method based on message queue and related equipment | |
| CN104182295A (en) | Data backup method and data backup device | |
| CN106571935B (en) | Resource scheduling method and equipment | |
| CN114884962A (en) | Load balancing method and device and electronic equipment | |
| CN105874426A (en) | Batch processing method and device for system invocation commands | |
| CN108205559B (en) | Data management method and equipment thereof | |
| CN112433669A (en) | Method, system, equipment and medium for online migration of distributed storage volume | |
| CN112631994A (en) | Data migration method and system | |
| CN112363980A (en) | Data processing method and device for distributed system | |
| CN111143053A (en) | Scheduling method of timing task, server and storage device | |
| CN112596880A (en) | Data processing method, device, equipment and storage medium | |
| CN111198756A (en) | Application scheduling method and device of kubernets cluster | |
| CN111104225A (en) | Data processing method, device, equipment and medium based on MapReduce | |
| CN111475277A (en) | Resource allocation method, system, equipment and machine readable storage medium | |
| CN111491040B (en) | IP distribution method and IP distribution device | |
| CN114721876A (en) | Data backup method, device and medium | |
| CN112817766B (en) | Memory management method, electronic equipment and medium | |
| CN110673797A (en) | Logical volume copying method in distributed block storage service |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |