CN111314401A

CN111314401A - Resource allocation method, device, system, terminal and computer readable storage medium

Info

Publication number: CN111314401A
Application number: CN201811517238.6A
Authority: CN
Inventors: 董铭
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2018-12-12
Filing date: 2018-12-12
Publication date: 2020-06-19
Anticipated expiration: 2038-12-12
Also published as: CN111314401B

Abstract

The embodiment of the invention provides a resource allocation method, a device, a system, a terminal and a computer readable storage medium, wherein the method comprises the following steps: receiving a task request sent by a user side; according to the task request, calling a node in an idle state in physical resources to execute a task carried in the task request, and modifying the state of the called node into an occupied state; and after the called node executes the task, modifying the state of the called node into an idle state. According to the method and the device, the node in the idle state can be quickly called to execute the task aiming at the task request according to the state of the node in the physical resource, and after the called node executes the task, the state of the called node is modified into the idle state, so that the node modified into the idle state can be called again to execute other tasks, and the resource utilization rate and the task processing efficiency are improved.

Description

Resource allocation method, device, system, terminal and computer readable storage medium

Technical Field

The present invention relates to the field of big data technologies, and in particular, to a resource allocation method, apparatus, system, terminal, and computer-readable storage medium.

Background

When big data calculation is started, physical resources are usually applied at one time, and simultaneously, all applied physical resources are released at one time after the task is completed. Two problems exist with such execution logic, the first is that when a new task needs to be computed, it will take a long time to re-apply for physical resources, even exceeding the time for task execution. Secondly, after a single node of the physical resource executes a task, it is necessary to wait for all the associated nodes to execute the task and then release the task, which results in a reduction in the utilization rate of the physical resource.

The above information disclosed in the background section is only for enhancement of understanding of the background of the invention and therefore it may contain information that does not form the prior art that is known to a person of ordinary skill in the art.

Disclosure of Invention

Embodiments of the present invention provide a resource allocation method, apparatus, system, terminal, and computer-readable storage medium, so as to solve one or more technical problems in the prior art.

In a first aspect, an embodiment of the present invention provides a resource allocation method, including:

receiving a task request sent by a user side;

according to the task request, calling a node in an idle state in physical resources to execute a task carried in the task request, and modifying the state of the called node into an occupied state;

and after the called node executes the task, modifying the state of the called node into an idle state.

In one embodiment, before receiving the task request sent by the user side, the method further includes:

and logically marking the nodes in the physical resources, marking the state of the node which is currently executing the task as an occupied state, and marking the state of the node which is not currently executing the task as an idle state.

In one embodiment, according to the task request, invoking a node in an idle state in a physical resource to execute a task carried in the task request includes:

acquiring the number of tasks carried in the task request;

and calling the nodes in the idle state from one or more physical resources to execute the tasks according to the number of the tasks.

In one embodiment, invoking the node in the idle state from one or more physical resources to execute the task according to the number of the tasks includes:

when the number of the nodes in the idle state in any physical resource is larger than or equal to the number of the tasks, calling a plurality of the nodes in the idle state from the physical resource to execute each task.

when the number of the nodes in the idle state in each physical resource is smaller than the number of the tasks, respectively calling the nodes in the idle state from the plurality of physical resources to execute each task.

In a second aspect, an embodiment of the present invention provides a resource allocation method, including:

a user side sends a task request to a resource center;

the resource center calls a node in an idle state in the physical resource to execute the task carried in the task request according to the task request, and modifies the state of the called node into an occupied state;

when the resource center detects that the called node finishes the task, the state of the called node is modified into an idle state;

and the resource center sends the execution result of the task request to the user side.

In a third aspect, an embodiment of the present invention provides a resource configuration apparatus, including:

the receiving module is used for receiving a task request sent by a user side;

the calling module is used for calling a node in an idle state in the physical resource to execute the task carried in the task request according to the task request and modifying the state of the called node into an occupied state;

and the state modification module is used for modifying the state of the called node into an idle state after the called node executes the task.

In one embodiment, further comprising:

and the marking module is used for logically marking the nodes in the physical resources, marking the state of the node which is currently executing the task as an occupied state, and marking the state of the node which is not currently executing the task as an idle state.

In one embodiment, the calling module comprises:

the obtaining submodule is used for obtaining the number of the tasks carried in the task request;

and the calling submodule is used for calling the nodes in the idle state from one or more physical resources to execute the tasks according to the number of the tasks.

In a fourth aspect, an embodiment of the present invention provides a resource allocation system, including:

a resource center adapted to use the resource allocation apparatus according to any of the above-described third aspect;

and the user side is used for sending a task request to the resource center and receiving an execution result of the task request sent by the resource center.

In a fifth aspect, an embodiment of the present invention provides a terminal for resource allocation, including:

the functions can be realized by hardware, and the functions can also be realized by executing corresponding software by hardware. The hardware or software includes one or more modules corresponding to the above-described functions.

In one possible design, the structure of the terminal for resource allocation includes a processor and a memory, the memory is used for storing a program for the terminal supporting resource allocation to execute the method for resource allocation in the first aspect, and the processor is configured to execute the program stored in the memory. The resource configured terminal may further comprise a communication interface for the resource configured terminal to communicate with other devices or a communication network.

In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium for storing computer software instructions for a terminal configured with resources, where the computer software instructions include a program for executing the method for configuring resources in the first aspect to the terminal configured with resources.

One of the above technical solutions has the following advantages or beneficial effects: according to the method and the device, the node in the idle state can be quickly called to execute the task aiming at the task request according to the state of the node in the physical resource, and after the called node executes the task, the state of the called node is modified into the idle state, so that the node modified into the idle state can be called again to execute other tasks, and the resource utilization rate and the task processing efficiency are improved.

The foregoing summary is provided for the purpose of description only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the present invention will be readily apparent by reference to the drawings and following detailed description.

Drawings

In the drawings, like reference numerals refer to the same or similar parts or elements throughout the several views unless otherwise specified. The figures are not necessarily to scale. It is appreciated that these drawings depict only some embodiments in accordance with the disclosure and are therefore not to be considered limiting of its scope.

Fig. 1 is a flowchart of a resource allocation method according to an embodiment of the first aspect of the present invention.

Fig. 2 is a flowchart of a resource allocation method according to another embodiment of the first aspect of the present invention.

Fig. 3 is a flowchart of a resource allocation method according to an embodiment of the second aspect of the present invention.

Fig. 4 is a flowchart of a resource allocation method according to another embodiment of the second aspect of the present invention.

Fig. 5 is a schematic structural diagram of a resource allocation apparatus according to an embodiment of the third aspect of the present invention.

Fig. 6 is a schematic diagram of a specific structure of the calling module in fig. 5.

Fig. 7 is a schematic structural diagram of a resource allocation terminal according to an embodiment of the present invention.

Detailed Description

In the following, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.

In a first aspect, an embodiment of the present invention provides a resource allocation method, as shown in fig. 1, including the following steps:

s100: and receiving a task request sent by a user side.

S200: and according to the task request, calling the nodes in the idle state in the physical resources to execute the tasks carried in the task request, and modifying the state of the called nodes into an occupied state.

S300: and after the called node executes the task, modifying the state of the called node into an idle state. And enabling the node modified into the idle state to be called again to execute other tasks carried in the task request or execute the tasks carried in other task requests.

It should be noted that the number of tasks carried in the task request may be one or more. The called nodes can be from the same physical resource or from a plurality of physical resources respectively. The specific configuration mode is selected preferentially according to the number and the state of the nodes in each applied physical resource. The physical resource can be understood as any resource with operation processing capability, such as a distributed network, a computer, a memory, a CPU, and the like. A node may be understood as each computer in a distributed network, each processing unit in a computer, multiple sub-memories in a memory, multiple CPUs in a computer, etc.

In an embodiment, the task request may also carry information about the node number requirement, that is, the user may specify the number of nodes for executing the task.

In one embodiment, before receiving a task request sent by a user side, the method further includes the following steps:

s400: and logically marking the nodes in the physical resources, marking the state of the node which is currently executing the task as an occupied state, and marking the state of the node which is not currently executing the task as an idle state.

In one embodiment, the steps S100-S400 described above may be performed by a resource center. Before the resource center performs step S400, the method further includes:

all the physical resources corresponding to the physical resources are acquired at one time. The nodes of the physical resources acquired at this time are never called, that is, have not executed any task.

It should be noted that, acquiring all the physical resources corresponding to the resource center at one time may be understood as applying for all the physical resources having a mapping relationship with the resource center (that is, capable of being applied for by the resource center), and each physical resource may be continuously used. The application of the physical resources is not required to be carried out again when the resource center processes a new task request, so that the time consumption of the resource center for applying the physical resources is obviously reduced, and the task processing efficiency is effectively improved.

In a variant embodiment, before the resource center performs step S400, the method further includes:

and acquiring a preset number of physical resources from all the physical resources having a mapping relation with the resource center at one time so as to avoid resource waste caused by applying for all the physical resources having the mapping relation. The specific preset number can be adaptively selected according to the working requirement, and is not specifically limited herein.

In one embodiment, a resource center applies for physical resources to a resource manager, and the resource manager allocates the physical resources having a mapping relationship with the resource center to the resource center according to an application request.

In one embodiment, according to a task request, a node in an idle state in a physical resource is called to execute a task carried in the task request, and the method includes the following steps:

and acquiring the number of tasks carried in the task request.

In a specific embodiment, when the number of nodes in an idle state in any physical resource is greater than or equal to the number of tasks, a plurality of nodes in the idle state are preferentially called from the physical resource to execute each task.

In another specific embodiment, when the number of the nodes in the idle state in each physical resource is less than the number of the tasks, the plurality of nodes in the idle state are respectively called from the plurality of physical resources to execute each task.

In a second aspect, an embodiment of the present invention provides a resource allocation method, as shown in fig. 3, including the following steps:

s10: and the user side sends a task request to the resource center.

S20: and the resource center calls the nodes in the idle state in the physical resources to execute the tasks carried in the task request according to the task request, and modifies the states of the called nodes into the occupied states.

S30: and when the resource center detects that the called node executes the task, modifying the state of the called node into an idle state.

S40: after each node executes all tasks carried in the task request, the resource center sends the execution result of the task request to the user side.

In an embodiment, before the user side sends the task request to the resource center, as shown in fig. 4, the method further includes the following steps:

s50: the resource center carries out logic marking on the nodes in the physical resources, marks the state of the node which is currently executing the task as an occupied state, and marks the state of the node which is not currently executing the task as an idle state.

In a third aspect, an embodiment of the present invention provides a resource configuration apparatus, as shown in fig. 5, including:

the receiving module 11 is configured to receive a task request sent by a user side.

And the calling module 12 is configured to call a node in an idle state in the physical resource to execute the task carried in the task request according to the task request, and modify the state of the called node into an occupied state.

And the state modification module 13 is used for modifying the state of the called node into an idle state after the called node executes the task.

In one embodiment, the resource configuration apparatus further includes:

the marking module 14 is configured to logically mark a node in the physical resource, mark a state of a node currently executing a task as an occupied state, and mark a state of a node not currently executing the task as an idle state.

In one embodiment, as shown in FIG. 6, the calling module 12 includes:

the obtaining submodule 121 is configured to obtain the number of tasks carried in the task request;

and the invoking submodule 122 is configured to invoke the node in the idle state from the one or more physical resources to execute the task according to the number of the tasks.

It should be noted that the resource configuration apparatus in the embodiment of the present invention at least includes modules for implementing the methods and functions in the embodiments in the first aspect.

and the resource center adopts the resource configuration device of the third aspect.

And the user side is used for sending the task request to the resource center and receiving the final execution result sent by the resource center.

An embodiment of the present invention provides a terminal for resource allocation, as shown in fig. 7, including:

a memory 910 and a processor 920, the memory 910 having stored therein computer programs executable on the processor 920. The processor 920 implements the method of resource allocation in the above embodiments when executing the computer program. The number of the memory 910 and the processor 920 may be one or more.

A communication interface 930 for the memory 910 and the processor 920 to communicate with the outside.

Memory 910 may include high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.

If the memory 910, the processor 920 and the communication interface 930 are implemented independently, the memory 910, the processor 920 and the communication interface 930 may be connected to each other through a bus and perform communication with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 7, but this is not intended to represent only one bus or type of bus.

Optionally, in an implementation, if the memory 910, the processor 920 and the communication interface 930 are integrated on a chip, the memory 910, the processor 920 and the communication interface 930 may complete communication with each other through an internal interface.

The embodiment of the invention provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to realize the method for configuring the resources according to any one of the embodiment.

In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.

Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.

Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps of the process, and alternate implementations are included within the scope of the preferred embodiment of the present invention in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present invention.

The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.

It should be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.

It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.

In addition, functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a separate product, may also be stored in a computer readable storage medium. The storage medium may be a read-only memory, a magnetic or optical disk, or the like.

The above description is only for the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive various changes or substitutions within the technical scope of the present invention, and these should be covered by the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the appended claims.

Claims

1. A method for resource allocation, comprising:

receiving a task request sent by a user side;

2. The method of claim 1, wherein before receiving the task request sent by the user side, the method further comprises:

3. The method of claim 1, wherein according to the task request, invoking a node in an idle state in a physical resource to execute a task carried in the task request comprises:

acquiring the number of tasks carried in the task request;

4. The method of claim 3, wherein invoking the node in the idle state from one or more of the physical resources to perform the task based on the number of tasks comprises:

5. The method of claim 3, wherein invoking the node in the idle state from one or more of the physical resources to perform the task based on the number of tasks comprises:

6. A method for resource allocation, comprising:

a user side sends a task request to a resource center;

7. A resource allocation apparatus, comprising:

the receiving module is used for receiving a task request sent by a user side;

8. The apparatus of claim 7, further comprising:

9. The apparatus of claim 7, wherein the calling module comprises:

10. A resource allocation system, comprising:

a resource center employing the resource configuration apparatus of any one of claims 7-9;

11. A resource allocation terminal, comprising:

one or more processors;

storage means for storing one or more programs;

the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-6.

12. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 6.