CN111694666A - Task distribution management method, device, equipment and medium - Google Patents
Task distribution management method, device, equipment and medium Download PDFInfo
- Publication number
- CN111694666A CN111694666A CN202010522290.1A CN202010522290A CN111694666A CN 111694666 A CN111694666 A CN 111694666A CN 202010522290 A CN202010522290 A CN 202010522290A CN 111694666 A CN111694666 A CN 111694666A
- Authority
- CN
- China
- Prior art keywords
- container group
- task
- group copy
- executed
- copy
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000007726 management method Methods 0.000 title claims abstract description 36
- 230000004044 response Effects 0.000 claims abstract description 35
- 230000006378 damage Effects 0.000 claims abstract description 22
- 238000000034 method Methods 0.000 claims description 22
- 230000007246 mechanism Effects 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 5
- 238000011161 development Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the invention discloses a task distribution management method, a device, equipment and a medium. The task distribution management method comprises the following steps: acquiring at least one item of task configuration information matched with a task to be executed, wherein the task configuration information comprises: destroying time at fixed time; creating container group copies in at least one node in a Kubernetes system, and distributing tasks to be executed to each container group copy for execution; and when a task ending response fed back by each container group copy is acquired, timing for each container group copy is started, and when the timing is up to the timing destruction time, the corresponding container group copy is destroyed. According to the technical scheme of the embodiment of the invention, the Broadcast Job resource object allocates the tasks to the container group copy, and destroys the container group copy after the tasks are completed, so that Kubernets resources can be saved.
Description
Technical Field
Embodiments of the present invention relate to computer technologies, and in particular, to a method, an apparatus, a device, and a medium for task distribution management.
Background
With the rapid development of the internet technology, the financial industry gradually develops towards the direction of internet, digitalization, mobility, automation and intellectualization, and in the face of the development direction, the verification period of the on-line mode of the traditional financial science and technology development is long, the change of the application architecture is difficult, and the operation and maintenance mode of the traditional manual script meets the challenge.
In the prior art, a DaemonSet object in kubernets is adopted to ensure that one Pod copy runs on all nodes, so that tasks such as log collection, system monitoring and cluster storage are executed.
Disclosure of Invention
Embodiments of the present invention provide a method, an apparatus, a device, and a medium for task distribution management, where a Broadcast Job resource object allocates a task to a container group copy, and destroys the container group copy after the task is completed, so as to save kubernets resources.
In a first aspect, an embodiment of the present invention provides a task distribution management method, where the method includes:
acquiring at least one item of task configuration information matched with a task to be executed, wherein the task configuration information comprises: destroying time at fixed time;
creating container group copies in at least one node in a Kubernetes system, and distributing the tasks to be executed to each container group copy for execution;
and when a task ending response fed back by each container group copy is acquired, timing is started for each container group copy, and when the timing is up to the timing destruction time, the corresponding container group copy is destroyed.
In a second aspect, an embodiment of the present invention further provides a task distribution management apparatus, where the apparatus includes:
the task configuration information acquisition module is used for acquiring at least one item of task configuration information matched with a task to be executed, and the task configuration information comprises: destroying time at fixed time;
the system comprises a to-be-executed task distribution module, a task execution module and a task execution module, wherein the to-be-executed task distribution module is used for creating container group copies in at least one node in a Kubernets system and distributing the to-be-executed tasks to each container group copy for execution;
and the container group copy destroying module is used for starting to time each container group copy when a task ending response fed back by each container group copy is acquired, and destroying the corresponding container group copy when the time is counted to the timed destroying time.
In a third aspect, an embodiment of the present invention further provides an electronic device, including:
one or more processors;
a memory for storing one or more programs;
when the one or more programs are executed by the one or more processors, the one or more processors implement the task distribution management method provided by any embodiment of the present invention.
In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the task distribution management method provided in any embodiment of the present invention.
According to the technical scheme of the embodiment of the invention, the Broadcast Job acquires at least one item of task configuration information matched with the task to be executed, and the task configuration information comprises the following steps: the method comprises the steps of destroying time at fixed time, creating container group copies in at least one node in a Kubernetes system, distributing tasks to be executed to each container group copy for execution, starting to time each container group copy when a task ending response fed back by each container group copy is obtained, destroying the corresponding container group copy when the time is up to the fixed time, destroying the container group copies after the task is completed, and saving Kubernetes resources because the container group copies do not run on a server for a long time.
Drawings
FIG. 1a is a flowchart of a task distribution management method according to a first embodiment of the present invention;
FIG. 1b is a schematic diagram of container group copy creation in the first embodiment of the present invention;
fig. 2 is a flowchart of a task distribution management method in the second embodiment of the present invention;
fig. 3 is a flowchart of a task distribution management method in the third embodiment of the present invention;
fig. 4 is a schematic structural diagram of a task distribution management apparatus in a fourth embodiment of the present invention;
fig. 5 is a schematic structural diagram of an apparatus according to a fifth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1a is a flowchart of a task distribution management method in an embodiment of the present invention, where the technical solution of this embodiment is suitable for a case where a corresponding container group is destroyed after a task is executed, and the method may be executed by a task distribution management device, and the device may be implemented by software and/or hardware, and may be integrated in various general-purpose computer devices, and specifically includes the following steps:
The task configuration information is specific parameters configured for a core resource predefined in the Broadcast Job, for the current task to be executed, and exemplarily, the core resource predefined in the Broadcast Job includes a parallelism field, where the field represents the number of maximum container group copies that can be simultaneously run, and for the specific task to be executed, the field may be assigned with a value, for example, the value is 2, which indicates that at most 2 container group copies can be simultaneously run, and the value is one item of the task configuration information.
In this embodiment, a plurality of task configuration information matched with the current task to be executed is acquired by the Broadcast Job resource object, where the task configuration information includes a timing destruction time, and illustratively, when the task to be executed is a timing container task, the user inputs the task configuration information matched with the timing container task in advance, for example, in order to save resources, the task configuration information is set to destroy the container group copy after the task is executed for 30 seconds, at this time, the task configuration information may be the timing destruction time, and the subsequent Broadcast Job resource object may destroy the container group copy according to the timing destruction time input by the user. Certainly, the task configuration information may include multiple items, and may also be a response type when the task fails to be executed, where the task configuration information may specify that the container group copy is destroyed when the task fails to be executed, or that at least one retry is performed, and the container group copy is destroyed after the retry fails.
Optionally, the Broadcast Job resource object is obtained by defining a resource set in the initialized Broadcast Job resource object after being initialized and created by the custom resource type CRD;
the resource set comprises: at least one resource field name and a data item to be filled corresponding to each resource field name;
the data item to be populated is associated with task configuration information in the Broadcast Job resource object.
In this optional embodiment, a declaration manner of the Broadcast Job resource object is provided, and specifically, the Broadcast Job resource is defined by a user-defined resource type CRD, and then a resource set of the Broadcast Job resource object is defined, where the resource set includes at least one resource field name and a to-be-filled data item corresponding to each resource field name, the resource field name is a spec declaration field of the Broadcast Job in kubernets, and the to-be-filled data item is associated with task configuration information in the Broadcast Job resource object, that is, the to-be-filled data item is filled for a specific to-be-executed task.
Illustratively, the at least one resource field name in the resource set includes at least one resource field name of template, used, parallelisms, failurePolicy, and completionPolicy, and the meaning and type of the field corresponding to each resource field name are shown in table 1.
TABLE 1
And step 120, creating container group copies in at least one node in the Kubernets system, and distributing the tasks to be executed to each container group copy for execution.
In this embodiment, after acquiring the task configuration information matched with the task to be executed, the Broadcast Job creates a container group copy in at least one node in the kubernets system, as shown in fig. 1b, specifically, a container group copy may be created on each node, or a container group copy matched with the content of a lab field may be created according to the preset lab field, and after the container group copy is created, the task to be executed is distributed to each container group copy for execution. The tasks to be executed may be tasks that need to be executed by at least one container group copy, such as database backup, log collection, system monitoring, and the like.
Illustratively, the task currently to be performed is database backup, and each node in the kubernets system of Broadcast Job runs a container group copy, that is, each node runs a container group copy to perform a database backup operation. The task execution state of the container group replica can be obtained on each node, for example, by inputting a kubecect broadcastjobs statement, the state table 2 of each container group replica can be obtained as shown in:
TABLE 2
| NAME | DESIRED | ACTIVE | SUCCEEDED | FAILED |
| Broadcastjob-sample | 1 | 0 | 1 | 0 |
The Desired number of container group copies is indicated in the Desired field, Active is the number of currently running container group copies, succeneded is the number of container group copies that have completed the task, and Failed is the number of container group copies that have Failed to execute the task.
Step 130, when a task ending response fed back by each container group copy is obtained, timing is started for each container group copy, and when the timing is up to the timing destruction time, the corresponding container group copy is destroyed.
In this embodiment, after acquiring a task end response fed back by at least one container group copy, the BroadcastJob resource object performs timed destruction of the container group copy according to the timed destruction time included in the task configuration information, specifically, when the task ends, timing is started, and when the timed destruction time is reached, the container group copy is destroyed.
Illustratively, if the timed destruction time is 30 seconds, the Broadcast Job starts timing when acquiring a task end response fed back by a certain container group copy, and destroys the container group copy after timing for 30 seconds, and at this time, the container group copy does not occupy kubernets resources for a long time.
According to the technical scheme of the embodiment of the invention, the Broadcast Job acquires at least one item of task configuration information matched with the task to be executed, and the task configuration information comprises the following steps: the method comprises the steps of destroying time at fixed time, creating container group copies in at least one node in a Kubernetes system, distributing tasks to be executed to each container group copy for execution, starting to time each container group copy when a task ending response fed back by each container group copy is obtained, destroying the corresponding container group copy when the time is up to the fixed time, destroying the container group copies after the task is completed, and saving Kubernetes resources because the container group copies do not run on a server for a long time.
Example two
Fig. 2 is a flowchart of a task distribution management method in a second embodiment of the present invention, and this embodiment provides specific steps of creating a container group copy in at least one node in a kubernets system based on further details of the above embodiment. A task distribution management method provided by the second embodiment of the present invention is described below with reference to fig. 2, including the following steps:
Optionally, the task configuration information further includes: template information for container group copies.
In this optional embodiment, another item of information included in the task configuration information is provided, and is template information of the container group copy, and this information may be used to construct the container group copy.
And step 220, in at least one node in the Kubernets system, creating container group copies according to the template information of the container group copies, and distributing the tasks to be executed to the container group copies for execution.
In this embodiment, after the Broadcast Job acquires the at least one item of task configuration information matched with the task to be executed, a container group copy is created in at least one node in the kubernets system according to the template information of the container group copy, and then the task to be executed is distributed to each container group copy for execution. Wherein the template information of the container group copy may define a series of rules or the like for creating the container group copy.
In another specific example of the embodiment of the present disclosure, the task configuration information further includes: a maximum number of container group copies that are running simultaneously;
creating a container group copy in at least one node in a Kubernets system, specifically comprising:
calculating the ideal number of container group copies according to the task amount of the task to be executed;
creating a container group copy in each of an ideal number of nodes in a kubernets system if the ideal number does not exceed the maximum number;
if the ideal number exceeds the maximum number, a container group copy is created in each of the maximum number of nodes in the Kubernets system.
In this specific example, another item of task configuration information is provided, that is, the maximum number of container group copies that run simultaneously, the data type of the task configuration information may be defined as integer, for example, the maximum number of container group copies that run simultaneously is defined as 3, at this time, container group copies may be created according to the maximum number, specifically, firstly, according to the task amount of the task to be executed, an ideal number of container group copies that are needed is determined, and if the ideal number does not exceed the maximum number defined by the user, one container group copy is created in each of the ideal number of nodes of the kubernets system, that is, an ideal number of container group copies are created; if the ideal number exceeds the maximum number, one container group copy is created in each of the maximum number of nodes in the kubernets system, that is, the maximum number of container group copies is created.
Illustratively, the maximum number of container group copies to be run simultaneously is defined to be 3, when the number of container group copies to be run is calculated to be 2 according to the task amount of a task to be executed, 2 container group copies can be directly created, and when the number of container group copies to be run is calculated to be 4, only 3 container group copies can be created.
In another specific example of the embodiment of the present disclosure, the task configuration information further includes: a container group label;
creating a container group copy in at least one node in a Kubernets system, specifically comprising:
in at least one node in the kubernets system, a container group copy is created that matches the container group tag.
In this specific example, the task configuration information may further include a container group tag, that is, a label field of the container group, where this field may be used as an identifier of the container group, and when creating a container group copy in the kubernets system, a container group copy matching the container group tag may be created according to a preset label field value.
And step 230, when a task ending response fed back by each container group copy is acquired, timing is started for each container group copy, and when the timing is up to the timing destruction time, the corresponding container group copy is destroyed.
According to the technical scheme of the embodiment, the Broadcast Job acquires at least one item of task configuration information matched with the task to be executed, and the task configuration information comprises: and (2) destroying time at fixed time, creating container group copies according to template information of the container group copies in at least one node in the Kubernets system, distributing tasks to be executed to each container group copy for execution, starting to time each container group copy when a task ending response fed back by each container group copy is obtained, destroying the corresponding container group copies when the timed destroying time is up, and destroying the container group copies within the set time after the task is completed, wherein the container group copies do not occupy Kubernets resources for a long time.
EXAMPLE III
Fig. 3 is a flowchart of a task distribution management method in a third embodiment of the present invention, and this embodiment is further detailed on the basis of the foregoing embodiment, and provides specific steps after tasks to be executed are distributed to copies of each container group for execution. A task distribution management method provided by a third embodiment of the present invention is described below with reference to fig. 3, including the following steps:
Optionally, the task configuration information further includes: type of response when task fails.
In this optional embodiment, another item of information included in the task configuration information is provided, and is a response type when the task fails, where the information is used to specify a specific response operation after the container group copy executes the task failure. For example, when the container group fails to execute the task, the container group copy may be destroyed directly, or retry operation may be performed for a set number of times, and the container group copy is destroyed after the retry fails.
And step 320, creating container group copies in at least one node in the Kubernets system, and distributing the tasks to be executed to each container group copy for execution.
And 330, when the execution failure of the second target container group copy task is detected, executing a failure response mechanism matched with the response type when the task fails.
In this embodiment, when the Broadcast Job resource object detects that the task execution of the second target container group copy fails, a failure response mechanism matching the user-defined response type when the task fails is executed. For example, after detecting that the task execution of the second target container group copy fails, the task execution of the container group copy is not directly marked, but the container group copy is instructed to retry, and when the execution fails for more than 3 times, the task execution failure of the container group copy can be marked, or the container group copy can be directly destroyed.
In another specific example of the embodiment of the present disclosure, the task configuration information further includes: the pause control information comprises pause enable or pause disable;
after distributing the task to be executed to each container group copy for execution, the method further comprises the following steps:
inquiring pause enabling information according to the detected pause instruction matched with the first target container group copy;
if the pause enabling information is pause enabling, indicating the first target container group copy to pause the distributed task;
and if the pause enabling information is pause forbidding, ignoring the pause instruction.
In this specific example, the task configuration information may further include pause control information, which is of a boolean type and includes two states of pause enable and pause disable. When a pause instruction matched with the first target container group is detected, the state of pause enabling information is inquired firstly, if the pause enabling information is in a pause enabling state, the first target container group copy can be indicated to pause the currently executed task, and if the pause enabling information is in a pause forbidding state, the pause instruction is directly ignored, and the currently executed task is not paused.
And 340, starting to time the container group copies when a task ending response fed back by the container group copies is acquired, and destroying the corresponding container group copies when the time is timed to the timed destruction time.
According to the technical scheme of the embodiment, the Broadcast Job acquires at least one item of task configuration information matched with the task to be executed, and the task configuration information comprises: the method comprises the steps of destroying time at fixed time, creating container group copies in at least one node in a Kubernetes system, distributing tasks to be executed to each container group copy for execution, executing a failure response mechanism matched with a response type when a task fails when the task execution failure of a certain container group copy is detected, starting timing for each container group copy when a task end response fed back by each container group copy is obtained, destroying the corresponding container group copy when the timing is up to the fixed destruction time, and the container group copy can be destroyed within the set time after the task is completed, so that the container group copy cannot run for a long time, and Kubernetes resources are saved.
Example four
Fig. 4 is a schematic structural diagram of a task distribution management apparatus according to a fourth embodiment of the present invention, where the task distribution management apparatus includes: the system comprises a task configuration information acquisition module 410, a to-be-executed task distribution module 420 and a container group copy destruction module 430.
A task configuration information obtaining module 410, configured to obtain at least one item of task configuration information matched with a task to be executed, where the task configuration information includes: destroying time at fixed time;
a to-be-executed task distribution module 420, configured to create a container group copy in at least one node in a kubernets system, and distribute the to-be-executed task to each container group copy for execution;
the container group copy destruction module 430 is configured to start timing for each container group copy when a task end response fed back by each container group copy is acquired, and destroy the corresponding container group copy when the timing is up to the time for destroying.
According to the technical scheme of the embodiment of the invention, the Broadcast Job acquires at least one item of task configuration information matched with the task to be executed, and the task configuration information comprises the following steps: the method comprises the steps of destroying time at fixed time, creating container group copies in at least one node in a Kubernetes system, distributing tasks to be executed to each container group copy for execution, starting to time each container group copy when a task ending response fed back by each container group copy is obtained, destroying the corresponding container group copy when the time is up to the fixed time, destroying the container group copies after the task is completed, and saving Kubernetes resources because the container group copies do not run on a server for a long time.
Optionally, the Broadcast Job resource object is obtained by defining a resource set in the initialized Broadcast Job resource object after being initialized and created by the custom resource type CRD;
the resource set comprises: at least one resource field name and a data item to be filled corresponding to each resource field name;
the data item to be populated is associated with task configuration information in the Broadcast Job resource object.
Optionally, the task configuration information further includes: template information of container group copies;
accordingly, the to-be-executed task distributing module 420 includes:
a first container group copy creating unit, configured to create, in at least one node in a kubernets system, a container group copy according to template information of the container group copy.
Optionally, the task configuration information further includes: a maximum number of container group copies that are running simultaneously;
correspondingly, the to-be-executed task distributing module 420 further includes:
the quantity calculating unit is used for calculating the ideal quantity of the container group copies according to the task quantity of the tasks to be executed;
a second container group copy creating unit configured to create one container group copy in each of an ideal number of nodes in the kubernets system if the ideal number does not exceed the maximum number;
a third container group copy creating unit configured to create one container group copy in each of the maximum number of nodes in the kubernets system if the ideal number exceeds the maximum number.
Optionally, the task configuration information further includes: pause control information, the pause control information including pause enable or pause disable;
correspondingly, the task distribution management device further includes:
the enabling information query module is used for querying pause enabling information according to a detected pause instruction matched with the first target container group copy after the task to be executed is distributed to each container group copy to be executed;
a pause indication module, configured to indicate the first target container group copy to pause the distributed task if the pause enable information is pause enable;
and the instruction ignoring module is used for ignoring the pause instruction if the pause enabling information is pause prohibition.
Optionally, the task configuration information further includes: a type of response when the task fails;
correspondingly, the task distribution management device further includes:
and the identification response mechanism execution module is used for executing a failure response mechanism matched with the response type when the task execution failure of the second target container group copy is detected after the task to be executed is distributed to each container group copy to be executed.
Optionally, the task configuration information further includes: a container group label;
correspondingly, the to-be-executed task distributing module 420 further includes:
a fourth container group copy creating unit, configured to create, in at least one node in the kubernets system, a container group copy matching the container group tag.
The task distribution management device provided by the embodiment of the invention can execute the task distribution management method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
EXAMPLE five
Fig. 5 is a schematic structural diagram of an electronic device according to a fifth embodiment of the present invention, as shown in fig. 5, the electronic device includes a processor 50 and a memory 51; the number of processors 50 in the device may be one or more, and one processor 50 is taken as an example in fig. 5; the processor 50 and the memory 51 in the device may be connected by a bus or other means, as exemplified by the bus connection in fig. 5.
The memory 51 is used as a computer-readable storage medium, and can be used for storing software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to a task distribution management method in the embodiment of the present invention (for example, the task configuration information obtaining module 410, the to-be-executed task distribution module 420, and the container group copy destruction module 430 in the task distribution management apparatus). The processor 50 executes various functional applications of the device and data processing by executing software programs, instructions, and modules stored in the memory 51, that is, implements the task distribution management method described above.
The method comprises the following steps:
acquiring at least one item of task configuration information matched with a task to be executed, wherein the task configuration information comprises: destroying time at fixed time;
creating container group copies in at least one node in a Kubernetes system, and distributing the tasks to be executed to each container group copy for execution;
and when a task ending response fed back by each container group copy is acquired, timing is started for each container group copy, and when the timing is up to the timing destruction time, the corresponding container group copy is destroyed.
The memory 51 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the memory 51 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the memory 51 may further include memory located remotely from the processor 50, which may be connected to the device over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
EXAMPLE six
An embodiment of the present invention further provides a computer-readable storage medium having stored thereon a computer program, which when executed by a computer processor is configured to perform a task distribution management method, the method including:
acquiring at least one item of task configuration information matched with a task to be executed, wherein the task configuration information comprises: destroying time at fixed time;
creating container group copies in at least one node in a Kubernetes system, and distributing the tasks to be executed to each container group copy for execution;
and when a task ending response fed back by each container group copy is acquired, timing is started for each container group copy, and when the timing is up to the timing destruction time, the corresponding container group copy is destroyed.
Of course, the storage medium provided by the embodiment of the present invention and containing the computer-executable instructions is not limited to the method operations described above, and may also perform related operations in the task distribution management method provided by any embodiment of the present invention.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, an application server, or a network device) to execute the methods according to the embodiments of the present invention.
It should be noted that, in the embodiment of the task distribution management apparatus, the included units and modules are only divided according to functional logic, but are not limited to the above division as long as the corresponding functions can be implemented; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A task distribution management method, performed by a Broadcast Job resource object, comprising:
acquiring at least one item of task configuration information matched with a task to be executed, wherein the task configuration information comprises: destroying time at fixed time;
creating container group copies in at least one node in a Kubernetes system, and distributing the tasks to be executed to each container group copy for execution;
and when a task ending response fed back by each container group copy is acquired, timing is started for each container group copy, and when the timing is up to the timing destruction time, the corresponding container group copy is destroyed.
2. The method as claimed in claim 1, wherein the Broadcast Job resource object is obtained by defining a resource set in the initialized Broadcast Job resource object after being initialized and created by the initialization of the custom resource type CRD;
the resource set comprises: at least one resource field name and a data item to be filled corresponding to each resource field name;
the data item to be populated is associated with task configuration information in the Broadcast Job resource object.
3. The method according to claim 1 or 2, wherein the task configuration information further comprises: template information of container group copies;
creating a container group copy in at least one node in a Kubernets system, specifically comprising:
and in at least one node in the Kubernets system, creating the container group copy according to the template information of the container group copy.
4. The method according to claim 1 or 2, wherein the task configuration information further comprises: a maximum number of container group copies that are running simultaneously;
creating a container group copy in at least one node in a Kubernets system, specifically comprising:
calculating the ideal number of container group copies according to the task amount of the task to be executed;
creating a container group copy in each of an ideal number of nodes in a kubernets system if the ideal number does not exceed the maximum number;
if the ideal number exceeds the maximum number, a container group copy is created in each of the maximum number of nodes in the Kubernets system.
5. The method according to claim 1 or 2, wherein the task configuration information further comprises: pause control information, the pause control information including pause enable or pause disable;
after the task to be executed is distributed to each container group copy for execution, the method further comprises the following steps:
inquiring pause enabling information according to the detected pause instruction matched with the first target container group copy;
if the pause enabling information is pause enabling, indicating the first target container group copy to pause the distributed task;
and if the pause enabling information is pause forbidding, ignoring the pause instruction.
6. The method according to claim 1 or 2, wherein the task configuration information further comprises: a type of response when the task fails;
after the task to be executed is distributed to each container group copy for execution, the method further comprises the following steps:
and when the execution failure of the second target container group copy task is detected, executing a failure response mechanism matched with the response type when the task fails.
7. The method according to claim 1 or 2, wherein the task configuration information further comprises: a container group label;
creating a container group copy in at least one node in a Kubernets system, specifically comprising:
in at least one node in the kubernets system, a container group copy is created that matches the container group tag.
8. A task distribution management apparatus, characterized by comprising:
the task configuration information acquisition module is used for acquiring at least one item of task configuration information matched with a task to be executed, and the task configuration information comprises: destroying time at fixed time;
the system comprises a to-be-executed task distribution module, a task execution module and a task execution module, wherein the to-be-executed task distribution module is used for creating container group copies in at least one node in a Kubernets system and distributing the to-be-executed tasks to each container group copy for execution;
and the container group copy destroying module is used for starting to time each container group copy when a task ending response fed back by each container group copy is acquired, and destroying the corresponding container group copy when the time is counted to the timed destroying time.
9. An electronic device, characterized in that the device comprises:
one or more processors;
a memory for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement a task distribution management method as recited in any of claims 1-7.
10. A computer-readable storage medium on which a computer program is stored, the program, when being executed by a processor, implementing a task distribution management method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010522290.1A CN111694666B (en) | 2020-06-10 | 2020-06-10 | Task distribution management method, device, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010522290.1A CN111694666B (en) | 2020-06-10 | 2020-06-10 | Task distribution management method, device, equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111694666A true CN111694666A (en) | 2020-09-22 |
| CN111694666B CN111694666B (en) | 2023-11-07 |
Family
ID=72480112
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010522290.1A Active CN111694666B (en) | 2020-06-10 | 2020-06-10 | Task distribution management method, device, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111694666B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112099957A (en) * | 2020-11-02 | 2020-12-18 | 腾讯科技（深圳）有限公司 | Resource allocation method, system, device and electronic equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108965485A (en) * | 2018-09-30 | 2018-12-07 | 北京金山云网络技术有限公司 | Management method, device and the cloud platform of container resource |
| CN109542605A (en) * | 2018-11-27 | 2019-03-29 | 长沙智擎信息技术有限公司 | A kind of container group life cycle management method based on Kubernetes system architecture |
| CN110489239A (en) * | 2019-08-22 | 2019-11-22 | 中国工商银行股份有限公司 | A kind of Container Management method, device and equipment |
| US20190391844A1 (en) * | 2018-11-06 | 2019-12-26 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Task orchestration method and system |
| CN110764896A (en) * | 2018-07-25 | 2020-02-07 | 北京京东金融科技控股有限公司 | Resource allocation method, system, computer system and computer-readable storage medium |
| CN110888714A (en) * | 2019-11-26 | 2020-03-17 | 北京京东尚科信息技术有限公司 | Container scheduling method, device and computer-readable storage medium |
| CN110888743A (en) * | 2019-11-27 | 2020-03-17 | 中科曙光国际信息产业有限公司 | GPU resource using method, device and storage medium |
| CN111245900A (en) * | 2019-12-31 | 2020-06-05 | 北京健康之家科技有限公司 | Distributed message sending processing system and processing method thereof |
-
2020
- 2020-06-10 CN CN202010522290.1A patent/CN111694666B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110764896A (en) * | 2018-07-25 | 2020-02-07 | 北京京东金融科技控股有限公司 | Resource allocation method, system, computer system and computer-readable storage medium |
| CN108965485A (en) * | 2018-09-30 | 2018-12-07 | 北京金山云网络技术有限公司 | Management method, device and the cloud platform of container resource |
| US20190391844A1 (en) * | 2018-11-06 | 2019-12-26 | Beijing Baidu Netcom Science And Technology Co., Ltd. | Task orchestration method and system |
| CN109542605A (en) * | 2018-11-27 | 2019-03-29 | 长沙智擎信息技术有限公司 | A kind of container group life cycle management method based on Kubernetes system architecture |
| CN110489239A (en) * | 2019-08-22 | 2019-11-22 | 中国工商银行股份有限公司 | A kind of Container Management method, device and equipment |
| CN110888714A (en) * | 2019-11-26 | 2020-03-17 | 北京京东尚科信息技术有限公司 | Container scheduling method, device and computer-readable storage medium |
| CN110888743A (en) * | 2019-11-27 | 2020-03-17 | 中科曙光国际信息产业有限公司 | GPU resource using method, device and storage medium |
| CN111245900A (en) * | 2019-12-31 | 2020-06-05 | 北京健康之家科技有限公司 | Distributed message sending processing system and processing method thereof |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112099957A (en) * | 2020-11-02 | 2020-12-18 | 腾讯科技（深圳）有限公司 | Resource allocation method, system, device and electronic equipment |
| CN112099957B (en) * | 2020-11-02 | 2021-04-13 | 腾讯科技（深圳）有限公司 | Resource allocation method, system, device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111694666B (en) | 2023-11-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110502546B (en) | Data processing method and device | |
| US9467460B1 (en) | Modularized database architecture using vertical partitioning for a state machine | |
| US11250166B2 (en) | Fingerprint-based configuration typing and classification | |
| CN111125013B (en) | Data warehousing method, device, equipment and medium | |
| CN112905323B (en) | Data processing method, device, electronic equipment and storage medium | |
| CN113626241B (en) | Abnormality processing method, device, equipment and storage medium for application program | |
| CN104794114A (en) | Data processing method and device | |
| CN114327374A (en) | Business process generation method and device and computer equipment | |
| CN102201922B (en) | Data charging method and relevant apparatus | |
| CN111694666A (en) | Task distribution management method, device, equipment and medium | |
| CN111126058A (en) | Text information automatic extraction method and device, readable storage medium and electronic equipment | |
| CN110750362A (en) | Method and apparatus for analyzing biological information, and storage medium | |
| CN110674383A (en) | Public opinion query method, device and equipment | |
| CN114401239A (en) | Metadata transmission method and device, computer equipment and storage medium | |
| CN111026517B (en) | Task decomposition device and task scheduler | |
| CN111488490A (en) | Video clustering method, device, server and storage medium | |
| CN108206933B (en) | Video data acquisition method and device based on video cloud storage system | |
| CN109408035B (en) | Flow configuration method, storage medium and server of business system | |
| CN115878924B (en) | Data processing method, device, medium and electronic equipment based on double dictionary trees | |
| CN112988457B (en) | Data backup method, device, system and computer equipment | |
| CN114598588B (en) | Server fault determination method and device and terminal equipment | |
| CN113177157B (en) | Label separation method, device, equipment and storage medium | |
| CN114327879B (en) | Extraction method and device for OS kernel data object and storage medium | |
| CN111309572B (en) | Test analysis method and device, computer equipment and storage medium | |
| CN108280048B (en) | Information processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220915 Address after: 25 Financial Street, Xicheng District, Beijing 100033 Applicant after: CHINA CONSTRUCTION BANK Corp. Address before: 25 Financial Street, Xicheng District, Beijing 100033 Applicant before: CHINA CONSTRUCTION BANK Corp. Applicant before: Jianxin Financial Science and Technology Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |