CN116382814B - Unified system scheduling adaptation method, device, computer equipment and storage medium - Google Patents
Unified system scheduling adaptation method, device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN116382814B CN116382814B CN202310360269.XA CN202310360269A CN116382814B CN 116382814 B CN116382814 B CN 116382814B CN 202310360269 A CN202310360269 A CN 202310360269A CN 116382814 B CN116382814 B CN 116382814B
- Authority
- CN
- China
- Prior art keywords
- scheduling
- adapter
- result
- information
- assembling
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 230000006978 adaptation Effects 0.000 title claims abstract description 35
- 230000006837 decompression Effects 0.000 claims description 24
- 238000004590 computer program Methods 0.000 claims description 21
- 230000001419 dependent effect Effects 0.000 claims description 7
- 238000012216 screening Methods 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 22
- 230000006870 function Effects 0.000 description 6
- 230000010354 integration Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000012217 deletion Methods 0.000 description 2
- 230000037430 deletion Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000011144 upstream manufacturing Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 235000019800 disodium phosphate Nutrition 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000008140 language development Effects 0.000 description 1
- 238000003032 molecular docking Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/448—Execution paradigms, e.g. implementations of programming paradigms
- G06F9/4488—Object-oriented
- G06F9/449—Object-oriented method invocation or resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/541—Interprogram communication via adapters, e.g. between incompatible applications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
The embodiment of the invention discloses a unified system scheduling adaptation method, a device, computer equipment and a storage medium. The method comprises the following steps: integrating a plurality of scheduling systems; acquiring information of scheduling parameters transmitted by a user to obtain scheduling information; assembling the scheduling information to obtain an assembling result; determining a scheduling adapter; and scheduling a corresponding scheduling system by using the scheduling adapter so as to execute the tasks in the assembly result by using an executor in the scheduling system. By implementing the method of the embodiment of the invention, the user can realize the service use of different dispatching systems by only calling the API interface and transmitting corresponding parameters, and the cost is low and the efficiency is high.
Description
Technical Field
The present invention relates to a computer, and more particularly, to a unified system scheduling adaptation method, apparatus, computer device, and storage medium.
Background
During the process of using various scheduling software, service personnel often need to adapt various scheduling systems so as to integrate external scheduling with own service systems. Although manufacturers of various scheduling systems provide own SDK (software development kit ) or API (application programming interface) calls, developers often start to access one module from zero in the process of docking, so that the development cost is high and the labor is repeated.
Therefore, a new method is needed to be designed, so that the user can realize service use of different dispatching systems by only calling an API interface and inputting corresponding parameters, and the cost is low and the efficiency is high.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide a unified system scheduling adaptation method, a device, computer equipment and a storage medium.
In order to achieve the above purpose, the present invention adopts the following technical scheme: the unified system scheduling adaptation method comprises the following steps:
integrating a plurality of scheduling systems;
acquiring information of scheduling parameters transmitted by a user to obtain scheduling information;
assembling the scheduling information to obtain an assembling result;
determining a scheduling adapter;
and scheduling a corresponding scheduling system by using the scheduling adapter so as to execute the tasks in the assembly result by using an executor in the scheduling system.
The further technical scheme is as follows: the integrated multiple scheduling system includes:
jar or a data packet formed by the package of dependent packets of various dispatching systems are integrated;
setting a plurality of kinds of registrars, wherein each kind of registrar corresponds to one scheduling system;
and writing the registry in a configuration file, and completing Java enumeration of the dispatch adapter type to obtain the dispatch adapter type of the Java enumeration class.
The further technical scheme is as follows: the obtaining the information of the scheduling parameters transmitted by the user to obtain scheduling information includes:
and acquiring information of scheduling parameters transmitted by a user through message queuing or RESTFUL interface so as to obtain scheduling information.
The further technical scheme is as follows: the scheduling parameters include scheduling adapter type, executor type, task details, start time, end time, number of repetitions, and execution interval.
The further technical scheme is as follows: the step of assembling the scheduling information to obtain an assembling result includes:
decompressing the scheduling information to obtain a decompression result;
and performing task scheduling of different types according to the decompression result, assembling tasks in the executor according to task details, and performing time control and control execution interval according to the starting time and the ending time to obtain an assembly result.
The further technical scheme is as follows: the determining a dispatch adapter includes:
and screening the dispatch adapter type consistent with the dispatch adapter type in the decompression result from the dispatch adapter type of the Java enumeration class.
The further technical scheme is as follows: the scheduling, by using the scheduling adapter, a corresponding scheduling system to execute the task in the assembly result by using an executor in the scheduling system includes:
scheduling the corresponding scheduling system by using the scheduling adapter;
and transmitting the assembly result to an adapter so as to utilize the adapter to schedule an executor in the scheduling system to execute the task in the assembly result.
The invention also provides a unified system scheduling adaptation device, which comprises:
the integrated unit is used for integrating various scheduling systems;
the information acquisition unit is used for acquiring the information of the scheduling parameters transmitted by the user so as to obtain scheduling information;
an assembling unit for assembling the scheduling information to obtain an assembling result;
a determining unit configured to determine a scheduling adapter;
and the scheduling unit is used for scheduling a corresponding scheduling system by utilizing the scheduling adapter so as to execute the tasks in the assembly result by utilizing an executor in the scheduling system.
The invention also provides a computer device which comprises a memory and a processor, wherein the memory stores a computer program, and the processor realizes the method when executing the computer program.
The present invention also provides a storage medium storing a computer program which, when executed by a processor, performs the above-described method.
Compared with the prior art, the invention has the beneficial effects that: the invention integrates various dispatching systems, assembles dispatching information with dispatching parameters, determines the dispatching adapter, and calls the corresponding dispatching system by using the dispatching adapter so as to execute tasks in an assembling result by using an executor in the dispatching system, thereby realizing service use of different dispatching systems by only calling an API interface and inputting corresponding parameters.
The invention is further described below with reference to the drawings and specific embodiments.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is an application scenario schematic diagram of a unified system scheduling adaptation method provided by an embodiment of the present invention;
fig. 2 is a flow chart of a unified system scheduling adaptation method according to an embodiment of the present invention;
FIG. 3 is a schematic sub-flowchart of a unified system scheduling adaptation method according to an embodiment of the present invention;
FIG. 4 is a schematic sub-flowchart of a unified system scheduling adaptation method according to an embodiment of the present invention;
FIG. 5 is a schematic sub-flowchart of a unified system scheduling adaptation method according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a random node implementation according to an embodiment of the present invention;
fig. 7 is a schematic diagram of broadcast execution according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a slicing implementation according to an embodiment of the present invention;
FIG. 9 is a schematic block diagram of a unified system scheduling adaptation device provided by an embodiment of the present invention;
FIG. 10 is a schematic block diagram of an integrated unit of a unified system dispatch adapter device provided by an embodiment of the present invention;
FIG. 11 is a schematic block diagram of an assembly unit of a unified system dispatch adapter device provided by an embodiment of the present invention;
FIG. 12 is a schematic block diagram of a scheduling unit of a unified system scheduling adaptation device according to an embodiment of the present invention;
FIG. 13 is a schematic block diagram of a computer device provided by an embodiment of the present invention;
FIG. 14 is a newly created schematic diagram of a workflow provided by an embodiment of the present invention;
FIG. 15 is a diagram illustrating view editing of a workflow according to an embodiment of the present invention
FIG. 16 is a schematic diagram of batch deletion, batch import, batch deletion of workflows provided by embodiments of the present invention;
FIG. 17 is a schematic diagram of a batch operation record of a workflow provided by an embodiment of the present invention;
FIG. 18 is a diagram illustrating the dependency of workload provided by an embodiment of the present invention;
FIG. 19 is a schematic diagram of a batch run operation of a workload provided by an embodiment of the present invention;
FIG. 20 is a schematic diagram of a conditional screening of a workflow provided by an embodiment of the present invention;
FIG. 21 is a schematic diagram of the operation of the workload provided by an embodiment of the present invention;
FIG. 22 is a schematic diagram of batch operations of workflow execution detail provided by an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be understood that the terms "comprises" and "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in the present specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
Referring to fig. 1 and fig. 2, fig. 1 is a schematic application scenario diagram of a unified system scheduling adaptation method according to an embodiment of the present invention. Fig. 2 is a schematic flow chart of a unified system scheduling adaptation method provided by an embodiment of the present invention. The unified system scheduling adaptation method is applied to a server. The server performs data interaction with the terminal, different scheduling systems are arranged on the terminal, the scheduling systems are integrated on the server, information input by a user through API call is utilized for assembly, a scheduling adapter is determined, and the corresponding scheduling system is scheduled by the scheduling adapter so as to execute corresponding tasks by an executor.
Fig. 2 is a flow chart of a unified system scheduling adaptation method according to an embodiment of the present invention. As shown in fig. 2, the method includes the following steps S110 to S150.
S110, integrating various scheduling systems.
In this embodiment, the various scheduling systems include xxl-job, erda-pipeline, dophinScheduler, stack scheduling capabilities, and other scheduling systems.
In one embodiment, referring to fig. 3, the step S110 may include steps S111 to S113.
S111, integrating jar or dependent packets of various scheduling systems to form a data packet;
s112, setting a plurality of kinds of registrars, wherein each kind of registrar corresponds to one kind of scheduling system;
s113, writing the registry in a configuration file, and completing Java enumeration of the dispatch adapter type to obtain the dispatch adapter type of the Java enumeration class.
In this embodiment, the type of the scheduling adapter of the Java enumeration class refers to all types of scheduling adapters obtained by using Java enumeration.
Specifically, jar or dependency package of other scheduling systems is integrated into the server, then a plurality of registries are realized in the server, each registry corresponds to an external scheduling system, the registries are written in configuration files, then Java enumeration class is completed in the program, each time the external terminal is judged to judge the type of scheduling adapter in the Java enumeration class through MQ (Message Queue) or the schedulerType transmitted by RESTFUL, and if the types of the scheduling adapters are equal, the adaptation of the scheduling adapter is performed.
S120, acquiring information of scheduling parameters transmitted by a user to obtain scheduling information.
In this embodiment, the scheduling information refers to a message packet with scheduling parameters including a scheduling adapter type, an executor type, task details, a start time, an end time, a repetition number, and an execution interval.
Specifically, information of scheduling parameters transmitted by a user through message queuing or RESTFUL interface is obtained to obtain scheduling information.
Other apps may invoke methods in the server in two ways. A first message queue, a second http or https restul interface; the message queue support RabbitMQ, kafka, pulsar, rocketMQ only needs to send a message body containing the scheduling parameters when other apps need to schedule tasks, and the module consumes task information internally and analyzes the message body to judge the scheduling information of the tasks to be executed. And triggering the executor according to the scheduling information, and executing corresponding tasks by the executor. And finally, periodically pushing the direct results of the tasks back to the message queue, wherein the mode fully utilizes the decoupling characteristic of the message queue. However, the design is strongly dependent on the message queue, and the expandability, the functions and the system load are greatly related to the message queue; the APP sends the scheduling message body to the server through the MQ, and the server returns the scheduling result to the terminal through the MQ.
The RESTFUL interface supports a standard RESTFUL interface protocol, other APP sends a scheduling parameter JSON through http or https, after receiving the JSON message, the server analyzes corresponding scheduling information and then sends the corresponding scheduling information to a corresponding executor, and the execution result of the executor is synchronously returned to an APP caller through the RESTFUL protocol, so that the whole calling link is completed; specifically, the terminal sends the scheduling message body to the server through the RESTFUL protocol, and the server sends the RESTFUL protocol receiving result to the terminal.
For the scheduling parameters, there are 7 total, respectively, "schedulerType": "scheduling adapter type", for example xxl-job, dophinScheduler, etc., "actuatorType": "actuator type", for example datax, spark, etc., "jobDetail": "task details", is a json body, "startTime": "start time", "endTime": "end time", "repeatCount": "repetition number", "repeatInterval": "execution interval (unit millisecond)". Examples of the parameter samples are shown below
"schedulerType" schedule adapter types, e.g. xxl-job, dophinScheduler etc.,
"actuatorType" actuator types, such as datax, spark, etc.,
"jobDetail" task details, is a json body,
"startTime": "start time",
"endTime": "end time",
"repeat count",
"repeat interval" is "execution interval (in milliseconds)".
S130, assembling the scheduling information to obtain an assembling result.
In this embodiment, the assembly result includes different types of task scheduling, performing time control and controlling execution intervals for tasks in the executor.
In one embodiment, referring to fig. 4, the step S130 may include steps S131 to S132.
S131, decompressing the scheduling information to obtain a decompression result.
In this embodiment, the decompression result refers to a result formed after the scheduling information is unpacked.
And S132, performing task scheduling of different types according to the decompression result, assembling tasks in the executor according to task details, and performing time control and control execution interval according to the starting time and the ending time to obtain an assembly result.
Specifically, the APP compresses the message and transmits the compressed message to the server through an MQ or RESTFUL protocol, the server decompresses the scheduling information, performs different types of task scheduling according to different decompressed parameters, assembles a task job in an executor according to task json details, and finally performs control of starting and ending time and execution interval according to time parameters.
S140, determining a scheduling adapter.
In this embodiment, a dispatch adapter with the same dispatch adapter type as the decompressed result is screened from the dispatch adapter types of the Java enumeration class.
And S150, scheduling a corresponding scheduling system by using the scheduling adapter so as to execute the tasks in the assembly result by using an executor in the scheduling system.
In one embodiment, referring to fig. 5, the step S150 may include steps S151 to S152.
S151, the corresponding dispatching system is dispatched by utilizing the dispatching adapter.
In this embodiment, after parameter adjustment by the scheduling adapter, the upstream scheduling information may be automatically forwarded and routed to the scheduling system with the scheduling parameters already installed and configured on the bottom layer according to the system configuration.
And S152, transmitting the assembly result to an adapter so as to utilize the adapter to schedule an executor in the scheduling system to execute the task in the assembly result.
In this embodiment, the upstream scheduling information is generally passed through the restul protocol with the downstream adapter. The dispatch adapter and executor are invoked via the RPC (remote procedure call ) protocol. The response between the scheduling adapter and the executor is that the scheduling adapter triggers the command, the executor responds to the triggering command; the executor asynchronously executes tasks and feeds back the results to the dispatching adapter, and the dispatching adapter processes callback results; the executor sends the registration command to the dispatching adapter every N seconds, and the dispatching adapter processes the registration command.
Refinement can be divided into three ways of scheduling adapters and executors for direct routing;
as shown in fig. 6, the random node performs: an available execution node in the cluster is selected to execute the scheduled task. The application scene is as follows: and (5) offline order settlement.
As shown in fig. 7, the broadcasting performs: all executing nodes in the cluster distribute the scheduled tasks and execute. The application scene is as follows: the application local cache is updated in batches.
As shown in fig. 8, the slicing performs: splitting according to user-defined slicing logic, distributing to different nodes in a cluster, and executing in parallel, so that the resource utilization efficiency is improved. The application scene is as follows: and (5) carrying out mass log statistics.
In this embodiment, the actuator functions as: is responsible for the execution of a specific scheduling task, for example, if it is a datax task, corresponding data synchronization is performed. If the task is a Python task, carrying out a Python script task; and the task trigger module is responsible for communicating with the scheduling adapter, acquiring scheduling information of the scheduling adapter, selecting a corresponding specific actuator instance according to the actuator routing rule to trigger task starting, and finally sending the task state and the health state of the actuator instance to the scheduling adapter at regular time.
Specifically, the executors of each scheduling system are as follows: xxl-job actuator, pipeline-agent, dophinScheduler-worker.
The method of the embodiment is realized by a fully-encapsulated bottom technology, and is used for interfacing most of scheduling services currently seen in the market, and users can realize service use of different scheduling systems by only calling an API interface and transmitting corresponding parameters.
The method in the embodiment has unified interface configuration management, unified interface scheduling and calling and unified technical stack (java language development in the embodiment), and the bottom layer can be used for interfacing various voices; quick access, simple and easy use. And the method is suitable for various open source scheduling frames, and upper layer business is not perceived.
For example: as shown in fig. 14 to 22, the editing main functions of the workflow include creating the workflow, modifying the workflow, deleting the workflow, saving the workflow, querying the workflow. Wherein each workflow can also be internally added with nodes, modified with nodes and deleted with nodes. The scheduling function of the workflow includes a start workflow, a stop workflow, a set start timing workflow, and a set cancel workflow. Some other ancillary operations of the workflow are also included, including importing the workflow, exporting the workflow, batch start-stop the workflow, and the like.
According to the unified system scheduling adaptation method, various scheduling systems are integrated, scheduling information with scheduling parameters, which is transmitted by a user, is assembled, the scheduling adapter is determined, the corresponding scheduling system is called by the scheduling adapter, tasks in an assembly result are executed by an executor in the scheduling system, service use of different scheduling systems can be achieved by only calling an API interface by the user and transmitting the corresponding parameters, and the cost is low and the efficiency is high.
Fig. 9 is a schematic block diagram of a unified system scheduling adaptation device 300 according to an embodiment of the present invention. As shown in fig. 9, the present invention further provides a unified system scheduling adaptation device 300, corresponding to the above unified system scheduling adaptation method. The unified system scheduling adaptation apparatus 300, which may be configured in a server, comprises means for performing the above-described unified system scheduling adaptation method. Specifically, referring to fig. 9, the unified system dispatch adaptation device 300 includes an integration unit 301, an information acquisition unit 302, an assembly unit 303, a determination unit 304, and a dispatch unit 305.
An integration unit 301 for integrating a plurality of scheduling systems; an information obtaining unit 302, configured to obtain information of a scheduling parameter transmitted by a user, so as to obtain scheduling information; an assembling unit 303, configured to assemble the scheduling information to obtain an assembly result; a determining unit 304, configured to determine a scheduling adapter; and the scheduling unit 305 is configured to schedule a corresponding scheduling system by using the scheduling adapter, so as to execute the tasks in the assembly result by using an executor in the scheduling system.
In one embodiment, as shown in fig. 10, the integration unit 301 includes a packet integration subunit 3011, a registrar setting subunit 3012, and a processing subunit 3013.
A data packet integration subunit 3011, configured to integrate jar of various scheduling systems or data packets formed by packaging dependent packets; a registrar setting subunit 3012, configured to set a plurality of registrars, where each registrar corresponds to a scheduling system; and the processing subunit 3013 is configured to write the registry in the configuration file and complete Java enumeration of the dispatch adapter type to obtain the dispatch adapter type of the Java enumeration class.
In an embodiment, the information obtaining unit 302 is configured to obtain information of the scheduling parameters that are transmitted by the user through the message queue or the restul interface, so as to obtain scheduling information.
In one embodiment, as shown in fig. 11, the assembling unit 303 includes a decompressing subunit 3031 and a content assembling subunit 3032.
A decompression subunit 3031, configured to decompress the scheduling information to obtain a decompression result; and the content assembly subunit 3032 is used for carrying out task scheduling of different types according to the decompression result, then assembling the tasks in the executor according to the task details, and carrying out time control and execution interval control according to the starting time and the ending time so as to obtain an assembly result.
In an embodiment, the determining unit 304 is configured to screen out a schedule adapter type consistent with the schedule adapter type in the decompression result from the schedule adapter types of the Java enumeration class.
In one embodiment, as shown in fig. 12, the scheduling unit 305 includes a system scheduling subunit 3051 and a transmission subunit 3052.
A system scheduling subunit 3051, configured to schedule a corresponding scheduling system by using the scheduling adapter; and the transmission subunit 3052 is configured to transmit the assembly result to an adapter, so as to schedule an executor in the scheduling system to execute the task in the assembly result by using the adapter.
It should be noted that, as will be clearly understood by those skilled in the art, the specific implementation process of the unified system scheduling adaptation device 300 and each unit may refer to the corresponding descriptions in the foregoing method embodiments, and for convenience and brevity of description, the description is omitted here.
The unified system dispatch adaptation means 300 described above may be implemented in the form of a computer program that may be run on a computer device as shown in fig. 13.
Referring to fig. 13, fig. 13 is a schematic block diagram of a computer device according to an embodiment of the present application. The computer device 500 may be a server, where the server may be a stand-alone server or may be a server cluster formed by a plurality of servers.
With reference to FIG. 13, the computer device 500 includes a processor 502, memory, and a network interface 505 connected by a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.
The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform a unified system scheduling adaptation method.
The processor 502 is used to provide computing and control capabilities to support the operation of the overall computer device 500.
The internal memory 504 provides an environment for the execution of a computer program 5032 in the non-volatile storage medium 503, which computer program 5032, when executed by the processor 502, causes the processor 502 to perform a unified system scheduling adaptation method.
The network interface 505 is used for network communication with other devices. It will be appreciated by those skilled in the art that the structure shown in fig. 13 is merely a block diagram of some of the structures associated with the present application and does not constitute a limitation of the computer device 500 to which the present application is applied, and that a particular computer device 500 may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
Wherein the processor 502 is configured to execute a computer program 5032 stored in a memory to implement the steps of:
integrating a plurality of scheduling systems; acquiring information of scheduling parameters transmitted by a user to obtain scheduling information; assembling the scheduling information to obtain an assembling result; determining a scheduling adapter; and scheduling a corresponding scheduling system by using the scheduling adapter so as to execute the tasks in the assembly result by using an executor in the scheduling system.
In one embodiment, when implementing the steps of integrating multiple scheduling systems, the processor 502 specifically implements the following steps:
jar or a data packet formed by the package of dependent packets of various dispatching systems are integrated; setting a plurality of kinds of registrars, wherein each kind of registrar corresponds to one scheduling system; and writing the registry in a configuration file, and completing Java enumeration of the dispatch adapter type to obtain the dispatch adapter type of the Java enumeration class.
In an embodiment, when the step of obtaining the information of the scheduling parameter transmitted by the user to obtain the scheduling information is implemented by the processor 502, the following steps are specifically implemented:
and acquiring information of scheduling parameters transmitted by a user through message queuing or RESTFUL interface so as to obtain scheduling information.
The scheduling parameters comprise a scheduling adapter type, an executor type, task details, a starting time, an ending time, repetition times and an execution interval.
In one embodiment, when the step of assembling the scheduling information to obtain the assembly result is implemented by the processor 502, the following steps are specifically implemented:
decompressing the scheduling information to obtain a decompression result; and performing task scheduling of different types according to the decompression result, assembling tasks in the executor according to task details, and performing time control and control execution interval according to the starting time and the ending time to obtain an assembly result.
In one embodiment, when implementing the step of determining a dispatch adapter, the processor 502 specifically implements the following steps:
and screening the dispatch adapter type consistent with the dispatch adapter type in the decompression result from the dispatch adapter type of the Java enumeration class.
In an embodiment, when implementing the scheduling system that uses the scheduling adapter to schedule the corresponding scheduling system to execute the task step in the assembly result by using an executor in the scheduling system, the processor 502 specifically implements the following steps:
scheduling the corresponding scheduling system by using the scheduling adapter; and transmitting the assembly result to an adapter so as to utilize the adapter to schedule an executor in the scheduling system to execute the task in the assembly result.
It should be appreciated that in embodiments of the present application, the processor 502 may be a central processing unit (Central Processing Unit, CPU), the processor 502 may also be other general purpose processors, digital signal processors (Digital Signal Processor, DSPs), application specific integrated circuits (Application Specific Integrated Circuit, ASICs), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. Wherein the general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Those skilled in the art will appreciate that all or part of the flow in a method embodying the above described embodiments may be accomplished by computer programs instructing the relevant hardware. The computer program comprises program instructions, and the computer program can be stored in a storage medium, which is a computer readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the flow steps of the embodiments of the method described above.
Accordingly, the present invention also provides a storage medium. The storage medium may be a computer readable storage medium. The storage medium stores a computer program which, when executed by a processor, causes the processor to perform the steps of:
integrating a plurality of scheduling systems; acquiring information of scheduling parameters transmitted by a user to obtain scheduling information; assembling the scheduling information to obtain an assembling result; determining a scheduling adapter; and scheduling a corresponding scheduling system by using the scheduling adapter so as to execute the tasks in the assembly result by using an executor in the scheduling system.
In one embodiment, the processor, when executing the computer program to implement the steps of integrating the multiple scheduling systems, specifically implements the steps of:
jar or a data packet formed by the package of dependent packets of various dispatching systems are integrated; setting a plurality of kinds of registrars, wherein each kind of registrar corresponds to one scheduling system; and writing the registry in a configuration file, and completing Java enumeration of the dispatch adapter type to obtain the dispatch adapter type of the Java enumeration class.
In one embodiment, when the processor executes the computer program to obtain the information of the scheduling parameter transmitted by the user, the following steps are specifically implemented:
and acquiring information of scheduling parameters transmitted by a user through message queuing or RESTFUL interface so as to obtain scheduling information.
The scheduling parameters comprise a scheduling adapter type, an executor type, task details, a starting time, an ending time, repetition times and an execution interval.
In one embodiment, when the processor executes the computer program to implement the step of assembling the scheduling information to obtain an assembly result, the following steps are specifically implemented:
decompressing the scheduling information to obtain a decompression result; and performing task scheduling of different types according to the decompression result, assembling tasks in the executor according to task details, and performing time control and control execution interval according to the starting time and the ending time to obtain an assembly result.
In one embodiment, the processor, when executing the computer program to implement the step of determining a dispatch adapter, specifically implements the steps of:
and screening the dispatch adapter type consistent with the dispatch adapter type in the decompression result from the dispatch adapter type of the Java enumeration class.
In one embodiment, when the processor executes the computer program to implement the scheduling system that uses the scheduling adapter to schedule the corresponding scheduling system, the task step in the assembly result is performed by using an executor in the scheduling system, and specifically implement the following steps:
scheduling the corresponding scheduling system by using the scheduling adapter; and transmitting the assembly result to an adapter so as to utilize the adapter to schedule an executor in the scheduling system to execute the task in the assembly result.
The storage medium may be a U-disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, or an optical disk, or other various computer-readable storage media that can store program codes.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps described in connection with the embodiments disclosed herein may be embodied in electronic hardware, in computer software, or in a combination of the two, and that the elements and steps of the examples have been generally described in terms of function in the foregoing description to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of each unit is only one logic function division, and there may be another division manner in actual implementation. For example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed.
The steps in the method of the embodiment of the invention can be sequentially adjusted, combined and deleted according to actual needs. The units in the device of the embodiment of the invention can be combined, divided and deleted according to actual needs. In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.
The integrated unit may be stored in a storage medium if implemented in the form of a software functional unit and sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention is essentially or a part contributing to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a terminal, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention.
While the invention has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.
Claims (7)
1. The unified system scheduling adaptation method is characterized by comprising the following steps:
integrating a plurality of scheduling systems;
acquiring information of scheduling parameters transmitted by a user to obtain scheduling information;
assembling the scheduling information to obtain an assembling result;
determining a scheduling adapter;
scheduling a corresponding scheduling system by using the scheduling adapter so as to execute tasks in the assembly result by using an executor in the scheduling system;
the step of assembling the scheduling information to obtain an assembling result includes:
decompressing the scheduling information to obtain a decompression result;
scheduling different types of tasks according to the decompression result, assembling tasks in the executor according to task details, and performing time control and control execution intervals according to the starting time and the ending time to obtain an assembly result;
the determining a dispatch adapter includes:
screening out a scheduling adapter consistent with the scheduling adapter type in the decompression result from the scheduling adapter types of Java enumeration classes;
the scheduling, by using the scheduling adapter, a corresponding scheduling system to execute the task in the assembly result by using an executor in the scheduling system includes:
scheduling the corresponding scheduling system by using the scheduling adapter;
transmitting the assembly result to an adapter, so that an actuator in the dispatching system is dispatched by the adapter to execute the task in the assembly result;
refinement can be divided into three ways of scheduling adapters and executors for direct routing;
the random node performs: selecting an available execution node in the cluster to execute a scheduling task; the application scene is as follows: offline order settlement;
the broadcast performs: distributing and executing scheduling tasks in all executing nodes in the cluster; the application scene is as follows: updating the application local cache in batches;
slicing and executing: splitting according to user-defined slicing logic, distributing to different nodes in a cluster for parallel execution, and improving the resource utilization efficiency; the application scene is as follows: and (5) carrying out mass log statistics.
2. The unified system scheduling adaptation method according to claim 1, wherein the integrated multiple scheduling systems comprise:
jar or a data packet formed by the package of dependent packets of various dispatching systems are integrated;
setting a plurality of kinds of registrars, wherein each kind of registrar corresponds to one scheduling system;
and writing the registry in a configuration file, and completing Java enumeration of the dispatch adapter type to obtain the dispatch adapter type of the Java enumeration class.
3. The unified system scheduling adaptation method according to claim 1, wherein the obtaining the information of the scheduling parameters entered by the user to obtain the scheduling information comprises:
and acquiring information of scheduling parameters transmitted by a user through message queuing or RESTFUL interface so as to obtain scheduling information.
4. A unified system scheduling adaptation method according to claim 3, wherein the scheduling parameters comprise a scheduling adapter type, an executor type, task details, a start time, an end time, a number of repetitions, and an execution interval.
5. The unified system scheduling adaptation device is characterized by comprising:
the integrated unit is used for integrating various scheduling systems;
the information acquisition unit is used for acquiring the information of the scheduling parameters transmitted by the user so as to obtain scheduling information;
an assembling unit for assembling the scheduling information to obtain an assembling result;
a determining unit configured to determine a scheduling adapter;
the scheduling unit is used for scheduling a corresponding scheduling system by utilizing the scheduling adapter so as to execute tasks in the assembly result by utilizing an executor in the scheduling system;
the determining unit is used for screening out the dispatch adapter type consistent with the dispatch adapter type in the decompression result from the dispatch adapter types of Java enumeration classes;
the assembly unit comprises a decompression subunit and a content assembly subunit;
the decompression subunit is used for decompressing the scheduling information to obtain a decompression result; the content assembly subunit is used for carrying out task scheduling of different types according to the decompression result, then assembling tasks in the executor according to task details, and carrying out time control and control execution interval according to the starting time and the ending time so as to obtain an assembly result;
the scheduling unit comprises a system scheduling subunit and a transmission subunit;
a system scheduling subunit, configured to schedule a corresponding scheduling system using the scheduling adapter; a transmission subunit, configured to transmit the assembly result to an adapter, so that an executor in the scheduling system is scheduled by using the adapter to execute a task in the assembly result; refinement can be divided into three ways of scheduling adapters and executors for direct routing;
the random node performs: selecting an available execution node in the cluster to execute a scheduling task; the application scene is as follows: offline order settlement;
the broadcast performs: distributing and executing scheduling tasks in all executing nodes in the cluster; the application scene is as follows: updating the application local cache in batches;
slicing and executing: splitting according to user-defined slicing logic, distributing to different nodes in a cluster for parallel execution, and improving the resource utilization efficiency; the application scene is as follows: and (5) carrying out mass log statistics.
6. A computer device, characterized in that it comprises a memory on which a computer program is stored and a processor which, when executing the computer program, implements the method according to any of claims 1-4.
7. A storage medium storing a computer program which, when executed by a processor, performs the method of any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310360269.XA CN116382814B (en) | 2023-03-31 | 2023-03-31 | Unified system scheduling adaptation method, device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310360269.XA CN116382814B (en) | 2023-03-31 | 2023-03-31 | Unified system scheduling adaptation method, device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116382814A CN116382814A (en) | 2023-07-04 |
| CN116382814B true CN116382814B (en) | 2024-03-12 |
Family
ID=86965206
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310360269.XA Active CN116382814B (en) | 2023-03-31 | 2023-03-31 | Unified system scheduling adaptation method, device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116382814B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1783123A (en) * | 2004-10-18 | 2006-06-07 | 国际商业机器公司 | Process execution management based on resource requirements and business impacts |
| CN110069329A (en) * | 2019-04-15 | 2019-07-30 | 北京达佳互联信息技术有限公司 | A kind of task processing method, device, server and storage medium |
| CN110336859A (en) * | 2019-06-06 | 2019-10-15 | 广州市玄武无线科技股份有限公司 | Task scheduling system under multi-tenant environment |
| CN110555074A (en) * | 2019-09-11 | 2019-12-10 | 山东浪潮商用系统有限公司 | method for extracting and processing data at fixed time, terminal and computer readable storage medium |
| CN111142977A (en) * | 2019-12-26 | 2020-05-12 | 深圳前海环融联易信息科技服务有限公司 | Processing method and device of timing task, computer equipment and storage medium |
| CN112486648A (en) * | 2020-11-30 | 2021-03-12 | 北京百度网讯科技有限公司 | Task scheduling method, device, system, electronic equipment and storage medium |
| CN112667383A (en) * | 2020-12-31 | 2021-04-16 | 北京高途云集教育科技有限公司 | Task execution and scheduling method, system, device, computing equipment and medium |
| CN114356521A (en) * | 2021-12-30 | 2022-04-15 | 挂号网（杭州）科技有限公司 | Task scheduling method and device, electronic equipment and storage medium |
| CN114416311A (en) * | 2021-12-30 | 2022-04-29 | 中国电信股份有限公司 | Method and device for managing message queue based on Go language |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160224944A1 (en) * | 2015-01-29 | 2016-08-04 | Ravinder Singh TOOR | Systems and methods for event scheduling |
-
2023
- 2023-03-31 CN CN202310360269.XA patent/CN116382814B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1783123A (en) * | 2004-10-18 | 2006-06-07 | 国际商业机器公司 | Process execution management based on resource requirements and business impacts |
| CN110069329A (en) * | 2019-04-15 | 2019-07-30 | 北京达佳互联信息技术有限公司 | A kind of task processing method, device, server and storage medium |
| CN110336859A (en) * | 2019-06-06 | 2019-10-15 | 广州市玄武无线科技股份有限公司 | Task scheduling system under multi-tenant environment |
| CN110555074A (en) * | 2019-09-11 | 2019-12-10 | 山东浪潮商用系统有限公司 | method for extracting and processing data at fixed time, terminal and computer readable storage medium |
| CN111142977A (en) * | 2019-12-26 | 2020-05-12 | 深圳前海环融联易信息科技服务有限公司 | Processing method and device of timing task, computer equipment and storage medium |
| CN112486648A (en) * | 2020-11-30 | 2021-03-12 | 北京百度网讯科技有限公司 | Task scheduling method, device, system, electronic equipment and storage medium |
| CN112667383A (en) * | 2020-12-31 | 2021-04-16 | 北京高途云集教育科技有限公司 | Task execution and scheduling method, system, device, computing equipment and medium |
| CN114356521A (en) * | 2021-12-30 | 2022-04-15 | 挂号网（杭州）科技有限公司 | Task scheduling method and device, electronic equipment and storage medium |
| CN114416311A (en) * | 2021-12-30 | 2022-04-29 | 中国电信股份有限公司 | Method and device for managing message queue based on Go language |
Non-Patent Citations (3)
| Title |
|---|
| Experimental Validation of a Torque-Controlled Variable Stiffness Actuator Tuned by Gain Scheduling;Lin Liu 等;《 IEEE/ASME Transactions on Mechatronics》;20180709;第23卷(第5期);2109 - 2120 * |
| 一种面向多调度系统支持多任务类型的资源调度平台的设计与实现;陈嘉夫;《中国优秀硕士学位论文全文数据库 信息科技辑》;20220415;I138-277 * |
| 算力网络研究进展综述;贾庆民 等;《网络与信息安全学报》;20210531;第7卷(第05期);1-12 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116382814A (en) | 2023-07-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7587638B2 (en) | Method and system for generating and monitoring variable load on an application under test | |
| US9740501B2 (en) | Generating and automatically loading reduced operating system based on usage pattern of applications | |
| US20240064058A1 (en) | Implementation of compliance settings by a mobile device for compliance with a configuration scenario | |
| EP1877918B1 (en) | System and method for scheduling device management | |
| CN109347934B (en) | Message pushing method and system, server, storage medium and electronic device | |
| US20190370080A1 (en) | Streaming traffic pattern for public cloud auto scaling | |
| CN113630372A (en) | Cloud edge coordination system for edge computing | |
| CN105975351A (en) | User behavior message reporting method and apparatus | |
| CN112261101B (en) | Method for introducing rule engine into MQTT message center | |
| CN107066339A (en) | Distributed job manager and distributed job management method | |
| US9106596B2 (en) | Method and apparatus of configuring a data broadcast service | |
| US20210089372A1 (en) | Extensible alerts platform | |
| CN116382814B (en) | Unified system scheduling adaptation method, device, computer equipment and storage medium | |
| US8458716B2 (en) | Enterprise resource planning with asynchronous notifications of background processing events | |
| CN112559461A (en) | File transmission method and device, storage medium and electronic equipment | |
| CN111475315A (en) | Server and subscription notification push control and execution method | |
| CN113934517A (en) | Timing task scheduling method and device, electronic device and storage medium | |
| CN114500638B (en) | Automatic cross-platform based medical data stream pushing system and method | |
| CN110262912B (en) | Calling method and device for procedure call gRPC | |
| CN112445597A (en) | Timing task scheduling method and device | |
| CN113238875A (en) | Queue-based request frequency control system and control method | |
| US10185577B2 (en) | Run-time adaption of external properties controlling operation of applications | |
| CN116866427B (en) | Unified pushing method and system for heterogeneous messages | |
| CN116974730B (en) | Large-batch task processing method | |
| CN114595223A (en) | Data acquisition method and device, storage medium and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |