WO2023102869A1 - 任务管理系统、方法、装置、设备及存储介质 - Google Patents

任务管理系统、方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2023102869A1
WO2023102869A1 PCT/CN2021/136975 CN2021136975W WO2023102869A1 WO 2023102869 A1 WO2023102869 A1 WO 2023102869A1 CN 2021136975 W CN2021136975 W CN 2021136975W WO 2023102869 A1 WO2023102869 A1 WO 2023102869A1
Authority
WO
WIPO (PCT)
Prior art keywords
task
executed
tasks
execution
interface
Prior art date
Application number
PCT/CN2021/136975
Other languages
English (en)
French (fr)
Inventor
姜子麒
范陕珊
刘阳
谈樑
Original Assignee
上海智药科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海智药科技有限公司 filed Critical 上海智药科技有限公司
Priority to PCT/CN2021/136975 priority Critical patent/WO2023102869A1/zh
Publication of WO2023102869A1 publication Critical patent/WO2023102869A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines

Definitions

  • the present application relates to the technical field of computer systems, and in particular to a task management system, method, device, device and storage medium.
  • the first aspect of the present application provides a task management system, the task management system includes:
  • the interface method layer is used to provide an interface connected to the human-computer interaction interface, and based on the interface, receives the task management operation input by the user through the human-computer interaction interface, and determines the task to be executed;
  • the parsing layer is used to determine the workflow description language of the task to be executed, parse the workflow description language to obtain the execution sequence of the task to be executed, and provide a task scheduling interface for scheduling the task to be executed , providing a task storage interface for storing the task to be executed;
  • the execution layer is configured to provide computing resources for executing the tasks to be executed and storage resources for storing the tasks to be executed, provide a task execution shell program to download and execute the data of the tasks to be executed, and After the execution is completed, the execution result will be uploaded.
  • the task includes a first-level task, a second-level task, and a third-level task, wherein each of the first-level tasks includes one or more of the second-level tasks, Each of the two-level tasks includes one or more of the three-level tasks, and the interface method layer includes:
  • the first-level task interface module is used to provide the management query service of the first-level task
  • the secondary task interface module is used to provide the management query service of the secondary task
  • the third-level task interface module is used to provide management query services for three-level tasks, wherein the management query services include task submission services, task status query services, task rerun services, and task stop services.
  • the parsing layer determines the workflow description language of the task to be executed, and parses the workflow description language to obtain the When executing a sequence, it can be used to:
  • For a to-be-executed first-level task analyze the workflow description language of the to-be-executed first-level task, determine the execution sequence of each to-be-executed second-level task in the to-be-executed first-level task, and each to-be-executed second-level task dependencies; and/or
  • the execution order of the tertiary tasks to be executed in the secondary tasks to be executed and the dependencies among the tertiary tasks to be executed are determined.
  • the parsing layer includes:
  • a workflow parser configured to analyze the workflow description language of the tasks to be executed, determine the order of the tasks to be executed, the dependencies between the tasks, and the parameter serialization method and parameter deserialization method;
  • a task scheduling interface encapsulates the minimum general requirements for task scheduling and execution, and is used to schedule the tasks to be executed, and the minimum general requirements for task scheduling and execution are used to regulate each computing platform to be included in the tasks Functional requirements of the management system;
  • a task storage interface encapsulates the minimum general requirements for task storage and execution, and is used to store the tasks to be executed, and the minimum general requirements for task storage and execution are used to regulate the access of each storage space to the task Functional requirements of the management system.
  • the execution layer executes the task execution shell program, it is also used to:
  • the interface method layer may also be used for:
  • the status of the corresponding task is sent to the human-computer interaction interface for display.
  • the second aspect of the present application provides a task management method, the task management method includes:
  • the third aspect of the present application provides a task management device, the task management device includes:
  • the human-computer interaction module is used to receive task management operations input by the user based on the human-computer interaction interface
  • a task calling module configured to determine a task to be executed based on the task management operation, call the task storage interface to store the task to be executed in the cloud, and call the task calling interface to submit the task to be executed to the cloud computing platform;
  • An analysis module configured to allocate container space for the task to be executed, load a shell program for the task to be executed, analyze the workflow description language of the task to be executed, and obtain task data of the task to be executed;
  • the task execution module is used to execute the task data and upload the execution result to cloud storage.
  • the fourth aspect of the present application provides an electronic device, including:
  • a memory on which executable code is stored, and when the executable code is executed by the processor, causes the processor to execute the task management method as described above.
  • the fifth aspect of the present application provides a computer-readable storage medium, on which executable code is stored, and when the executable code is executed by a processor of an electronic device, the processor is made to execute the above-mentioned task management method .
  • FIG. 1 is a schematic structural diagram of a task management system shown in an embodiment of the present application
  • Fig. 2 is a directed acyclic graph shown in the embodiment of the present application.
  • Fig. 3 is a schematic flowchart of a method for updating task status shown in the embodiment of the present application
  • Fig. 4 is an execution flowchart of a shell program shown in the embodiment of the present application.
  • FIG. 5 is a schematic flowchart of a task status query method shown in an embodiment of the present application.
  • FIG. 6 is a schematic flowchart of a task management method shown in the embodiment of the present application.
  • FIG. 7 is a schematic structural diagram of a task management device shown in an embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of an electronic device shown in an embodiment of the present application.
  • first, second, third and so on may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another.
  • first information may also be called second information, and similarly, second information may also be called first information.
  • second information may also be called first information.
  • a feature defined as “first” and “second” may explicitly or implicitly include one or more of these features.
  • “plurality” means two or more, unless otherwise specifically defined.
  • embodiments of the present application provide a task management system, method, electronic device, and computer-readable storage medium, which can call tasks in a hybrid cloud environment and enhance the versatility and scalability of the task management system.
  • Fig. 1 is a schematic structural diagram of a task management system shown in an embodiment of the present application.
  • the task management system includes:
  • the interface method layer 101 is used to provide an interface connected with the human-computer interaction interface, and based on the interface, receive the task management operation input by the user through the human-computer interaction interface, and determine the task to be executed.
  • a system architecture of a task management system includes an interface method layer, an analysis layer, and an execution layer.
  • the interface connected to the interactive interface, and through this interface, the task management operation input by the user through the human-computer interaction interface is received, wherein the task management operation can be task status query operation, task submission operation, task execution operation, task rerun operation, and task stop operations, etc.
  • the interface form provided can be SDK (Software Development Kit, software development kit) or HTTP (Hyper Text Transfer Protocol, hypertext transfer protocol), and the number of interfaces can be one or more, for different Tasks, interfaces can be different.
  • a task to be executed refers to a task to be executed determined by the user based on the human-computer interaction interface, and the operations to be executed of the task may be operations such as task submission, status query, rerun, and stop.
  • the parsing layer 102 is configured to determine the workflow description language of the task to be executed, analyze the workflow description language to obtain the execution sequence of the task to be executed, and provide a task scheduling interface for scheduling the task to be executed Tasks, providing a task storage interface for storing the tasks to be executed.
  • the parsing layer is an intermediate layer between the interface method layer and the execution layer, and is mainly used to analyze the tasks to be executed, determine the execution order, dependencies, parameter serialization methods and parameter deserialization methods of the tasks to be executed etc., and provide corresponding interfaces for the scheduling and storage of tasks to be executed.
  • the workflow description language of the task to be executed is analyzed, the directed acyclic graph of the task to be executed is determined, and the task to be executed is determined based on the directed acyclic graph. The execution sequence of each step in the task and the dependencies between each step, and determine the parameter serialization method and parameter deserialization method in the algorithm.
  • the parsing layer also provides a task scheduling interface for scheduling the tasks to be executed, and provides a task storage interface for storing the tasks to be executed, wherein the task scheduling interface schedules the tasks to be executed, and stores the tasks to be executed Upload to the cloud container to facilitate the execution of tasks to be executed in the container; the task storage interface is used to upload tasks to the cloud storage space and store the tasks to be executed.
  • the execution layer 103 is configured to provide computing resources for executing the tasks to be executed and storage resources for storing the tasks to be executed, provide a task execution shell program to download and execute the data of the tasks to be executed, and Upload the execution result after the execution is completed.
  • the execution layer is responsible for providing computing resources for executing the tasks to be executed and storage resources for storing the tasks to be executed, wherein the computing resources and storage resources may be provided by cloud providers Computing services and storage services, such as Amazon Cloud's object storage service, Amazon Cloud's batch computing service, etc.
  • the execution layer is also used for the task execution shell program to download and execute the data of the task to be executed, and upload the execution result after the execution is completed.
  • the task execution shell program is wrapped around the algorithm script or computing software submitted by the user.
  • the workflow of the task can be analyzed, and the The task scheduling interface provided by the analysis layer schedules the tasks to be executed, uploads the tasks to be executed to the container in the cloud computing center, executes the tasks in the container, and downloads the executed task data from the cloud after the execution is completed, and uses The task storage interface provided by the parsing layer stores the completed task data to the cloud storage space.
  • the task includes a first-level task, a second-level task, and a third-level task, wherein each of the first-level tasks includes one or more of the second-level tasks, Each of the two-level tasks includes one or more of the three-level tasks, and the interface method layer includes:
  • the first-level task interface module is used to provide the management query service of the first-level task
  • the secondary task interface module is used to provide the management query service of the secondary task
  • the third-level task interface module is used to provide management query services for three-level tasks, wherein the management query services include task submission services, task status query services, task rerun services, and task stop services.
  • tasks can be divided into three levels based on the concept of tasks, including first-level tasks, second-level tasks, and third-level tasks, wherein each of the first-level tasks includes one or more of the second-level tasks , each of the secondary tasks includes one or more of the tertiary tasks, and different task interfaces are provided for tasks of different levels.
  • the classification of tasks is not limited to three levels, but tasks can be divided into one, two, three or more levels according to actual conditions.
  • the first-level task is named task
  • the second-level task is named step
  • the third-level task is named job
  • task is the last task of the task
  • the upper-level concept a task can contain one or more steps
  • step is the middle-level concept of the task
  • a step can contain one or more jobs
  • the job is the lowest-level concept of the task.
  • the task interface is the upper-level control of the entire workflow task, and as the top-level granular unit, it provides submission, monitoring, and management capabilities for the entire workflow task.
  • the step interface controls a step node in a workflow task, and provides submission, monitoring, and management capabilities for specific step nodes as a finer-grained unit.
  • the job interface controls the specific subtasks in the steps of the workflow task, and provides submission, monitoring, and management capabilities for the specific subtasks in the specific job nodes as the most granular unit.
  • different task interfaces are provided for different levels of tasks, which facilitates supervision operations such as submission, status query, rerun, and stop of different tasks.
  • the task management system has a clearer division of labor and is more convenient for users to operate.
  • the parsing layer determines the workflow description language of the task to be executed, and parses the workflow description language to obtain the When executing a sequence, it can be used to:
  • For a to-be-executed first-level task analyze the workflow description language of the to-be-executed first-level task, determine the execution sequence of each to-be-executed second-level task in the to-be-executed first-level task, and each to-be-executed second-level task dependencies; and/or
  • the execution order of the tertiary tasks to be executed in the secondary tasks to be executed and the dependencies among the tertiary tasks to be executed are determined.
  • the execution order of tasks in the tasks to be executed when determining the execution order of tasks in the tasks to be executed, it can be determined according to the level of the tasks to be executed, for example, when the task to be executed is a first-level task, the workflow description of the task to be executed Language analysis, determine the secondary tasks in the task to be executed, and determine the execution sequence between the secondary tasks and the dependencies between the secondary tasks, for each secondary task, determine the secondary tasks in the secondary task.
  • the three-level tasks and determine the execution order of the three-level tasks and the dependencies between the three-level tasks.
  • the execution order and dependencies between the above tasks can be determined in the form of a directed acyclic graph .
  • the workflow description language of the second-level task is analyzed to determine the third-level task in the second-level task, and Determine the execution sequence and dependencies among the three-level tasks.
  • the determined task to be executed is a first-level task task1
  • the task is obtained task1 includes three secondary tasks, namely step1, step2 and step3, and determine the execution order of the three secondary tasks as step1-step2-step3, analyze the three secondary tasks to determine that step1 includes Two three-level tasks, namely job1 and job2, step2 includes a third-level task job3, and step3 includes three third-level tasks job4, job5, and job6.
  • job3 needs to be executed after step2 and step3 are completed.
  • FIG. 2 the directed acyclic graph of this embodiment is shown.
  • the number of tasks of each level in the tasks to be executed and the execution sequence and dependencies among the tasks are determined to ensure the normal execution of the tasks.
  • the parsing layer 102 includes:
  • the workflow parser 121 is configured to parse the workflow description language of the tasks to be executed, and determine the order of the tasks to be executed, the dependencies among the tasks, and the parameter serialization method and parameter deserialization method.
  • the parsing layer 102 includes a workflow parser 121, wherein the workflow parser 121 is used to parse the standardized workflow description language in the task to be executed, and perform description conversion for scientific computing software and algorithms , to obtain the execution order of the tasks to be executed, the dependencies among the tasks, the parameter serialization method and the parameter deserialization method, etc.
  • Task scheduling interface 122 the task scheduling interface encapsulates the minimum general requirements for task scheduling execution, and is used to schedule the tasks to be executed, and the minimum general requirements for task scheduling execution are used to regulate each computing platform to be included in the Functional requirements of the task management system.
  • the parsing layer 102 includes a task scheduling interface 122, wherein the task scheduling interface 122 encapsulates the minimum general requirements of the entire task management system for task scheduling execution, which can be used to standardize different cloud computing platforms, super The functional requirements for accessing computers or local computer rooms declares a set of minimum common methods. After any computing platform realizes this interface by writing simple codes, the task management system provided by the embodiment of this application can use the computing platform. computing resources.
  • Task storage interface 123 the task storage interface encapsulates the minimum general requirements for task storage and execution, and is used to store the tasks to be executed, and the minimum general requirements for task storage and execution are used to regulate the access of each storage space to the Functional requirements of the task management system.
  • the parsing layer 102 includes a task storage interface 123, wherein the task storage interface 123 encapsulates the minimum general requirements of the entire task management system for data processing during task execution, and can be used to standardize different cloud storage , NFS (Network File System, network file system), or the functional requirements of local hard disk access, a set of minimum general methods are declared.
  • NFS Network File System, network file system
  • the provided task management system can use the storage resources of the storage platform.
  • the parsing layer includes a workflow parser, a task scheduling interface and a task storage interface
  • the workflow parser is used to parse a standardized workflow description language to describe scientific computing software or algorithms Conversion to obtain the specific execution sequence, dependencies, parameter serialization method and parameter deserialization method of the task, etc.
  • the task scheduling interface and task storage interface can ensure the normal call of cloud computing resources and cloud storage resources.
  • the execution layer executes the task execution shell program, it is also used to:
  • Step S301 updating the task status of the task to be executed based on the execution result of the task to be executed.
  • the execution layer provides a task execution shell program except for the calculation code during task execution, wherein the task execution shell program is wrapped in the algorithm script or calculation software submitted by the user before and after execution, and the shell program is executed During execution, it can also be used to update the task status of the task to be executed according to the execution result of the task to be executed.
  • the tasks to be executed include job1 and job2, after the task data of job1 is calculated in the container, the task execution shell program is executed , according to the task status of job1 is execution complete.
  • Step S302 detecting whether the execution of the task to be executed is completed, and when the execution of the task to be executed is not completed, continue to execute the task to be executed; when the execution of the task to be executed is completed, detecting whether the task to be executed is There is a follow-up task, if there is a follow-up task, the follow-up task is called, and the follow-up task is continued to be executed.
  • the shell program when it is detected that the execution of the task to be executed is completed, the shell program can also be used to call the subsequent task and continue to execute the subsequent task.
  • the shell program when it is detected that the execution of job1 is completed, it is detected that job2 has not yet been executed, and the shell program will schedule job2 to be executed in the container.
  • Figure 4 it shows a flow chart of a task shell program provided by the embodiment of the present application. After entering the shell program, it starts to analyze the workflow language of the task to be executed, and determines the task to be executed.
  • the number of tasks, the execution order of tasks, and the dependencies between tasks, etc. download task data, deserialize task data into memory objects, call user programs, upload task data to containers for calculation, and obtain output data. Serialize the output data into a file, and upload the file as the task result (that is, the execution result), and then judge whether all the current nodes are completed correctly. When it is judged that all nodes are not completed correctly, exit the shell program and prompt the program to make an error. When all are completed correctly , to determine whether the current workflow has a follow-up task, if there is a follow-up task, submit the follow-up task, wait for execution, otherwise exit the shell program.
  • the above is a possible implementation manner provided by the embodiment of the present application, and the specific functions of the shell program can be set according to user requirements.
  • the task execution shell program can raise the granularity of task management to the container level, and all the preparation and finishing work that are different from the cloud environment are handled by the shell program, which shields the services of the same cloud vendor in the hybrid cloud environment.
  • the difference in tools is to adapt the containerization technology on the outer layer of the computing program.
  • the interface method layer can also be used for:
  • Step S501 receiving a task status inquiry request input by a user based on a human-computer interaction interface.
  • the task management system is connected to the human-computer interaction interface through an interface, and the user can manage various tasks in the task management system based on the human-computer interaction interface, including task execution, task status query, and task rerun. , task stop, etc., as a possible implementation of this application, the user can query the task status through the human-computer interaction interface, such as clicking a certain task interface through the human-computer interaction interface, and submit the task status query to the task management system Request, the task status query request must include the unique identifier or universal identifier of the task to be queried.
  • the unique identifier refers to an identifier that can specifically correspond to a unique specific task; the universal identifier refers to an identifier shared by a type of specific task, such as the execution time of a task executed within a certain period of time, which can be used to query the execution time of this period of time The status of multiple tasks; the general identifier can be used for fuzzy query and precise query.
  • Step S502 based on the task status query request, the status of the corresponding task is sent to the human-computer interaction interface for display.
  • the task management system receives the task status inquiry request input by the user based on the human-computer interaction interface through the interface, based on the unique identifier or universal identifier of the task in the request, the status of the corresponding task is sent to the human-computer interaction interface.
  • the interface is displayed to facilitate users to understand the progress of task execution.
  • the user can check the status of each task in the task management system in real time through the human-computer interaction interface, so that the user can know the progress of the task in a timely manner.
  • a task management method is provided, and the task management method includes:
  • Step S601 receiving a task management operation input by a user based on a human-computer interaction interface.
  • the task management method provided is applied to the task management system provided in the foregoing embodiments.
  • the task management system is connected to the human-computer interaction interface through an interface.
  • Each task is managed, including task execution, task status query, task rerun, task stop, etc.
  • the user can input task management operations through the man-machine interface, such as through man-machine Click on a certain task interface in the interactive interface to submit a task status query request to the task management system.
  • the task status query request includes the unique identifier or general identifier of the task to be queried.
  • the unique identifier refers to an identifier that can specifically correspond to a unique specific task; the universal identifier refers to an identifier shared by a type of specific task, such as the execution time of a task executed within a certain period of time, which can be used to query the execution time of this period of time The status of multiple tasks; the general identifier can be used for fuzzy query and precise query.
  • Step S602 Determine the task to be executed based on the task management operation, call the task storage interface to store the to-be-executed task in the cloud, and call the task calling interface to submit the to-be-executed task to the cloud computing platform.
  • the task scheduling interface encapsulates the minimum general requirements for task scheduling and execution, and is used to schedule the tasks to be executed.
  • the minimum general requirements for task scheduling and execution are used to regulate each computing platform to include Functional requirements of the task management system.
  • the task storage interface encapsulates the minimum general requirements for task storage and execution, and is used to store the tasks to be executed.
  • the minimum general requirements for task storage and execution are used to standardize the functional requirements for each storage space to access the task management system.
  • Step S603 allocate container space for the task to be executed, load the task shell program to be executed, analyze the workflow description language of the task to be executed, and obtain task data of the task to be executed.
  • Step S604 execute the task data, and upload the execution result to cloud storage.
  • the task data is executed in an independent container space, and the execution result is uploaded to cloud storage to realize the execution of the task.
  • the embodiment of the present application schedules and stores tasks through an independent interface, and can arbitrarily connect with various cloud computing platforms and cloud storage platforms, realizes the invocation and execution of the same task in a mixed environment, and enhances the versatility and performance of the task management system.
  • Scalability the design of the shell program raises the granularity of task management to the container level. All the preparation/finishing work that is different from the cloud environment is handled by the shell program, which shields the differences in the service tools of different cloud vendors in the hybrid cloud environment.
  • the outer layer of the computing program is adapted to containerization technology, which can effectively utilize cloud computing resources, reduce computing costs, and reduce the amount of computing.
  • the task management method after uploading the execution result to the cloud storage, the task management method further includes:
  • the provided shell program is wrapped and executed before and after the algorithm script or computing software submitted by the user.
  • the shell program can also be used to update the task to be performed according to the execution result of the task to be performed. task status.
  • the tasks to be executed include job1 and job2, after the task data of job1 is calculated in the container, the task execution shell program is executed , according to the task status of job1 is execution complete.
  • the task execution shell program can raise the granularity of task management to the container level, and all the preparation and finishing work that are different from the cloud environment are handled by the shell program, which shields the services of the same cloud vendor in the hybrid cloud environment.
  • the difference in tools is to adapt the containerization technology on the outer layer of the computing program.
  • tasks include primary tasks, secondary tasks, and tertiary tasks, wherein each of the primary tasks includes one or more Each of the two-level tasks, each of the two-level tasks includes one or more of the three-level tasks, and the interface method layer includes:
  • the first-level task interface module is used to provide the management query service of the first-level task
  • the secondary task interface module is used to provide the management query service of the secondary task
  • the third-level task interface module is used to provide management query services for three-level tasks, wherein the management query services include task submission services, task status query services, task rerun services, and task stop services.
  • the interface method layer when the task is managed through the interface method layer, the interface method layer provides a three-level task interface module, including a first-level task interface module, which is used to provide a first-level task interface module.
  • Management query service the secondary task interface module is used to provide the management query service of the secondary task; the tertiary task interface module is used to provide the management query service of the tertiary task, wherein the management query service includes task submission service, task Status query service, task rerun service, task stop service.
  • the task management method determines the workflow description language of the task to be executed, parses the workflow description language, and obtains the execution order of the tasks to be executed, it includes:
  • For a to-be-executed first-level task analyze the workflow description language of the to-be-executed first-level task, determine the execution sequence of each to-be-executed second-level task in the to-be-executed first-level task, and each to-be-executed second-level task dependencies; and/or
  • the execution order of the tertiary tasks to be executed in the secondary tasks to be executed and the dependencies among the tertiary tasks to be executed are determined.
  • the analysis layer includes a workflow parser, which is used to analyze the workflow description language of the task to be executed , determine the order of the tasks to be executed, the dependencies between the tasks, and the parameter serialization method and parameter deserialization method;
  • the task scheduling interface the task scheduling interface encapsulates the minimum general requirements for task scheduling execution, and is used for Scheduling the tasks to be executed, the minimum general requirements for task scheduling and execution are used to standardize the functional requirements of each computing platform included in the task management system;
  • task storage interface the task storage interface encapsulates the task storage and execution The minimum general requirements are used to store the tasks to be executed, and the minimum general requirements for task storage and execution are used to standardize the functional requirements for each storage space to access the task management system.
  • the task management method provided includes when executing the task execution shell program:
  • the task management method provided can also be used to receive a task status query request input by the user based on the human-computer interaction interface; based on the task status query request, the status of the corresponding task is sent to the human-computer interaction interface to show.
  • the implementation principle of the task management method provided in the embodiment of the present application is the same as the implementation principle of the task management system provided in the foregoing embodiments, and the present application will not repeat them here.
  • the present application also provides a task management device, electronic equipment, and corresponding embodiments.
  • Fig. 7 is a schematic structural diagram of a task management device shown in an embodiment of the present application.
  • the task management device 70 includes a human-computer interaction module 710, a task calling module 720, an analysis module 730, and a task execution module 740, wherein:
  • the human-computer interaction module 710 is configured to receive task management operations input by the user based on the human-computer interaction interface
  • the task calling module 720 is configured to determine the task to be executed based on the task management operation, call the task storage interface to store the task to be executed in the cloud, and call the task calling interface to submit the task to be executed to the cloud computing platform;
  • the parsing module 730 is configured to allocate a container space for the task to be executed, load a shell program of the task to be executed, analyze the workflow description language of the task to be executed, and obtain task data of the task to be executed;
  • the task execution module 740 is configured to execute the task data and upload the execution result to cloud storage.
  • the task execution module 740 can also be used to:
  • FIG. 8 is a schematic structural diagram of an electronic device shown in an embodiment of the present application.
  • the electronic device 1000 includes a memory 1010 and a processor 1020 .
  • the processor 1020 can be a central processing unit (Central Processing Unit, CPU), and can also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), on-site Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • the memory 1010 may include various types of storage units such as system memory, read only memory (ROM), and persistent storage.
  • the ROM may store static data or instructions required by the processor 1020 or other modules of the computer.
  • the persistent storage device may be a readable and writable storage device.
  • Persistent storage may be a non-volatile storage device that does not lose stored instructions and data even if the computer is powered off.
  • the permanent storage device adopts a mass storage device (such as a magnetic or optical disk, flash memory) as the permanent storage device.
  • the permanent storage device may be a removable storage device (such as a floppy disk, an optical drive).
  • the system memory can be a readable and writable storage device or a volatile readable and writable storage device, such as dynamic random access memory.
  • System memory can store some or all of the instructions and data that the processor needs at runtime.
  • the memory 1010 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and magnetic disks and/or optical disks may also be used.
  • memory 1010 may include a readable and/or writable removable storage device, such as a compact disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, dual-layer DVD-ROM), Read-only Blu-ray Disc, Super Density Disc, Flash memory card (such as SD card, min SD card, Micro-SD card, etc.), magnetic floppy disk, etc.
  • a readable and/or writable removable storage device such as a compact disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, dual-layer DVD-ROM), Read-only Blu-ray Disc, Super Density Disc, Flash memory card (such as SD card, min SD card, Micro-SD card, etc.), magnetic floppy disk, etc.
  • Computer-readable storage media do not contain carrier waves and transient electronic signals transmitted by wireless or wire.
  • Executable codes are stored in the memory 1010, and when the executable codes are processed by the processor 1020, the processor 1020 can be made to execute part or all of the methods mentioned above.
  • the method according to the present application can also be implemented as a computer program or computer program product, the computer program or computer program product including computer program code instructions for executing some or all of the steps in the above method of the present application.
  • the present application may also be implemented as a computer-readable storage medium (or a non-transitory machine-readable storage medium or a machine-readable storage medium), on which executable code (or computer program or computer instruction code) is stored,
  • executable code or computer program or computer instruction code
  • the processor of the electronic device or server, etc.
  • the processor is made to perform part or all of the steps of the above-mentioned method according to the present application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

本申请涉及一种任务管理系统、方法、装置、设备及存储介质。该系统包括：接口方法层，用于提供与人机交互界面连接的接口；解析层，用于提供工作流描述语言解析器，并提供任务调度接口和任务存储接口；执行层，用于计算资源和存储资源，提供任务执行壳程序对待执行任务的数据进行下载和执行，并在执行结束后将执行结果上传。本申请实施例通过独立的接口对任务进行调度和存储，可以任意对接多种云计算平台和云存储平台，实现了相同任务在混合环境中的调用和执行，壳程序的设计将任务管理的粒度上升到了容器层级，屏蔽了混合云环境中不同云厂商服务工具的差异，在计算程序外层适配容器化技术，能够有效利用云计算资源，降低计算成本，减少计算量。

Description

任务管理系统、方法、装置、设备及存储介质 技术领域
本申请涉及计算机系统技术领域，尤其涉及一种任务管理系统、方法、装置、设备及存储介质。
背景技术
随着计算机实用的普及以及算力的不断增长，所需要的计算资源也在不停的飞涨，对于计算管理系统的解决方案层出不穷，由于海量计算资源的需求，计算上云乃至于上混合云成为了主要趋势。
在相关技术方案中，一般都采用极简Linux资源管理工具或者是Apache基金会的Airflow Platform，但是这些通用的工作流管理系统的计算任务需要绑定执行环境，执行任务需要使用本地资源，且存储调用都需要用户根据资源类型编写执行操作，难以在混合云环境中使用，或者想要将其适配到混合云环境中需要大量的二次开发工作，导致工作量激增，需要解决。
技术问题
现有的系统在管理任务时，需要绑定执行环境，执行任务时需要使用本地资源，且存储调用都需要用户根据资源类型编写执行操作，难以在混合云环境中使用，或者想要将其适配到混合云环境中需要大量的二次开发工作，导致工作量激增。
技术解决方案
本申请第一方面提供一种任务管理系统，该任务管理系统包括：
接口方法层，用于提供与人机交互界面连接的接口，并基于所述接口接收用户通过所述人机交互界面输入的任务管理操作，确定待执行任务；
解析层，用于确定所述待执行任务的工作流描述语言，对所述工作流描述语言进行解析，得到所述待执行任务的执行顺序，并提供任务调度接口用于调度所述待执行任务，提供任务存储接口用于存储所述待执行任务；
执行层，用于提供用于执行所述待执行任务的计算资源和用于存储所述待执行任务的存储资源，提供任务执行壳程序对所述待执行任务的数据进行下载和执行，并在执行结束后将执行结果上传。
作为本申请一种可能的实施方式，在该实施方式中，任务包括一级任务、二级任务以及三级任务，其中，每个所述一级任务包括一个或多个所述二级任务，每个所述二级任务包括一个或多个所述三级任务，所述接口方法层包括：
一级任务接口模块，用于提供一级任务的管理查询服务；
二级任务接口模块，用于提供二级任务的管理查询服务；
三级任务接口模块，用于提供三级任务的管理查询服务，其中所述管理查询服务包括任务提交服务、任务状态查询服务、任务重跑服务、任务停止服务。
作为本申请一种可能的实施方式，在该实施方式中，所述解析层在确定所述待执行任务的工作流描述语言，对所述工作流描述语言进行解析，得到所述待执行任务的执行顺序时，可以用于：
对于一个待执行一级任务，解析所述待执行一级任务的工作流描述语言，确定所述待执行一级任务中各待执行二级任务的执行顺序、各所述待执行二级任务的依赖关系；和/或
对于一个所述待执行二级任务，确定所述待执行二级任务中各待执行三级任务执行顺序以及各所述待执行三级任务之间的依赖关系。
作为本申请一种可能的实施方式，在该实施方式中，所述解析层包括：
工作流解析器，用于对所述待执行任务的工作流描述语言进行解析，确定所述待执行任务的顺序、各任务之间的依赖关系以及参数序列化方法和参数反序列化方法；
任务调度接口，所述任务调度接口封装有任务调度执行的最小通用需求，用于对所述待执行任务进行调度，所述任务调度执行的最小通用需求用于规范各计算平台计入所述任务管理系统的功能要求；
任务存储接口，所述任务存储接口封装有任务存储执行的最小通用需求，用于对所述待执行任务进行存储，所述任务存储执行的最小通用需求用于规范各存储空间接入所述任务管理系统的功能要求。
作为本申请一种可能的实施方式，在该实施方式中，所述执行层在执行所述任务执行壳程序时，还用于：
基于所述待执行任务的执行结果，更新所述待执行任务的任务状态；
检测所述待执行任务是否执行完成，当所述待执行任务未执行完成时，继续执行所述待执行任务；当所述待执行任务执行完成后，检测所述待执行任务之后是否有后续任务，若存在后续任务，则调用后续任务，继续执行所述后续任务。
作为本申请一种可能的实施方式，在该实施方式中，所述接口方法层还可以用于：
接收用户基于人机交互界面输入的任务状态查询请求；
基于所述任务状态查询请求将对应任务的状态发送至所述人机交互界面进行展示。
本申请第二方面提供一种任务管理方法，该任务管理方法包括：
接收用户基于人机交互界面输入的任务管理操作；
基于所述任务管理操作确定待执行任务，调用任务存储接口将所述待执行任务存储至云端，调用任务调用接口将所述待执行任务提交至云计算平台；
为所述待执行任务分配容器空间，加载待执行任务壳程序，对所述待执行任务的工作流描述语言进行解析，获得所述待执行任务的任务数据；
执行所述任务数据，并将执行结果上传至云端存储。
作为本申请一种可能的实施方式，在该实施方式中，所述将执行结果上传至云端存储之后，还包括：
检测所述待执行任务是否执行完成，当所述待执行任务未执行完成时，继续执行所述待执行任务；当所述待执行任务执行完成后，检测所述待执行任务之后是否有后续任务，若存在后续任务，则调用后续任务，继续执行后续任务。
本申请第三方面提供了一种任务管理装置，该任务管理装置包括：
人机交互模块，用于接收用户基于人机交互界面输入的任务管理操作；
任务调用模块，用于基于所述任务管理操作确定待执行任务，调用任务存储接口将所述待执行任务存储至云端，调用任务调用接口将所述待执 行任务提交至云计算平台；
解析模块，用于为所述待执行任务分配容器空间，加载待执行任务壳程序，对所述待执行任务的工作流描述语言进行解析，获得所述待执行任务的任务数据；
任务执行模块，用于执行所述任务数据，并将执行结果上传至云端存储。
本申请第四方面提供一种电子设备，包括：
处理器；以及
存储器，其上存储有可执行代码，当所述可执行代码被所述处理器执行时，使所述处理器执行如上所述的任务管理方法。
本申请第五方面提供一种计算机可读存储介质，其上存储有可执行代码，当所述可执行代码被电子设备的处理器执行时，使所述处理器执行如上所述的任务管理方法。
有益效果
本申请实施例通过将任务的调度和存储进行解耦，通过独立的接口对任务进行调度和存储，可以任意对接多种云计算平台和云存储平台，实现了相同任务在混合环境中的调用和执行，增强了任务管理系统的通用性和扩展性，壳程序的设计将任务管理的粒度上升到了容器层级，所有与云环境有差异性的准备/收尾工作都由壳程序处理，屏蔽了混合云环境中不同云厂商服务工具的差异，在计算程序外层适配容器化技术，能够有效利用云计算资源，降低计算成本，减少计算量。
应当理解的是，以上的一般描述和后文的细节描述仅是示例性和解释性的，并不能限制本申请。
附图说明
通过结合附图对本申请示例性实施方式进行更详细的描述，本申请的上述以及其它目的、特征和优势将变得更加明显，其中，在本申请示例性实施方式中，相同的参考标号通常代表相同部件。
图1是本申请实施例示出的一种任务管理系统的架构示意图；
图2是本申请实施例示出的一种有向无环图；
图3是本申请实施例示出的一种任务状态更新方法的流程示意图；
图4是本申请实施例示出的一种壳程序的执行流程图；
图5是本申请实施例示出的一种任务状态查询方法的流程示意图；
图6是本申请实施例示出的一种任务管理方法的流程示意图；
图7是本申请实施例示出的一种任务管理装置的结构示意图；
图8是本申请实施例示出的电子设备的结构示意图。
本发明的实施方式
下面将参照附图更详细地描述本申请的实施方式。虽然附图中显示了本申请的实施方式，然而应该理解，可以以各种形式实现本申请而不应被这里阐述的实施方式所限制。相反，提供这些实施方式是为了使本申请更加透彻和完整，并且能够将本申请的范围完整地传达给本领域的技术人员。
在本申请使用的术语是仅仅出于描述特定实施例的目的，而非旨在限制本申请。在本申请和所附权利要求书中所使用的单数形式的“一种”、“所述”和“该”也旨在包括多数形式，除非上下文清楚地表示其他含义。还应当理解，本文中使用的术语“和/或”是指并包含一个或多个相关联的列出项目的任何或所有可能组合。
应当理解，尽管在本申请可能采用术语“第一”、“第二”、“第三”等来描述各种信息，但这些信息不应限于这些术语。这些术语仅用来将同一类型的信息彼此区分开。例如，在不脱离本申请范围的情况下，第一信息也可以被称为第二信息，类似地，第二信息也可以被称为第一信息。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征。在本申请的描述中，“多个”的含义是两个或两个以上，除非另有明确具体的限定。
随着计算机实用的普及以及算力的不断增长，所需要的计算资源也在不停的飞涨，对于计算管理系统的解决方案层出不穷，由于海量计算资源的需求，计算上云乃至于上混合云成为了主要趋势。在相关技术方案中，一般都采用极简Linux资源管理工具或者是Apache基金会的Airflow Platform，但是这些通用的工作流管理系统的计算任务需要绑定执行环境， 执行任务需要使用本地资源，且存储调用都需要用户根据资源类型编写执行操作，难以在混合云环境中使用，或者想要将其适配到混合云环境中需要大量的二次开发工作，导致工作量激增，需要解决
针对上述问题，本申请实施例提供一种任务管理系统、方法、电子设备及计算机可读存储介质，能够将任务在混合云环境中调用，增强任务管理系统的通用性与扩展性。
以下结合附图详细描述本申请实施例的技术方案。
图1是本申请实施例示出的任务管理系统的结构示意图。
参见图1，在本申请实施例中，提供的任务管理系统包括：
接口方法层101，用于提供与人机交互界面连接的接口，并基于所述接口接收用户通过所述人机交互界面输入的任务管理操作，确定待执行任务。
在本申请实施例中，提供了一种任务管理系统的系统架构，在该任务管理系统中，包括接口方法层、解析层和执行层，其中，接口方法层用于提供与用户所在的人机交互界面连接的接口，并通过该接口接收用户通过该人机交互界面输入的任务管理操作，其中，该任务管理操作可以是任务状态查询操作、任务提交操作、任务执行操作、任务重跑操作、以及任务停止操作等。在本申请实施例中，提供的接口形式可以是SDK(Software Development Kit，软件开发工具包)或者HTTP(Hyper Text Transfer Protocol，超文本传输协议)，接口数量可以是一个或者多个，针对不同的任务，接口可以不同。在本申请实施例中，待执行任务是指用户基于人机交互界面确定的需要执行的任务，该任务的待执行操作可以是任务提交、状态查询、重跑、停止等操作。
解析层102，用于确定所述待执行任务的工作流描述语言，对所述工作流描述语言进行解析，得到所述待执行任务的执行顺序，并提供任务调度接口用于调度所述待执行任务，提供任务存储接口用于存储所述待执行任务。
在本申请实施例中，解析层是接口方法层和执行层的中间层，主要用于对待执行任务进行解析，确定待执行任务的执行顺序、依赖关系、参数序列化方法和参数反序列化方法等，以及用于为待执行任务的调度和存储 提供对应的接口。在本申请实施例中，在确定待执行任务之后，对该待执行任务的工作流描述语言进行解析，确定该待执行任务的有向无环图，基于该有向无环图确定该待执行任务中各步骤的执行顺序以及各步骤之间的依赖关系，并确定算法中的参数序列化方法和参数反序列化方法。在本申请实施例中，解析层还提供任务调度接口用于调度所述待执行任务，提供任务存储接口用于存储所述待执行任务，其中，任务调度接口调度待执行任务，将待执行任务上传至云端容器，便于待执行任务在容器中执行；任务存储接口用于将任务上传至云端存储空间中，对待执行任务进行存储。
执行层103，用于提供用于执行所述待执行任务的计算资源和用于存储所述待执行任务的存储资源，提供任务执行壳程序对所述待执行任务的数据进行下载和执行，并在执行结束后将执行结果上传。
在本申请实施例中，执行层负责提供用于执行所述待执行任务的计算资源和用于存储所述待执行任务的存储资源，其中，该计算资源和存储资源可以是采用云厂商提供的计算服务和存储服务，例如亚马逊云的对象存储服务，亚马逊云的批量计算服务等。在本申请实施例中，执行层还用于任务执行壳程序对待执行任务的数据进行下载和执行，并在执行结束后将执行结果上传。作为本申请一种可能的实施方式，在该实施方式中，任务执行壳程序包裹在用户提交的算法脚本或计算软件前后，当任务执行壳程序执行时，可以对任务的工作流进行解析，利用解析层提供的任务调度接口对待执行任务进行调度，将待执行任务上传到云端计算中心的容器中，在容器中对任务进行执行，在执行完成后，从云端下载执行完成的任务数据，并利用解析层提供的任务存储接口，将执行完成的任务数据存储至云端存储空间。
本申请实施例通过将任务的调度和存储进行解耦，通过独立的接口对任务进行调度和存储，可以任意对接多种云计算平台和云存储平台，实现了相同任务在混合环境中的调用和执行，增强了任务管理系统的通用性和扩展性，壳程序的设计将任务管理的粒度上升到了容器层级，所有与云环境有差异性的准备/收尾工作都由壳程序处理，屏蔽了混合云环境中不同云厂商服务工具的差异，在计算程序外层适配容器化技术，能够有效利用云计算资源，降低计算成本，减少计算量。
作为本申请一种可能的实施方式，在该实施方式中，任务包括一级任务、二级任务以及三级任务，其中，每个所述一级任务包括一个或多个所述二级任务，每个所述二级任务包括一个或多个所述三级任务，所述接口方法层包括：
一级任务接口模块，用于提供一级任务的管理查询服务；
二级任务接口模块，用于提供二级任务的管理查询服务；
三级任务接口模块，用于提供三级任务的管理查询服务，其中所述管理查询服务包括任务提交服务、任务状态查询服务、任务重跑服务、任务停止服务。
在本申请实施例中，基于任务概念可以将任务分为三级，包括一级任务、二级任务以及三级任务，其中，每个所述一级任务包括一个或多个所述二级任务，每个所述二级任务包括一个或多个所述三级任务，针对不同级别的任务，提供不同的任务接口。在本申请实施例中，对任务的分级不仅限制于将任务分为三级，根据实际情况，可以将任务分为一级、两级、三级或者更多的级别。作为本申请一种可能的实施方式，为方便说明，以一个具体实施例为例，一级任务命名为task，二级任务命名为step，三级任务命名为job，其中，task为任务的最上层概念，一个task中可以包含一个或多个step，step为任务的中层概念，一个step可以包含一个或多个job，job为任务的最底层概念，针对不同级别的任务，设置有不同的接口，其中，task接口为整个工作流任务的上层控制，作为最上层粒度单元对于整个工作流task提供提交、监控、管理能力。step接口为工作流任务中的一个步骤节点控制，作为较细粒度单元对于具体步骤节点提供提交、监控、管理能力。job接口为工作流任务的步骤中具体子任务控制，作为最细粒度单元对于具体job节点中的具体子任务提供提交、监控、管理能力。
在本申请实施例中，针对不同等级的任务，提供不同的任务接口，方便对不同任务进行提交、状态查询、重跑、停止等监管操作，任务管理系统分工更加明确，方便用户操作。
作为本申请一种可能的实施方式，在该实施方式中，所述解析层在确定所述待执行任务的工作流描述语言，对所述工作流描述语言进行解析，得到所述待执行任务的执行顺序时，可以用于：
对于一个待执行一级任务，解析所述待执行一级任务的工作流描述语言，确定所述待执行一级任务中各待执行二级任务的执行顺序、各所述待执行二级任务的依赖关系；和/或
对于一个所述待执行二级任务，确定所述待执行二级任务中各待执行三级任务执行顺序以及各所述待执行三级任务之间的依赖关系。
在本申请实施例中，在确定待执行任务中各任务的执行顺序时，可以根据待执行任务的等级确定，例如，当待执行任务是一级任务时，对该待执行任务的工作流描述语言进行解析，确定该待执行任务中的二级任务，并确定各二级任务之间的执行顺序和各二级任务之间的依赖关系，针对每一个二级任务，确定该二级任务中的三级任务，并确定各三级任务的执行顺序和各三级任务之间的依赖关系，可选的，可以通过有向无环图的形式确定上述各任务之间的执行顺序和依赖关系。作为本申请一种可能的实施方式，在该实施方式中，待执行任务为二级任务时，对该二级任务的工作流描述语言进行解析，确定该二级任务中的三级任务，并确定各三级任务之间的执行顺序和依赖关系。
作为本申请一种可能的实施方式，为方便说明，以一个具体实施例为例，确定的待执行任务为一级任务task1，在对该任务task1的工作流描述语言进行解析时，得到该任务task1中包括三个二级任务，分别为step1、step2以及step3，并确定该三个二级任务之间的执行顺序为step1-step2-step3，对该三个二级任务进行解析确定step1中包括两个三级任务，分别为job1、job2，step2中包括一个三级任务job3，step3中包括三个三级任务job4、job5以及job6，其中，job3需要在step2和step3都完成之后，才开始执行，为方便理解，如图2所示，示出了该实施例的有向无环图。
本申请实施例通过对工作流描述语言进行解析，确定待执行任务中各等级任务的数量以及各任务之间的执行顺序和依赖关系，保证任务正常执行。
作为本申请一种可能的实施方式，在该实施方式中，解析层102包括：
工作流解析器121，用于对所述待执行任务的工作流描述语言进行解析，确定所述待执行任务的顺序、各任务之间的依赖关系以及参数序列化 方法和参数反序列化方法。
在本申请实施例中，解析层102包括工作流解析器121，其中，该工作流解析器121用于对待执行任务中的标准化工作流描述语言进行解析，对科学计算的软件和算法进行描述转换，得到待执行任务的执行顺序、各任务之间的依赖关系以及参数序列化方法和参数反序列化方法等。
任务调度接口122，所述任务调度接口封装有任务调度执行的最小通用需求，用于对所述待执行任务进行调度，所述任务调度执行的最小通用需求用于规范各计算平台计入所述任务管理系统的功能要求。
在本申请实施例中，解析层102中包括任务调度接口122，其中，该任务调度接口122中封装有整个任务管理系统对于任务调度执行的最小通用需求，可以用于规范不同云计算平台、超级计算机或者本地机房接入时的功能需求，声明了一组最小通用方法，任何的计算平台在通过编写简单的代码实现这一接口后，本申请实施例提供的任务管理系统即可使用该计算平台的计算资源。
任务存储接口123，所述任务存储接口封装有任务存储执行的最小通用需求，用于对所述待执行任务进行存储，所述任务存储执行的最小通用需求用于规范各存储空间接入所述任务管理系统的功能要求。
在本申请实施例中，解析层102中包括任务存储接口123，其中，该任务存储接口123中封装有整个任务管理系统对于任务执行过程中数据处理的最小通用需求，可以用于规范不同云存储、NFS(Network File System，网络文件系统)、或本地硬盘接入时的功能需求，声明了一组最小通用方法，任何的存储平台在通过编写简单的代码实现这一接口后，本申请实施例提供的任务管理系统即可使用该存储平台的存储资源。
本申请实施例提供的任务管理系统中，解析层中包括工作流解析器、任务调度接口和任务存储接口，工作流解析器用于解析标准化的工作流描述语言，对科学计算的软件或算法进行描述转换，得到任务的具体执行顺序，依赖关系，参数序列化方法和参数反序列化方法等，任务调度接口和任务存储接口能够保证云计算资源和云存储资源的正常调用。
作为本申请一种可能的实施方式，在该实施方式中，如图3所示，所述执行层在执行所述任务执行壳程序时，还用于：
步骤S301，基于所述待执行任务的执行结果，更新所述待执行任务的任务状态。
在本申请实施例中，执行层提供了任务执行时除计算代码外的任务执行壳程序，其中，该任务执行壳程序包裹在用户提交的算法脚本或计算软件的前后执行，在该壳程序被执行时，还可以用于根据待执行任务的执行结果，更新该待执行任务的任务状态。作为本申请一种可能的实施方式，为方便说明，以一个具体实施例为例，待执行任务包括job1和job2，当将job1的任务数据在容器中计算完成后，任务执行壳程序在执行时，根据job1的任务状态为执行完成。
步骤S302，检测所述待执行任务是否执行完成，当所述待执行任务未执行完成时，继续执行所述待执行任务；当所述待执行任务执行完成后，检测所述待执行任务之后是否有后续任务，若存在后续任务，则调用后续任务，继续执行所述后续任务。
在本申请实施例中，在检测到待执行任务执行完成时，壳程序还可以用于调用后续任务，继续执行后续任务。为方便说明，以前述实施例为例，当检测到job1执行结束之后，检测到job2还未执行，壳程序会调度job2到容器中执行。为方便说明，如图4所示，示出了本申请实施例提供的一种任务壳程序流程图，在进入到壳程序中之后，开始解析待执行任务的工作流语言，确定待执行任务中任务的数量、任务的执行顺序以及各任务之间的依赖关系等，并下载任务数据，将任务数据反序列化为内存对象，调用用户程序，将任务数据上传至容器进行计算，得到输出数据，将输出数据序列化成文件，并将文件作为任务结果(即执行结果)上传，然后判断当前节点是否全部正确结束，当判定不是全部正确结束时，退出壳程序，提示程序出错，当全部正确结束时，判断当前工作流是否有后续任务，当存在后续任务时，提交后续任务，等待执行，否则退出壳程序。以上为本申请实施例提供的一种可能的实施方式，具体壳程序的功能，可以根据用户需求自行设定。
在本申请实施例中，任务执行壳程序可以将任务管理的粒度上升到了容器层级，所有与云环境有差异性的准备和收尾工作都由壳程序处理，屏蔽了混合云环境中同云厂商服务工具的差异，在计算程序外层适配容器化 技术。
作为本申请一种可能的实施方式，在该实施方式中，如图5所示，所述接口方法层还可以用于：
步骤S501，接收用户基于人机交互界面输入的任务状态查询请求。
在本申请实施例中，任务管理系统通过接口与人机交互界面连接，用户可以基于该人机交互界面对任务管理系统中的各个任务进行管理，其中包括任务执行、任务状态查询、任务重跑、任务停止等，作为本申请一种可能的实施方式，用户可以通过该人机交互界面对任务状态进行查询，如通过人机交互界面点击某个任务界面，可向任务管理系统提交任务状态查询请求，该任务状态查询请求需包括拟查询任务的唯一标识或通用标识。所述唯一标识指能够特定对应到一个唯一的具体任务的标识；所述通用标识指一类具体任务共有的标识，比如某一时间段内执行的任务的执行时间，可用于查询该时间段执行的多个任务的状态；所述通用标识可用于模糊查询和精确查询。
步骤S502，基于所述任务状态查询请求将对应任务的状态发送至所述人机交互界面进行展示。
在本申请实施例中，任务管理系统通过接口接收到用户基于人机交互界面输入的任务状态查询请求之后，基于该请求中任务的唯一标识或通用标识，将对应任务的状态发送至人机交互界面进行展示，便于用户了解任务执行进度。
本申请实施例中用户可以通过人机交互界面实时查看任务管理系统中的各任务的状态，方便用户及时了解任务进度。
本申请实施例通过将任务的调度和存储进行解耦，通过独立的接口对任务进行调度和存储，可以任意对接多种云计算平台和云存储平台，实现了相同任务在混合环境中的调用和执行，增强了任务管理系统的通用性和扩展性，壳程序的设计将任务管理的粒度上升到了容器层级，所有与云环境有差异性的准备/收尾工作都由壳程序处理，屏蔽了混合云环境中不同云厂商服务工具的差异，在计算程序外层适配容器化技术，能够有效利用云计算资源，降低计算成本，减少计算量。
在本申请实施例中，如图6所示，提供了一种任务管理方法，该任务 管理方法包括：
步骤S601，接收用户基于人机交互界面输入的任务管理操作。
在本申请实施例中，提供的任务管理方法应用于前述实施例中提供的任务管理系统中，任务管理系统通过接口与人机交互界面连接，用户可以基于该人机交互界面对任务管理系统中的各个任务进行管理，其中包括任务执行、任务状态查询、任务重跑、任务停止等，作为本申请一种可能的实施方式，用户可以通过该人机交互界输入任务管理操作，如通过人机交互界面点击某个任务界面，可向任务管理系统提交任务状态查询请求，该任务状态查询请求包括拟查询任务的唯一标识或通用标识。所述唯一标识指能够特定对应到一个唯一的具体任务的标识；所述通用标识指一类具体任务共有的标识，比如某一时间段内执行的任务的执行时间，可用于查询该时间段执行的多个任务的状态；所述通用标识可用于模糊查询和精确查询。
步骤S602，基于所述任务管理操作确定待执行任务，调用任务存储接口将所述待执行任务存储至云端，调用任务调用接口将所述待执行任务提交至云计算平台。
在本申请实施例中，任务调度接口封装有任务调度执行的最小通用需求，用于对所述待执行任务进行调度，所述任务调度执行的最小通用需求用于规范各计算平台计入所述任务管理系统的功能要求。任务存储接口封装有任务存储执行的最小通用需求，用于对所述待执行任务进行存储，所述任务存储执行的最小通用需求用于规范各存储空间接入所述任务管理系统的功能要求。在确定待执行任务之后，基于该待执行任务，调用任务存储接口将任务度存储至云端，调用任务调度接口将待执行任务提交至云计算平台。
步骤S603，为所述待执行任务分配容器空间，加载待执行任务壳程序，对所述待执行任务的工作流描述语言进行解析，获得所述待执行任务的任务数据。
在本申请实施例中，在任务执行时，需要为任务分配独立的容器空间，并加载预先设置的任务壳程序，对待执行任务的工作流描述语言进行解析，获取该任务的任务数据，并确定该任务的执行顺序、依赖关系。
步骤S604，执行所述任务数据，并将执行结果上传至云端存储。
在本申请实施例中，在独立的容器空间中，执行任务数据，并将执行结果上传至云端存储，实现任务的执行。
本申请实施例通过独立的接口对任务进行调度和存储，可以任意对接多种云计算平台和云存储平台，实现了相同任务在混合环境中的调用和执行，增强了任务管理系统的通用性和扩展性，壳程序的设计将任务管理的粒度上升到了容器层级，所有与云环境有差异性的准备/收尾工作都由壳程序处理，屏蔽了混合云环境中不同云厂商服务工具的差异，在计算程序外层适配容器化技术，能够有效利用云计算资源，降低计算成本，减少计算量。
作为本申请一种可能的实施方式，在该实施方式中，该任务管理方法中在将执行结果上传至云端存储之后，还包括：
检测所述待执行任务是否执行完成，当所述待执行任务未执行完成时，继续执行所述待执行任务；当所述待执行任务执行完成后，检测所述待执行任务之后是否有后续任务，若存在后续任务，则调用后续任务，继续执行后续任务。
在本申请实施例中，提供的壳程序包裹在用户提交的算法脚本或计算软件的前后执行，在该壳程序被执行时，还可以用于根据待执行任务的执行结果，更新该待执行任务的任务状态。作为本申请一种可能的实施方式，为方便说明，以一个具体实施例为例，待执行任务包括job1和job2，当将job1的任务数据在容器中计算完成后，任务执行壳程序在执行时，根据job1的任务状态为执行完成。
在本申请实施例中，任务执行壳程序可以将任务管理的粒度上升到了容器层级，所有与云环境有差异性的准备和收尾工作都由壳程序处理，屏蔽了混合云环境中同云厂商服务工具的差异，在计算程序外层适配容器化技术。
作为本申请一种可能的实施方式，在该实施方式中，所述任务管理方法中，任务包括一级任务、二级任务以及三级任务，其中，每个所述一级任务包括一个或多个所述二级任务，每个所述二级任务包括一个或多个所述三级任务，所述接口方法层包括：
一级任务接口模块，用于提供一级任务的管理查询服务；
二级任务接口模块，用于提供二级任务的管理查询服务；
三级任务接口模块，用于提供三级任务的管理查询服务，其中所述管理查询服务包括任务提交服务、任务状态查询服务、任务重跑服务、任务停止服务。
在本申请实施例中，提供的任务管理方法，在通过接口方法层对任务进行管理时，接口方法层提供了三级任务接口模块，其中包括一级任务接口模块，用于提供一级任务的管理查询服务；二级任务接口模块，用于提供二级任务的管理查询服务；三级任务接口模块，用于提供三级任务的管理查询服务，其中所述管理查询服务包括任务提交服务、任务状态查询服务、任务重跑服务、任务停止服务。
在本申请实施例中，提供的任务管理方法在确定所述待执行任务的工作流描述语言，对所述工作流描述语言进行解析，得到所述待执行任务的执行顺序时，包括：
对于一个待执行一级任务，解析所述待执行一级任务的工作流描述语言，确定所述待执行一级任务中各待执行二级任务的执行顺序、各所述待执行二级任务的依赖关系；和/或
对于一个所述待执行二级任务，确定所述待执行二级任务中各待执行三级任务执行顺序以及各所述待执行三级任务之间的依赖关系。
在本申请实施例中，提供的任务管理方法在通过前述实施例提供的解析层对任务进行管理时，解析层包括工作流解析器，用于对所述待执行任务的工作流描述语言进行解析，确定所述待执行任务的顺序、各任务之间的依赖关系以及参数序列化方法和参数反序列化方法；任务调度接口，所述任务调度接口封装有任务调度执行的最小通用需求，用于对所述待执行任务进行调度，所述任务调度执行的最小通用需求用于规范各计算平台计入所述任务管理系统的功能要求；任务存储接口，所述任务存储接口封装有任务存储执行的最小通用需求，用于对所述待执行任务进行存储，所述任务存储执行的最小通用需求用于规范各存储空间接入所述任务管理系统的功能要求。
在本申请实施例中，提供的任务管理方法在执行所述任务执行壳程序 时，包括：
基于所述待执行任务的执行结果，更新所述待执行任务的任务状态；
检测所述待执行任务是否执行完成，当所述待执行任务未执行完成时，继续执行所述待执行任务；当所述待执行任务执行完成后，检测所述待执行任务之后是否有后续任务，若存在后续任务，则调用后续任务，继续执行所述后续任务。
在本申请实施例中，提供的任务管理方法还可以用于接收用户基于人机交互界面输入的任务状态查询请求；基于所述任务状态查询请求将对应任务的状态发送至所述人机交互界面进行展示。
本申请实施例提供的任务管理方法的实现原理，与前述实施例中提供的任务管理系统的实现原理相同，本申请在此不再赘述。与前述应用功能实现方法实施例相对应，本申请还提供了一种任务管理装置、电子设备及相应的实施例。
图7是本申请实施例示出的任务管理装置的结构示意图。
参见图7，该任务管理装置70包括人机交互模块710、任务调用模块720、解析模块730、以及任务执行模块740，其中：
人机交互模块710，用于接收用户基于人机交互界面输入的任务管理操作；
任务调用模块720，用于基于所述任务管理操作确定待执行任务，调用任务存储接口将所述待执行任务存储至云端，调用任务调用接口将所述待执行任务提交至云计算平台；
解析模块730，用于为所述待执行任务分配容器空间，加载待执行任务壳程序，对所述待执行任务的工作流描述语言进行解析，获得所述待执行任务的任务数据；
任务执行模块740，用于执行所述任务数据，并将执行结果上传至云端存储。
作为本申请一种可能的实施方式，在该实施方式中，任务执行模块740在执行结果上传至云端存储之后，还可以用于：
检测所述待执行任务是否执行完成，当所述待执行任务未执行完成时，继续执行所述待执行任务；当所述待执行任务执行完成后，检测所述待执 行任务之后是否有后续任务，若存在后续任务，则调用后续任务，继续执行后续任务。
关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不再做详细阐述说明。
图8是本申请实施例示出的电子设备的结构示意图。
参见图8，电子设备1000包括存储器1010和处理器1020。
处理器1020可以是中央处理单元(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
存储器1010可以包括各种类型的存储单元，例如系统内存、只读存储器(ROM)和永久存储装置。其中，ROM可以存储处理器1020或者计算机的其他模块需要的静态数据或者指令。永久存储装置可以是可读写的存储装置。永久存储装置可以是即使计算机断电后也不会失去存储的指令和数据的非易失性存储设备。在一些实施方式中，永久性存储装置采用大容量存储装置(例如磁或光盘、闪存)作为永久存储装置。另外一些实施方式中，永久性存储装置可以是可移除的存储设备(例如软盘、光驱)。系统内存可以是可读写存储设备或者易失性可读写存储设备，例如动态随机访问内存。系统内存可以存储一些或者所有处理器在运行时需要的指令和数据。此外，存储器1010可以包括任意计算机可读存储媒介的组合，包括各种类型的半导体存储芯片(例如DRAM，SRAM，SDRAM，闪存，可编程只读存储器)，磁盘和/或光盘也可以采用。在一些实施方式中，存储器1010可以包括可读和/或写的可移除的存储设备，例如激光唱片(CD)、只读数字多功能光盘(例如DVD-ROM，双层DVD-ROM)、只读蓝光光盘、超密度光盘、闪存卡(例如SD卡、min SD卡、Micro-SD卡等)、磁性软盘等。计算机可读存储媒介不包含载波和通过无线或有线传输的瞬间电子信号。
存储器1010上存储有可执行代码，当可执行代码被处理器1020处理 时，可以使处理器1020执行上文述及的方法中的部分或全部。
此外，根据本申请的方法还可以实现为一种计算机程序或计算机程序产品，该计算机程序或计算机程序产品包括用于执行本申请的上述方法中部分或全部步骤的计算机程序代码指令。
或者，本申请还可以实施为一种计算机可读存储介质(或非暂时性机器可读存储介质或机器可读存储介质)，其上存储有可执行代码(或计算机程序或计算机指令代码)，当可执行代码(或计算机程序或计算机指令代码)被电子设备(或服务器等)的处理器执行时，使处理器执行根据本申请的上述方法的各个步骤的部分或全部。
以上已经描述了本申请的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其他普通技术人员能理解本文披露的各实施例。
以上已经描述了本申请的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。

Claims (16)

  1. 一种任务管理系统，其特征在于，所述系统包括：接口方法层，用于提供与人机交互界面连接的接口，并基于所述接口接收用户通过所述人机交互界面输入的任务管理操作，确定待执行任务；解析层，用于确定所述待执行任务的工作流描述语言，对所述工作流描述语言进行解析，得到所述待执行任务的执行顺序，并提供任务调度接口用于调度所述待执行任务，提供任务存储接口用于存储所述待执行任务；执行层，用于提供用于执行所述待执行任务的计算资源和用于存储所述待执行任务的存储资源，提供任务执行壳程序对所述待执行任务的数据进行下载和执行，并在执行结束后将执行结果上传。
  2. 根据权利要求1所述的任务管理系统，其特征在于，任务包括一级任务、二级任务以及三级任务，其中，每个所述一级任务包括一个或多个所述二级任务，每个所述二级任务包括一个或多个所述三级任务，所述接口方法层包括：一级任务接口模块，用于提供一级任务的管理查询服务；二级任务接口模块，用于提供二级任务的管理查询服务；三级任务接口模块，用于提供三级任务的管理查询服务，其中所述管理查询服务包括任务提交服务、任务状态查询服务、任务重跑服务、任务停止服务。
  3. 根据权利要求2所述的任务管理系统，其特征在于，所述解析层在确定所述待执行任务的工作流描述语言，对所述工作流描述语言进行解析，得到所述待执行任务的执行顺序时，可以用于：对于一个待执行一级任务，解析所述待执行一级任务的工作流描述语言，确定所述待执行一级任务中各待执行二级任务的执行顺序、各所述待执行二级任务的依赖关系；和/或对于一个所述待执行二级任务，确定所述待执行二级任务中各待执行三级任务执行顺序以及各所述待执行三级任务之间的依赖关系。
  4. 根据权利要求1所述的任务管理系统，其特征在于，所述解析层 包括：工作流解析器，用于对所述待执行任务的工作流描述语言进行解析，确定所述待执行任务的顺序、各任务之间的依赖关系以及参数序列化方法和参数反序列化方法；任务调度接口，所述任务调度接口封装有任务调度执行的最小通用需求，用于对所述待执行任务进行调度，所述任务调度执行的最小通用需求用于规范各计算平台计入所述任务管理系统的功能要求；任务存储接口，所述任务存储接口封装有任务存储执行的最小通用需求，用于对所述待执行任务进行存储，所述任务存储执行的最小通用需求用于规范各存储空间接入所述任务管理系统的功能要求。
  5. 根据权利要求1所述的任务管理系统，其特征在于，所述执行层在执行所述任务执行壳程序时，还用于：基于所述待执行任务的执行结果，更新所述待执行任务的任务状态；检测所述待执行任务是否执行完成，当所述待执行任务未执行完成时，继续执行所述待执行任务；当所述待执行任务执行完成后，检测所述待执行任务之后是否有后续任务，若存在后续任务，则调用后续任务，继续执行所述后续任务。
  6. 根据权利要求1所述的任务管理系统，其特征在于，所述接口方法层还可以用于：接收用户基于人机交互界面输入的任务状态查询请求；基于所述任务状态查询请求将对应任务的状态发送至所述人机交互界面进行展示。
  7. 一种任务管理方法，其特征在于，所述方法包括：接收用户基于人机交互界面输入的任务管理操作；基于所述任务管理操作确定待执行任务，调用任务存储接口将所述待执行任务存储至云端，调用任务调用接口将所述待执行任务提交至云计算平台；为所述待执行任务分配容器空间，加载待执行任务壳程序，对所述待执行任务的工作流描述语言进行解析，获得所述待执行任务的任务数据；执行所述任务数据，并将执行结果上传至云端存储。
  8. 根据权利要求7所述的任务管理方法，其特征在于，所述将执行结果上传至云端存储之后，还包括：检测所述待执行任务是否执行完成，当所述待执行任务未执行完成时，继续执行所述待执行任务；当所述待执行任务执行完成后，检测所述待执行任务之后是否有后续任务，若存在后续任务，则调用后续任务，继续执行后续任务。
  9. 根据权利要求7所述的任务管理方法，其特征在于，任务包括一级任务、二级任务以及三级任务，其中，每个所述一级任务包括一个或多个所述二级任务，每个所述二级任务包括一个或多个所述三级任务，所述方法包括：对于一个待执行一级任务，解析所述待执行一级任务的工作流描述语言，确定所述待执行一级任务中各待执行二级任务的执行顺序、各所述待执行二级任务的依赖关系；和/或对于一个所述待执行二级任务，确定所述待执行二级任务中各待执行三级任务执行顺序以及各所述待执行三级任务之间的依赖关系。
  10. 根据权利要求7所述的任务管理方法，其特征在于，所述加载待执行任务壳程序，包括：基于所述待执行任务的执行结果，更新所述待执行任务的任务状态；检测所述待执行任务是否执行完成，当所述待执行任务未执行完成时，继续执行所述待执行任务；当所述待执行任务执行完成后，检测所述待执行任务之后是否有后续任务，若存在后续任务，则调用后续任务，继续执行所述后续任务。
  11. 一种任务管理装置，其特征在于，所述任务管理装置包括：
    人机交互模块，用于接收用户基于人机交互界面输入的任务管理操作；
    任务调用模块，用于基于所述任务管理操作确定待执行任务，调用任务存储接口将所述待执行任务存储至云端，调用任务调用接口将所述待执行任务提交至云计算平台；
    解析模块，用于为所述待执行任务分配容器空间，加载待执行任务壳程序，对所述待执行任务的工作流描述语言进行解析，获得所述待执行任务的任务数据；
    任务执行模块，用于执行所述任务数据，并将执行结果上传至云端存储。
  12. 根据权利要求11所述的任务管理装置，其特征在于，所述任务执行模块在将执行结果上传至云端存储之后，还可以用于：检测所述待执行任务是否执行完成，当所述待执行任务未执行完成时，继续执行所述待执行任务；当所述待执行任务执行完成后，检测所述待执行任务之后是否有后续任务，若存在后续任务，则调用后续任务，继续执行后续任务。
  13. 根据权利要求11所述的任务管理装置，其特征在于，任务包括一级任务、二级任务以及三级任务，其中，每个所述一级任务包括一个或多个所述二级任务，每个所述二级任务包括一个或多个所述三级任务，所述装置还可以用于：对于一个待执行一级任务，解析所述待执行一级任务的工作流描述语言，确定所述待执行一级任务中各待执行二级任务的执行顺序、各所述待执行二级任务的依赖关系；和/或对于一个所述待执行二级任务，确定所述待执行二级任务中各待执行三级任务执行顺序以及各所述待执行三级任务之间的依赖关系。
  14. 根据权利要求11所述的任务管理装置，其特征在于，所述解析模块在加载待执行任务壳程序时，可以用于：基于所述待执行任务的执行结果，更新所述待执行任务的任务状态；检测所述待执行任务是否执行完成，当所述待执行任务未执行完成时，继续执行所述待执行任务；当所述待执行任务执行完成后，检测所述待执行任务之后是否有后续任务，若存在后续任务，则调用后续任务，继续执行所述后续任务。
  15. 一种电子设备，其特征在于，包括：处理器；以及存储器，其上存储有可执行代码，当所述可执行代码被所述处理器执行时，使所述处理器执行如权利要求7-8中任一项所述的任务管理方法。
  16. 一种计算机可读存储介质，其上存储有可执行代码，当所述可执 行代码被电子设备的处理器执行时，使所述处理器执行如权利要求7-8中任一项所述的任务管理方法。
PCT/CN2021/136975 2021-12-10 2021-12-10 任务管理系统、方法、装置、设备及存储介质 WO2023102869A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/136975 WO2023102869A1 (zh) 2021-12-10 2021-12-10 任务管理系统、方法、装置、设备及存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/136975 WO2023102869A1 (zh) 2021-12-10 2021-12-10 任务管理系统、方法、装置、设备及存储介质

Publications (1)

Publication Number Publication Date
WO2023102869A1 true WO2023102869A1 (zh) 2023-06-15

Family

ID=86729341

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/136975 WO2023102869A1 (zh) 2021-12-10 2021-12-10 任务管理系统、方法、装置、设备及存储介质

Country Status (1)

Country Link
WO (1) WO2023102869A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107071004A (zh) * 2017-03-24 2017-08-18 上海云首信息科技有限公司 一种混合云控制调度系统和调度方法
CN111506412A (zh) * 2020-04-22 2020-08-07 上海德拓信息技术股份有限公司 基于Airflow的分布式异步任务构建、调度系统及方法
CN111897622A (zh) * 2020-06-10 2020-11-06 中国科学院计算机网络信息中心 基于容器技术的高通量计算方法及系统
CN113220431A (zh) * 2021-04-29 2021-08-06 西安易联趣网络科技有限责任公司 跨云的分布式数据任务调度方法、设备及存储介质
CN113225269A (zh) * 2021-04-16 2021-08-06 鹏城实验室 基于容器的工作流调度方法、装置、系统及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107071004A (zh) * 2017-03-24 2017-08-18 上海云首信息科技有限公司 一种混合云控制调度系统和调度方法
CN111506412A (zh) * 2020-04-22 2020-08-07 上海德拓信息技术股份有限公司 基于Airflow的分布式异步任务构建、调度系统及方法
CN111897622A (zh) * 2020-06-10 2020-11-06 中国科学院计算机网络信息中心 基于容器技术的高通量计算方法及系统
CN113225269A (zh) * 2021-04-16 2021-08-06 鹏城实验室 基于容器的工作流调度方法、装置、系统及存储介质
CN113220431A (zh) * 2021-04-29 2021-08-06 西安易联趣网络科技有限责任公司 跨云的分布式数据任务调度方法、设备及存储介质

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
HALO CLOUD COMMUNITY: "Why Do Businesses Need Hybrid Cloud? How to design hybrid cloud storage architecture?", 12 June 2020 (2020-06-12), XP009546204, Retrieved from the Internet <URL:https://zhuanlan.zhihu.com/p/147749205> *

Similar Documents

Publication Publication Date Title
Calheiros et al. CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms
US20180365005A1 (en) Distributed parallel build system
US9720709B1 (en) Software container recommendation service
US9712410B1 (en) Local metrics in a service provider environment
US10534582B2 (en) Application deployment on a host platform based on text tags descriptive of application requirements
AU2020201712A1 (en) Prescriptive analytics based committed compute reservation stack for cloud computing resource scheduling
Malawski Towards Serverless Execution of Scientific Workflows-HyperFlow Case Study.
JP2022097438A (ja) ロボティックプロセスオートメーション（ｒｐａ）ロボットの動的クラウドデプロイメント
Ciavotta et al. A mixed integer linear programming optimization approach for multi-cloud capacity allocation
Spinner et al. Online model learning for self-aware computing infrastructures
Galleguillos et al. AccaSim: a customizable workload management simulator for job dispatching research in HPC systems
Mujezinović et al. Serverless architecture for workflow scheduling with unconstrained execution environment
Harichane et al. KubeSC‐RTP: Smart scheduler for Kubernetes platform on CPU‐GPU heterogeneous systems
CN111144830A (zh) 一种企业级计算资源管理方法、系统和计算机设备
WO2022109932A1 (zh) 基于slurm计算平台的多任务提交系统
Lublinsky et al. A kubernetes ‘bridge’operator between cloud and external resources
WO2023102869A1 (zh) 任务管理系统、方法、装置、设备及存储介质
WO2023287563A1 (en) Apparatus, articles of manufacture, and methods for managing processing units
Lebesbye et al. Boreas–a service scheduler for optimal kubernetes deployment
CN114185660A (zh) 任务管理系统、方法、装置、设备及存储介质
US11971705B2 (en) Autoscaling strategies for robotic process automation
Sukhoroslov Supporting efficient execution of many-task applications with Everest
Goodwin et al. What goes wrong in serverless runtimes? A survey of bugs in Knative Serving
US11966726B2 (en) Operating system (OS) scheduler and compiler for code generation optimization in a (simultaneous multi-threading) SMT enabled CPU
US20240232622A9 (en) Apparatus, articles of manufacture, and methods for managing processing units

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21966791

Country of ref document: EP

Kind code of ref document: A1