CN112748997A - Workflow scheduling method and system - Google Patents

Workflow scheduling method and system Download PDF

Info

Publication number
CN112748997A
CN112748997A CN202110076449.6A CN202110076449A CN112748997A CN 112748997 A CN112748997 A CN 112748997A CN 202110076449 A CN202110076449 A CN 202110076449A CN 112748997 A CN112748997 A CN 112748997A
Authority
CN
China
Prior art keywords
task
workflow
tasks
end task
hash table
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110076449.6A
Other languages
Chinese (zh)
Inventor
杜泉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Minglue Zhaohui Technology Co Ltd
Original Assignee
Beijing Minglue Zhaohui Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Minglue Zhaohui Technology Co Ltd filed Critical Beijing Minglue Zhaohui Technology Co Ltd
Priority to CN202110076449.6A priority Critical patent/CN112748997A/en
Publication of CN112748997A publication Critical patent/CN112748997A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/48Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806Task transfer initiation or dispatching
    • G06F9/4843Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals

Abstract

The application discloses a method and a system for scheduling workflow. The workflow scheduling method comprises the following steps: a submitting step: submitting tasks of the workflow through the front end; the setting step: setting a virtual root and a hash table according to the task; the execution steps are as follows: executing the task according to the virtual root and the hash table; and (3) a persistence step: and after all the tasks are finished, the task state of the tasks in the workflow is persisted into a database. The invention provides a method and a system for dispatching a workflow, which organize tasks through DAG, organize the state dependency relationship of a plurality of subtasks for managing a workflow task when a plurality of clusters execute the task, and store the state of the corresponding subtask through a hash table.

Description

Workflow scheduling method and system
Technical Field
The present application relates to the technical field of workflow management, and in particular, to a method and a system for scheduling a workflow.
Background
argo is a workflow tool implemented based on kubernets CRD. However, argo is applicable to workflow scheduling for a single cluster, and still cannot be supported in terms of submitting workflows across kubernets clusters. It is temporarily impossible to organize the state dependencies of the multiple subtasks managing one workflow task when the multiple clusters execute the task.
Therefore, in view of the above situation, the present invention provides a method and a system for scheduling a workflow, where the invention organizes tasks through a DAG, organizes a state dependency relationship when a plurality of subtasks managing one workflow task execute tasks in a plurality of clusters, and saves the states of the corresponding subtasks through a hash table.
Disclosure of Invention
The embodiment of the application provides a method and a system for scheduling a workflow, which are used for at least solving the problem of subjective factor influence in the related technology.
The invention provides a method for scheduling a workflow, which comprises the following steps:
a submitting step: submitting tasks of the workflow through the front end;
the setting step: setting a virtual root and a hash table according to the task;
the execution steps are as follows: executing the task according to the virtual root and the hash table;
and (3) a persistence step: and after all the tasks are finished, the task state of the tasks in the workflow is persisted into a database.
In the method for scheduling a workflow, the submitting step includes, after the tasks of the workflow are filled in through the front end and the dependency relationships of the tasks are selected, submitting the tasks.
In the above method for scheduling a workflow, the setting step includes setting an initial value of the task in the hash table to 0, where the task includes the front-end task and the back-end task.
In the method for scheduling a workflow, the executing step includes setting the initial value of the front-end task to 1 after the front-end task is correctly executed, and sending a signal to notify a scanning state queue of the back-end task depending on the front-end task that the back-end task starts to execute, and if the front-end task fails, the back-end task is not executed any more.
In the above method for scheduling a workflow, the step of persisting includes persisting the task state of the task in the workflow in the database after all the tasks are executed or the tasks fail.
The present invention further provides a workflow scheduling system, wherein the workflow scheduling system is suitable for the above workflow scheduling method, and the workflow scheduling system includes:
a submission unit: submitting tasks of the workflow through the front end;
a setting unit: setting a virtual root and a hash table according to the task;
an execution unit: executing the task according to the virtual root and the hash table;
a persistence unit: and after all the tasks are finished, the task state of the tasks in the workflow is persisted into a database.
The system for dispatching the workflow fills in the tasks of the workflow through the front end, selects the dependency relationship of the tasks and submits the tasks through the submitting unit.
In the above workflow scheduling system, the setting unit sets an initial value of the task in the hash table to 0, where the task includes the front-end task and the back-end task.
In the above workflow scheduling system, after the front-end task is correctly executed, the execution unit sets the initial value of the front-end task to 1, and sends a signal to notify a scanning state queue of the back-end task that depends on the front-end task, and then the execution unit starts to execute the back-end task, and if the front-end task fails, the back-end task is not executed any more.
In the above workflow scheduling system, when all the tasks are finished or the tasks fail to be executed, the persistence unit persists the task state of the task in the workflow in the database.
Compared with the prior art, the invention provides a method and a system for dispatching a workflow, the invention organizes tasks through DAG, organizes the state dependency relationship of a plurality of subtasks managing one workflow task when the plurality of clusters execute the task, and saves the state of the corresponding subtasks through a hash table.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the application.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a flow chart of a method for scheduling workflows according to an embodiment of the present application;
FIG. 2 is a flow diagram of a workflow scheduling framework according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a workflow scheduling system according to the present invention;
fig. 4 is a frame diagram of an electronic device according to an embodiment of the present application.
Wherein the reference numerals are:
a submission unit: 51;
a setting unit: 52;
an execution unit: 53;
a persistence unit: 54, a first electrode;
81: a processor;
82: a memory;
83: a communication interface;
80: a bus.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments provided in the present application without any inventive step are within the scope of protection of the present application.
It is obvious that the drawings in the following description are only examples or embodiments of the present application, and that it is also possible for a person skilled in the art to apply the present application to other similar contexts on the basis of these drawings without inventive effort. Moreover, it should be appreciated that such a development effort might be complex and tedious, but would nevertheless be a routine undertaking of design, fabrication, and manufacture for those of ordinary skill having the benefit of this disclosure, and thus should not be construed as a limitation of this disclosure.
Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the specification. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of ordinary skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments without conflict.
Unless defined otherwise, technical or scientific terms referred to herein shall have the ordinary meaning as understood by those of ordinary skill in the art to which this application belongs. Reference to "a," "an," "the," and similar words throughout this application are not to be construed as limiting in number, and may refer to the singular or the plural. The present application is directed to the use of the terms "including," "comprising," "having," and any variations thereof, which are intended to cover non-exclusive inclusions; for example, a process, method, system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to the listed steps or elements, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. Reference to "connected," "coupled," and the like in this application is not intended to be limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" as referred to herein means two or more. "and/or" describes an association relationship of associated objects, meaning that three relationships may exist, for example, "A and/or B" may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. Reference herein to the terms "first," "second," "third," and the like, are merely to distinguish similar objects and do not denote a particular ordering for the objects.
The invention is based on the Kubernetes cluster, which is briefly described below.
kubernets, K8s for short, is an abbreviation for 8 instead of 8 characters "ubernet". The Kubernetes is an open source and used for managing containerized applications on a plurality of hosts in a cloud platform, aims to make the application of the containerization simple and efficient to deploy (powerfull), and provides a mechanism for deploying, planning, updating and maintaining the applications. A traditional deployment of applications is to install the applications through plug-ins or scripts. The disadvantage of this is that the running, configuration, management, and all life cycles of the application will be bound to the current operating system, which is not beneficial to the upgrade update/rollback and other operations of the application, and certainly, some functions can be implemented by creating a virtual machine, but the virtual machine is very heavy and not beneficial to portability. The new mode is realized by deploying containers, each container is isolated from each other, each container has a file system, processes among the containers cannot influence each other, and computing resources can be distinguished. Compared with a virtual machine, the container can be deployed rapidly, and the container can be migrated among different clouds and different versions of operating systems because the container is decoupled from underlying facilities and a machine file system. The container occupies less resources and is fast to deploy, each application can be packaged into a container mirror image, the container has greater advantages due to the one-to-one relationship between each application and the container, and the container mirror image can be created for the application at the stage of build or release by using the container, because each application does not need to be combined with the rest of application stacks and does not depend on the production environment infrastructure, and a consistent environment can be provided from research and development to test and production. Similarly, containers are lighter weight, more "transparent" than virtual machines, which is more convenient to monitor and manage. Kubernetes is a container orchestration engine for Google open sources that supports automated deployment, large-scale scalable, application containerization management. When an application is deployed in a production environment, multiple instances of the application are typically deployed to load balance application requests. In Kubernetes, we can create multiple containers, each container runs an application instance inside, and then manage, discover and access the group of application instances through a built-in load balancing policy, and all the details do not need operation and maintenance personnel to perform complicated manual configuration and processing. A trunked communication system is a computer system that cooperates to perform computing tasks with a high degree of compactness through a set of loosely integrated computer software and/or hardware connections. In a sense, they may be considered a computer. The individual computers in a clustered system, often referred to as nodes, are typically connected by a local area network, but there are other possible connections. Clustered computers are often used to improve the computing speed and/or reliability of individual computers. Typically, clustered computers are much more cost effective than individual computers, such as workstations or supercomputers. The most characteristic of the trunking communication is that the voice communication uses PTT (push To talk) and is connected in a one-press instant communication mode, the called party can answer without off-hook, the connection speed is fast, and the functions of group calling and the like can be supported. China introduced the analog trunking system starting in 1989 and was put into use in 1990. With the development of digital communication technology, trunked communication systems are also beginning to develop to the second generation digital technology, and the most important characteristic is that TDMA (time division multiple access) and CDMA (code division multiple access) communication modes are adopted. However, the trunking communication application in china mainly stays in the analog technology level, and the digital trunking application is less. Meanwhile, as each cluster using enterprise adopts a scheme of independently constructing a cluster communication network in order to meet different use requirements of each cluster using enterprise, the cluster networks of numerous enterprises have a plurality of problems in the aspects of internetworking interoperability, frequency resource use, overall construction and the like. In addition, foreign communication is leading to basically undisclosed internal interfaces, poor technical openness and high market prices of systems and terminal equipment by controlling core technologies and setting intellectual property protection barriers such as patents and the like, and also restricts the industrialization process and scale application of Chinese digital clusters. Aiming at the embarrassing situation of the development of the Chinese digital trunking industry, the department of information industry draws a lead to make the development plan of the Chinese trunking technology, brings the digital trunking into the category of basic telecommunication services for the first time in a new telecommunication management regulation, and organizes six domestic telecommunication operators to develop 800 million digital trunking commercial experiments in China. From the experimental situation of the operator, there are the china defending developing the china GoTa co-networking commercial experiment based on the CDMA technical system in the south of china, Nanjing and Tianjin, and the china Tietong developing the china digital cluster co-networking commercial experiment based on the GoTa of the CDMA technical system and the GT800 of the GSM technical system in Shenyang, Changchun and Chongqing. The method is not ideal from the commercial experimental situation in recent years, and no commercial operation mode satisfying the country has been explored in the relation of operation cost, market demand, operation mode, maturity and applicability of relevant standards and public mobile communication service. Because the target customers facing the trunking communication and the public mobile communication are different, the development of the digital trunking is different from the development of the traditional public mobile communication, and the development mode of 'establishing a network first and then marketing' is adopted, so that the marketing is the most objective standard for checking a product. Considering the imbalance of economic development of regions and industries in China, in order to reduce risks and avoid blind investment, a development strategy of digital cluster common network must break through the development mode of traditional public mobile communication, and a market development strategy of 'market first and then network construction' is followed, and a development idea of grading, dividing industries and dividing regions is adopted. The development status of digital trunking communication in China is still in an early stage compared with that of developed regions in China, a digital trunking system with independent intellectual property rights is just developed, and a large development space exists from the aspects of market development of digital trunking communication and development of national industries, so that the status is a challenge and opportunity for development of digital trunking communication systems in China in the future. In the process of digital trunking communication development in China, the relationship with a public mobile communication system is firstly processed. The digital trunking communication is a private mobile communication system mainly providing a command and dispatch service for group users, and the public mobile communication is a public mobile communication system mainly providing voice and data services for common mass users. The two are positioned differently and have different technical characteristics. The repeated and past curves for providing public network services via a trunking system should be avoided. Meanwhile, from the technical development level of the public mobile communication system, the public mobile communication system cannot replace a trunking system, the public network provides the PTT service only as a value added service, the performance index of the PTT service far cannot meet the requirement of group users on professional command and scheduling communication, and meanwhile, various service quality levels and priorities required by trunking communication cannot be provided. These functions are very important in clustered networks, especially in clustered co-networks.
Workflow (Workflow) refers to "automation of a portion or the whole of a business process in a computer application environment". The method is an abstract and general description of the workflow and the business rules among the operation steps of the workflow. In computers, workflows are part of Computer Supported Collaborative Work (CSCW). The latter is a general study on how a group can achieve cooperative work with the help of computers. The main problems mainly solved by the workflow are as follows: to achieve a business goal, documents, information, or tasks are automatically communicated between multiple participants using a computer according to some predetermined rule. The workflow concept originates from the field of production organization and office automation, is a concept provided aiming at the fixed program activities in daily work, and aims to decompose the work into well-defined tasks or roles, execute the tasks according to certain rules and processes and monitor the tasks, so that the aims of improving the work efficiency, better controlling the process, enhancing the service to customers, effectively managing the business process and the like are fulfilled. Although workflows have achieved considerable success, the definition of workflows has not been unified and unambiguous. The workflow definition given by georgakokoulos is: a workflow is the organization of a set of tasks to complete a business process: the triggering sequence and triggering conditions of the tasks are defined, and each task can be completed by one or more software systems, one or a group of people, and one or more people and the software systems in cooperation. WorkFlow (WorkFlow) is a computational model of a WorkFlow, i.e., the logic and rules of how the work in the WorkFlow is organized one after the other are represented in a computer in an appropriate model and computed. The main problems to be solved by the workflow are: to achieve a business goal, automatic delivery is made between multiple participants, using computers, according to certain predefined rules. The workflow is part of a Computer Supported Collaborative Work (CSCW). The latter is a general study on how a group can achieve cooperative work with the help of computers. Many companies adopt a mode of paper form and manual transmission, the signature is approved at the first level, the working efficiency is very low, and the function of the statistical statement cannot be realized. And by adopting workflow software, a user can automatically run down according to a defined flow by only filling related forms in a computer, and a next-level approver can receive related data and modify, track, manage, query, count, print and the like according to needs, so that the efficiency is greatly improved, the knowledge management is realized, and the core competitiveness of a company is improved. The workflow has the following characteristics: a graphical and visual design flow chart; various complex processes are supported; organization level handlers specify functions; B/S structure, pure browser application; strong safety features; the form has strong functions and is convenient to expand; flexible outgoing and overtime management strategies; the processing process can be tracked and managed; abundant statistics, inquiry and report functions; can be integrated with a MAIL system. A workflow management system is a system that defines, creates, and executes workflows. At the highest level, a WFMS should provide functional support in three areas: building functions: defining and modeling a workflow process and the composition activities thereof; the operation control function is as follows: managing a workflow process in an operating environment, and scheduling activities in the workflow process; and (4) operating an interactive function: refers to the functionality of the WFMS interacting with users (participants or controllers of business jobs) and external application tools during workflow operations. Due to the development of information technology and the ever-increasing commercial competition, people are no longer satisfied with independent, sporadic office automation and computer applications, but rather require comprehensive, integrated solutions. The advent of WFMS is a technology for managing and integrating regular transactions. It can bring the following benefits: the business process is improved and optimized, and the business working efficiency is improved; better business process control is realized, and the service quality of customers is improved; and the flexibility of the business process is improved.
The invention provides a method and a system for dispatching a workflow, which organize tasks through DAG, organize the state dependency relationship of a plurality of subtasks for managing a workflow task when a plurality of clusters execute the task, and store the state of the corresponding subtask through a hash table.
The following describes an embodiment of the present application by taking scheduling of workflows as an example.
Example one
The embodiment provides a workflow scheduling method. Referring to fig. 1-2, fig. 1 is a flowchart illustrating a method for scheduling workflows according to an embodiment of the present disclosure; fig. 2 is a framework diagram of a scheduling flow of a workflow according to an embodiment of the present application, and as shown in the figure, a scheduling method of a workflow includes the following steps:
submission step S1: submitting tasks of the workflow through the front end;
setting step S2: setting a virtual root and a hash table according to the task;
step S3 is executed: executing the task according to the virtual root and the hash table;
persistence step S4: and after all the tasks are finished, the task state of the tasks in the workflow is persisted into a database.
In an embodiment, the submitting step S1 includes submitting the task after filling in the task of the workflow through a front end and selecting the dependency relationship of the task.
In specific implementation, a self-developed workflow framework is adopted to support the simultaneous or sequential execution of tasks in a plurality of kubernets clusters according to the dependency relationship, a user fills in the tasks of the workflow through the front end and submits the tasks after selecting the dependency relationship of the workflow tasks, and the system organizes the workflow by adopting a directed acyclic graph after receiving the tasks.
In an embodiment, the setting step S2 includes setting an initial value of the task in the hash table to 0, where the task includes the front-end task and the back-end task.
In a specific implementation, since a workflow may have a plurality of root nodes, for example, there is no pre-dependency, tasks may be run simultaneously, and for convenience of organization, a virtual root is established to ensure that the workflow is always in a tree structure. Thus, the execution order of a plurality of tasks in a workflow can be determined. In order to ensure that the task of a workflow can be executed correctly at the appropriate time, a hash table is additionally arranged, wherein a key is the name of a subtask, the task value is initially set to 0, and the hash table is set to 1 when the task is executed correctly. When the user modifies the workflow and reruns, the previously successful subtasks are skipped by switching over the previously successful subtasks in order to save time, and the system checks the previous status and skips the previously successful subtasks.
In an embodiment, the executing step S3 includes setting the initial value of the front-end task to 1 after the front-end task is correctly executed, and sending a signal to notify that the back-end task starts executing after the scanning status queue of the back-end task dependent on the front-end task is notified, and if the front-end task fails, the back-end task is not executed any more.
In specific implementation, in order to prevent a task from failing due to unexpected factors such as network downtime and machine downtime, the failed task is retried twice by default.
In an embodiment, the persisting step S4 includes persisting the task state of the task in the workflow in the database when all the tasks are finished executing or the tasks fail.
The invention organizes tasks through DAG, organizes state dependency relationship when a plurality of subtasks managing a workflow task execute tasks in a plurality of clusters, and saves the state of the corresponding subtasks through a hash table.
Example two
Referring to fig. 3, fig. 3 is a schematic structural diagram of a workflow scheduling system according to the present invention. As shown in fig. 3, the workflow scheduling system of the present invention is applicable to the above workflow scheduling method, and includes:
the submission unit 51: submitting tasks of the workflow through the front end;
the setting unit 52: setting a virtual root and a hash table according to the task;
the execution unit 53: executing the task according to the virtual root and the hash table;
the persistence unit 54: and after all the tasks are finished, the task state of the tasks in the workflow is persisted into a database.
In this embodiment, the tasks of the workflow are filled in through the front end, and the tasks are submitted through the submitting unit 51 after the dependency relationships of the tasks are selected.
In this embodiment, the setting unit 52 sets an initial value of the task in the hash table to 0, where the task includes the front-end task and the back-end task.
In this embodiment, after the front-end task completes correct execution, the execution unit 53 sets the initial value of the front-end task to 1, and sends a signal to notify the scanning state queue of the back-end task that depends on the front-end task, and then the execution unit starts to execute the back-end task, and if the front-end task fails, the back-end task is not executed any more.
In this embodiment, when all the tasks are finished executing or the tasks fail to execute, the persistence unit 54 persists the task state of the task in the workflow in the database.
EXAMPLE III
Referring to fig. 4, this embodiment discloses a specific implementation of an electronic device. The electronic device may include a processor 81 and a memory 82 storing computer program instructions.
Specifically, the processor 81 may include a Central Processing Unit (CPU), or A Specific Integrated Circuit (ASIC), or may be configured to implement one or more Integrated circuits of the embodiments of the present Application.
Memory 82 may include, among other things, mass storage for data or instructions. By way of example, and not limitation, memory 82 may include a Hard Disk Drive (Hard Disk Drive, abbreviated to HDD), a floppy Disk Drive, a Solid State Drive (SSD), flash memory, an optical Disk, a magneto-optical Disk, tape, or a Universal Serial Bus (USB) Drive or a combination of two or more of these. Memory 82 may include removable or non-removable (or fixed) media, where appropriate. The memory 82 may be internal or external to the data processing apparatus, where appropriate. In a particular embodiment, the memory 82 is a Non-Volatile (Non-Volatile) memory. In particular embodiments, Memory 82 includes Read-Only Memory (ROM) and Random Access Memory (RAM). The ROM may be mask-programmed ROM, Programmable ROM (PROM), Erasable PROM (FPROM), Electrically Erasable PROM (EFPROM), Electrically rewritable ROM (EAROM), or FLASH Memory (FLASH), or a combination of two or more of these, where appropriate. The RAM may be a Static Random-Access Memory (SRAM) or a Dynamic Random-Access Memory (DRAM), where the DRAM may be a Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), an Extended data output Dynamic Random-Access Memory (EDODRAM), a Synchronous Dynamic Random-Access Memory (SDRAM), and the like.
The memory 82 may be used to store or cache various data files for processing and/or communication use, as well as possible computer program instructions executed by the processor 81.
The processor 81 implements the scheduling method of any workflow in the above embodiments by reading and executing computer program instructions stored in the memory 82.
In some of these embodiments, the electronic device may also include a communication interface 83 and a bus 80. As shown in fig. 4, the processor 81, the memory 82, and the communication interface 83 are connected via the bus 80 to complete communication therebetween.
The communication interface 83 is used for implementing communication between modules, devices, units and/or equipment in the embodiment of the present application. The communication port 83 may also be implemented with other components such as: and data communication is carried out among external equipment, image/workflow scheduling equipment, a database, external storage, an image/data processing workstation and the like.
The bus 80 includes hardware, software, or both to couple the components of the electronic device to one another. Bus 80 includes, but is not limited to, at least one of the following: data Bus (Data Bus), Address Bus (Address Bus), Control Bus (Control Bus), Expansion Bus (Expansion Bus), and Local Bus (Local Bus). By way of example, and not limitation, Bus 80 may include an Accelerated Graphics Port (AGP) or other Graphics Bus, an Enhanced Industry Standard Architecture (EISA) Bus, a Front-Side Bus (FSB), a Hyper Transport (HT) Interconnect, an ISA (ISA) Bus, an InfiniBand (InfiniBand) Interconnect, a Low Pin Count (LPC) Bus, a memory Bus, a microchannel Architecture (MCA) Bus, a PCI (Peripheral Component Interconnect) Bus, a PCI-Express (PCI-X) Bus, a Serial Advanced Technology Attachment (SATA) Bus, a Video Electronics Bus (audio Electronics Association), abbreviated VLB) bus or other suitable bus or a combination of two or more of these. Bus 80 may include one or more buses, where appropriate. Although specific buses are described and shown in the embodiments of the application, any suitable buses or interconnects are contemplated by the application.
The electronic device may be connected to a workflow scheduling system to implement the methods described in connection with fig. 1-2.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (10)

1. A method for scheduling a workflow, comprising:
a submitting step: submitting tasks of the workflow through the front end;
the setting step: setting a virtual root and a hash table according to the task;
the execution steps are as follows: executing the task according to the virtual root and the hash table;
and (3) a persistence step: and after all the tasks are finished, the task state of the tasks in the workflow is persisted into a database.
2. The method of claim 1, wherein the submitting step comprises submitting the tasks after the tasks of the workflow are filled in and the dependencies of the tasks are selected by a front end.
3. The method according to claim 1, wherein the setting step comprises setting an initial value of the tasks in the hash table to 0, wherein the tasks comprise the front-end task and the back-end task.
4. The workflow scheduling method according to claim 3, wherein the executing step includes setting the initial value of the front-end task to 1 after the front-end task is correctly executed, and sending a signal to notify a scanning state queue of the back-end task depending on the front-end task that the back-end task starts to execute, and if the front-end task fails, the back-end task is not executed any more.
5. The method according to claim 1, wherein the persisting step comprises persisting the task state of the task in the workflow to the database when all the tasks are finished executing or the tasks fail.
6. A workflow scheduling system, which is applied to the workflow scheduling method according to any one of claims 1 to 5, the workflow scheduling system comprising:
a submission unit: submitting tasks of the workflow through the front end;
a setting unit: setting a virtual root and a hash table according to the task;
an execution unit: executing the task according to the virtual root and the hash table;
a persistence unit: and after all the tasks are finished, the task state of the tasks in the workflow is persisted into a database.
7. The workflow scheduling system of claim 6 wherein the tasks of the workflow are submitted by the submission unit after the tasks are filled in by a front end and the dependencies of the tasks are selected.
8. The workflow scheduling system according to claim 7, wherein the setting unit sets an initial value of the tasks in the hash table to 0, wherein the tasks include the front-end task and the back-end task.
9. The workflow scheduling system of claim 8, wherein the execution unit sets the initial value of the front-end task to 1 after the front-end task is executed correctly, and sends a signal to notify a scan status queue of the back-end task that depends on the front-end task, and the execution unit starts executing the back-end task, and if the front-end task fails, the back-end task is not executed.
10. The workflow scheduling system of claim 9 wherein said persistence unit persists the task state of the tasks in the workflow to the database after all of the tasks have completed execution or the tasks have failed.
CN202110076449.6A 2021-01-20 2021-01-20 Workflow scheduling method and system Pending CN112748997A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110076449.6A CN112748997A (en) 2021-01-20 2021-01-20 Workflow scheduling method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110076449.6A CN112748997A (en) 2021-01-20 2021-01-20 Workflow scheduling method and system

Publications (1)

Publication Number Publication Date
CN112748997A true CN112748997A (en) 2021-05-04

Family

ID=75652600

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110076449.6A Pending CN112748997A (en) 2021-01-20 2021-01-20 Workflow scheduling method and system

Country Status (1)

Country Link
CN (1) CN112748997A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023045478A1 (en) * 2021-09-22 2023-03-30 中科寒武纪科技股份有限公司 Graph task scheduling method, execution-end device, storage medium, and program product
WO2023122362A1 (en) * 2021-12-23 2023-06-29 Illumina Software, Inc. Facilitating secure execution of external workflows for genomic sequencing diagnostics

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023045478A1 (en) * 2021-09-22 2023-03-30 中科寒武纪科技股份有限公司 Graph task scheduling method, execution-end device, storage medium, and program product
CN115878272A (en) * 2021-09-22 2023-03-31 中科寒武纪科技股份有限公司 Graph task scheduling method, execution end device, storage medium and program product
WO2023122362A1 (en) * 2021-12-23 2023-06-29 Illumina Software, Inc. Facilitating secure execution of external workflows for genomic sequencing diagnostics

Similar Documents

Publication Publication Date Title
US8990812B2 (en) Task decomposition with throttled message processing in a heterogeneous environment
CN108614726B (en) Virtual machine creation method and device
US8150889B1 (en) Parallel processing framework
CN110825535B (en) Job scheduling method and system
US20230333914A1 (en) Management instrumentation and discovery (mid) server support for executing automated flows within a cloud based system
CN106033373A (en) A method and a system for scheduling virtual machine resources in a cloud computing platform
CN107077379B (en) A kind of virtual machine creation method and device
CN112748997A (en) Workflow scheduling method and system
CN104050042A (en) Resource allocation method and resource allocation device for ETL (Extraction-Transformation-Loading) jobs
CN113298503A (en) Government affair-oriented workflow management system and database and table dividing method thereof
CN102968303A (en) Program design system and program design method
CN110019123A (en) A kind of data migration method and device
CN1326567A (en) Job-parallel processor
CN112559143A (en) Task scheduling method and system and computing device
CN102243655B (en) Method and device for managing database connections
EP3997568B1 (en) Workflow-based scheduling and batching in multi-tenant distributed systems
CN114510317A (en) Virtual machine management method, device, equipment and storage medium
CN111767126A (en) System and method for distributed batch processing
CN101754156A (en) Office data management method, device and charging system
US11755297B2 (en) Compiling monoglot function compositions into a single entity
CN115964176A (en) Cloud computing cluster scheduling method, electronic device and storage medium
CN110928659A (en) Numerical value pool system remote multi-platform access method with self-adaptive function
CN113485812B (en) Partition parallel processing method and system based on large-data-volume task
CN114237858A (en) Task scheduling method and system based on multi-cluster network
CN111880905B (en) Method for migrating virtual machines across cells by using openstack

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination