WO2016101423A1 - 作业调度方法和云调度服务器 - Google Patents

作业调度方法和云调度服务器 Download PDF

Info

Publication number
WO2016101423A1
WO2016101423A1 PCT/CN2015/074641 CN2015074641W WO2016101423A1 WO 2016101423 A1 WO2016101423 A1 WO 2016101423A1 CN 2015074641 W CN2015074641 W CN 2015074641W WO 2016101423 A1 WO2016101423 A1 WO 2016101423A1
Authority
WO
WIPO (PCT)
Prior art keywords
job
server
scheduling
cloud
zookeeper
Prior art date
Application number
PCT/CN2015/074641
Other languages
English (en)
French (fr)
Inventor
郭蕊
Original Assignee
中兴通讯股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中兴通讯股份有限公司 filed Critical 中兴通讯股份有限公司
Publication of WO2016101423A1 publication Critical patent/WO2016101423A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]

Definitions

  • This paper covers the field of network communication, especially related to job scheduling methods and cloud scheduling servers.
  • a scheduling server cluster for example, Quartz Scheduler in the Java field
  • Quartz Scheduler in the Java field
  • the scheduling server cluster is difficult to maintain. When updating the scheduled jobs, it is often necessary to stop and restart all the servers in the cluster.
  • the weight cannot be set, and the higher load cannot be allocated to the server with better performance
  • the system does not distinguish between the scheduling server and the scheduling client, and cannot increase or decrease the number of servers arbitrarily.
  • the embodiment of the invention provides a job scheduling method and a cloud scheduling server, which solves the problem that the scheduling server cluster is difficult to maintain and deploy.
  • the job on the cloud job server is remotely scheduled based on the acquired job information.
  • the step of acquiring job information deployed in the cloud job server includes:
  • the cloud job server is registered on the job node of the ZooKeeper server;
  • the job information deployed in the cloud job server is obtained according to the job node of the monitored ZooKeeper server.
  • the step of remotely invoking a job on the cloud job server according to the acquired job information further includes:
  • the step of remotely invoking a job on the cloud job server according to the acquired job information includes:
  • the timing scheduling, the stop timing scheduling, and the pause timing scheduling are initiated by the unique identifier of the job.
  • the step of remotely invoking a job on the cloud job server according to the acquired job information includes:
  • the load balancing algorithm is used for scheduling.
  • the embodiment of the invention further provides a cloud scheduling server, where the cloud scheduling server includes:
  • the scheduling module is configured to remotely schedule jobs on the cloud job server according to the acquired job information.
  • the obtaining module includes:
  • a monitoring unit configured to monitor a job node of the ZooKeeper server, where the cloud job server is registered on a job node of the ZooKeeper server;
  • the obtaining unit is configured to acquire job information deployed in the cloud job server according to the job node of the monitored ZooKeeper server.
  • the cloud scheduling server further includes:
  • Expose modules set to expose jobs and web services as remote services.
  • the scheduling module includes:
  • the scheduling unit is configured to initiate timing scheduling, stop timing scheduling, and pause timing scheduling by a unique identifier of the job.
  • the scheduling module is further configured to perform scheduling by using a load balancing algorithm if the same job is detected to be simultaneously provided by multiple servers in the cloud job server.
  • the embodiment of the present invention further provides a computer readable storage medium, which stores program instructions, and when the program instructions are executed, the foregoing method provided by the embodiments of the present invention may be implemented.
  • the embodiment of the invention can implement large-scale distributed cloud batch job scheduling; the number of servers can be dynamically increased or decreased, and the maintenance and deployment are flexible.
  • FIG. 1 is a schematic flowchart of a first embodiment of a job scheduling method according to the present invention
  • FIG. 2 is a schematic flowchart of the steps of acquiring the job information deployed in the cloud job server in FIG. 1;
  • FIG. 3 is a schematic flowchart of a second embodiment of a job scheduling method according to the present invention.
  • FIG. 4 is a schematic diagram showing the refinement of the steps of remotely invoking a job on the cloud job server according to the acquired job information
  • FIG. 5 is a schematic flowchart diagram of a third embodiment of a job scheduling method according to the present invention.
  • FIG. 6 is a schematic diagram of functional modules of a first embodiment of a cloud scheduling server according to the present invention.
  • FIG. 7 is a schematic diagram of functional modules of the acquisition module of FIG. 6;
  • FIG. 8 is a schematic diagram of functional modules of a second embodiment of a cloud scheduling server according to the present invention.
  • FIG. 9 is a schematic diagram of functional modules of the scheduling module in FIG. 6.
  • FIG. 1 is a schematic flowchart of a first embodiment of a job scheduling method according to the present invention.
  • the job scheduling method includes the following steps:
  • Step S100 Obtain job information deployed in the cloud job server.
  • ZooKeeper is a distributed, open source distributed application coordination service that includes a simple set of primitives, an important component of Hadoop and Hbase, and provides an interface between the Java language and the C language.
  • Hadoop is a distributed system infrastructure developed by the Apache Foundation.
  • HBase Hadoop Database
  • HBase Hadoop Database
  • Storage cluster Among them, Zookeeper adopts a tree directory structure, and the process of using Zookeeper is the process of creating, deleting and monitoring tree nodes.
  • Server A creates a persistent node through the client application API (Application Programming Interface) provided by Zookeeper, and registers the listener at the node. Server A will be notified when any other server creates a child node under this node and deletes the child node. If the node type created is a temporary node, after the server is disconnected from Zookeeper (for example, server redundancy), the node will be automatically deleted and Server A will be notified. If you create a temporary node with the name of the job as the node name, the listener can perceive all jobs in the entire cloud; develop the scheduled job and assign it a unique name, which is the job in the cloud job server. Unique identifier.
  • client application API Application Programming Interface
  • the Zookeeper server may be a Zookeeper server cluster
  • the cloud scheduling server may be a cloud scheduling server cluster
  • the cloud job server may be a cloud job server cluster.
  • the Zookeeper server query gets all the job scheduling nodes and adds listeners on those nodes. Therefore, the cloud scheduling server can obtain information about all jobs through the Zookeeper server, and for each job, know which servers provide these jobs.
  • Step S200 Remotely schedule a job on the cloud job server according to the acquired job information.
  • the cloud scheduling server remotely schedules jobs on the cloud job server according to the acquired job information, and the scheduling job refers to a process without a human task developed by the workflow engine, and each process has A process-defined key can be used as a name; a scheduling job can also be an implementation class that implements a user-defined Java interface, using the class name as the name; where the job is not limited to Java implementation, but can also be a database Stored procedures, shell scripts under Linux, and can be called via the Java interface.
  • the job name is the node name of the Zookeeper server, and a persistent node is created on Zookeeper.
  • a cloud job server in the cloud starts, connect to the Zookeeper server; create a persistent node on the Zookeeper server with all the jobs on the server, and use the address (IP port number) of the Zookeeper server as the address of the Zookeeper server.
  • Node name the child node is created under each persistent node; when multiple servers try to create a persistent job node with the same name, Zookeeper will guarantee the order, if the first server created will be created successfully, All server creation server address subnodes will then succeed.
  • the job scheduling method provided in this embodiment can implement large-scale distributed cloud batch job scheduling; the number of servers can be dynamically increased or decreased, thereby maintaining maintenance and deployment flexibility.
  • FIG. 2 is a schematic flowchart of the refinement of step S100 in FIG.
  • Step S110 Listen to the job node of the ZooKeeper server, and the cloud job server is registered on the job node of the ZooKeeper server.
  • the cloud scheduling server When the cloud scheduling server starts, it will connect to the Zookeeper server, listen to the ZooKeeper server's job node, query all job scheduling nodes, and add listeners on these nodes. Among them, the cloud job server itself is registered on the job node on Zookeeper.
  • Step S120 Acquire job information deployed in the cloud job server according to the job node of the monitored ZooKeeper server.
  • the cloud scheduling server can obtain information about all jobs deployed in the cloud job server according to the job node of the monitored ZooKeeper server, and for each job, know which servers provide the jobs; according to the characteristics of the Zookeeper server temporary nodes, When a job execution server in the cloud stops, the temporary node created on the Zookeeper server will be Delete, at this time the cloud scheduling server will receive a notification; the cloud scheduling server needs to cache the information of the job and its execution server, and add or delete the cache information according to the notification received by the listener.
  • the job scheduling method provided by the embodiment obtains the job information deployed in the cloud job server through the job node of the ZooKeeper server, and separates the cloud scheduling server and the cloud job server, and has the advantages of strong operability and convenient maintenance.
  • FIG. 3 is a schematic flowchart of a second embodiment of a job scheduling method according to the present invention. On the basis of the first embodiment, before step S200, the method further includes:
  • Step S200A exposing the job and the network (Web) service as a remote service.
  • jobs When the cloud scheduling server starts, jobs must be exposed as remote services at the same time so that they can be invoked remotely, exposing interfaces or web services to provide remote invocation capabilities.
  • the exposure means that it is exposed, and the exposed interface means that the interface is exposed to the outside for use.
  • the method of exposing to a remote service through a Web service means that it is published as a SOAP (Simple Object Access Protocol) or a RESTFul (Representational State Transfer Software Architecture Style) via an HTTP channel. These two forms of web services. It is exposed as an Http service through the HttpInvoker (a Spring Remote Call Protocol) in the Spring Framework application architecture.
  • SOAP Simple Object Access Protocol
  • RESTFul Real State Transfer Software Architecture Style
  • HttpInvoker is the Http caller.
  • the service can be conveniently published as a remote server, referenced and called on the client.
  • the exposure to the remote service is not limited to HTTP calls, but also through Socket and RPC (Remote Procedure Call Protocol), CORBA (Common Object Request Broker Architecture). Wait for it.
  • FIG. 4 is a schematic flowchart of the refinement of step S200 in FIG.
  • step S210 the Quartz Scheduler is used to perform scheduled scheduling.
  • the cloud scheduling server uses the Quartz Scheduler open source timing scheduler to develop the scheduling server.
  • the scheduling server can obtain all the job information through the Zookeeper query.
  • the local scheduling stub can be obtained by referring to the remote service.
  • Local scheduling stub refers to the local interface of the Web Service, Http Invoker (HTTP invoker) local interface, RPC (Remote Procedure Call Protocol) or CORBA (Common ORB Architecture) local implementation of the local implementation Calling the interface, etc.; with the local call stub, the remote call will be made when the specified scheduling time arrives.
  • Step S220 Start timing scheduling, stop timing scheduling, and pause timing scheduling by using a unique identifier of the job.
  • the cloud scheduling server provides the user with the functions of starting scheduled scheduling, stopping scheduled scheduling, and suspending scheduled scheduling. These functions are scheduled by the key of the job.
  • FIG. 5 is a schematic flowchart of a third embodiment of a job scheduling method according to the present invention. Based on the first embodiment, the step S200 includes:
  • Step S200B If it is detected that the same job is simultaneously provided by multiple servers in the cloud job server, the load balancing algorithm is used for scheduling.
  • a load balancing algorithm can be used to make a reasonable call.
  • a random number load balancing algorithm can also be used. This algorithm can randomly distribute the load to different servers. It can also use a polling load balancing algorithm, which in turn distributes the load to different servers.
  • a custom weighted load balancing algorithm that requires the cloud job server to write its own weight information to the Zookeeper designated node when it starts, and the cloud scheduling server reads the content of the node and calculates the cloud when actually called. Different weights of each cloud job server are distributed to different cloud job servers.
  • the cloud scheduling server needs to connect to the same database server.
  • the Quartz Scheduler itself supports the cluster. After connecting to the same database server, each cloud scheduling server has the clustering feature. Among them, you can increase or decrease the number of cloud scheduling servers at will, because the jobs are in the cloud. Executed in the server does not affect the execution of the job. At the same time, the Quartz Scheduler itself supports the cluster feature. As long as there is one server running in the cloud scheduling server, the entire system will be guaranteed to operate normally.
  • the cloud job server itself is registered on ZooKeeper, and the load balancing algorithm determines which server to execute on.
  • the addition of one server can improve the processing capacity of the entire system.
  • the reduction of one server only affects the operation. Jobs currently running on this server will not affect jobs that have not yet been triggered in the future.
  • the job scheduling method if the same job is detected by multiple servers in the cloud job server, uses a load balancing algorithm for scheduling, thereby reducing the traffic volume and burden of the cloud job server, and scheduling the cloud.
  • the server is separated from the cloud job server to dynamically increase or decrease the number of servers in the cluster.
  • FIG. 6 is a schematic diagram of a function module of a first embodiment of a cloud scheduling server according to the present invention.
  • the cloud scheduling server includes:
  • the obtaining module 10 is configured to obtain job information deployed in the cloud job server;
  • the scheduling module 20 is configured to remotely schedule jobs on the cloud job server according to the acquired job information.
  • ZooKeeper is a distributed, open source distributed application coordination service that includes a simple set of primitives, an important component of Hadoop and Hbase, and provides an interface between the Java language and the C language.
  • Hadoop is a distributed system infrastructure developed by the Apache Foundation.
  • HBase Hadoop Database
  • HBase Hadoop Database
  • Storage cluster Among them, Zookeeper adopts a tree directory structure, and the process of using Zookeeper is the process of creating, deleting and monitoring tree nodes.
  • Server A creates a persistent node through the client application API (Application Programming Interface) provided by Zookeeper, and registers the listener at the node. Server A will be notified when any other server creates a child node under this node and deletes the child node. If the created node type is a temporary node, the server is disconnected from Zookeeper. After the connection (such as server redundancy), the node will be automatically deleted and Server A will be notified. If you create a temporary node with the name of the job as the node name, the listener can perceive all jobs in the entire cloud; develop the scheduled job and assign it a unique name, which is the job in the cloud job server. Unique identifier.
  • client application API Application Programming Interface
  • the Zookeeper server may be a Zookeeper server cluster
  • the cloud scheduling server may be a cloud scheduling server cluster
  • the cloud job server may be a cloud job server cluster.
  • the Zookeeper server query gets all the job scheduling nodes and adds listeners on those nodes. Therefore, the acquisition module 10 of the cloud scheduling server can obtain information of all the jobs through the Zookeeper server, and for each job, know which servers provide the jobs.
  • the scheduling module 20 of the cloud scheduling server remotely schedules jobs on the cloud job server according to the acquired job information.
  • the scheduling job refers to a process without a human task developed by the workflow engine, and each process has a process-defined key. Used as a name; the scheduling job can also be an implementation class that implements a user-defined Java interface, using the class name as the name; wherein the job is not limited to Java implementation, but can also be a database stored procedure, under Linux. Shell scripts, and can be called through the Java interface.
  • the job name is the node name of the Zookeeper server, and a persistent node is created on Zookeeper.
  • a cloud job server in the cloud starts, connect to the Zookeeper server; create a persistent node on the Zookeeper server with all the jobs on the server, and use the address of the Zookeeper server (IP port number).
  • IP port number IP port number
  • child nodes are created under each persistent node; when multiple servers try to create a persistent job node with the same name, Zookeeper will guarantee its order, if the first server created will be created successfully. After that, all server creation server address subnodes will succeed.
  • the cloud scheduling server provided in this embodiment can implement large-scale distributed cloud batch job scheduling; the number of servers can be dynamically increased or decreased, thereby maintaining maintenance and deployment flexibility.
  • FIG. 7 is a schematic diagram of functional modules of the acquiring module in FIG. 6 , where the acquiring module 10 includes:
  • the monitoring unit 11 is configured to monitor a job node of the ZooKeeper server, and the cloud job server is registered on the job node of the ZooKeeper server;
  • the obtaining unit 12 is configured to acquire job information deployed in the cloud job server according to the job node of the monitored ZooKeeper server.
  • the listening unit 11 listens to the working node of the ZooKeeper server, queries all the job scheduling nodes, and adds listeners on these nodes. Among them, the cloud job server itself is registered on the job node on Zookeeper.
  • the acquiring unit 12 of the cloud scheduling server can acquire information of all jobs deployed in the cloud job server according to the job node of the monitored ZooKeeper server, and for each job, know which servers provide the jobs; according to the Zookeeper server temporary The characteristics of the node.
  • the temporary node created on the Zookeeper server will be deleted, and the cloud scheduling server will receive the notification; the cloud scheduling server needs to cache the job and its execution server. Information, and add or remove cached information based on notifications received by the listener.
  • the cloud scheduling server provided by the embodiment obtains the job information deployed in the cloud job server through the job node of the ZooKeeper server, and the cloud job server is well separated, and the operability is strong and the maintenance is convenient.
  • FIG. 8 is a schematic diagram of a functional module of a second embodiment of a cloud scheduling server according to the present invention.
  • the cloud scheduling server further includes:
  • the expose module 30 is configured to expose the job and web service as a remote service.
  • the expose module 30 When the cloud scheduling server starts, the expose module 30 must expose the job as a remote service at the same time so that it can be called remotely; the interface or web service will be exposed to provide remote invocation capabilities.
  • the method of exposing to a remote service through a Web service means that the Web service is released as a SOAP or a RESTFul through an HTTP channel. It is exposed as an Http service through the HttpInvoker (a Spring Remote Call Protocol) in the Spring Framework application architecture. Spring is a dependency injection framework in the Java world, which provides a lot of out-of-the-box components, HttpInvoker is the Http caller.
  • the service can be conveniently published as a remote server, referenced and called on the client.
  • the exposure to the remote service is not limited to HTTP calls, and It can be done by Socket, RPC, CORBA, etc.
  • FIG. 9 is a schematic diagram of functional modules of the scheduling module in FIG. 6.
  • the acquiring scheduling module 20 includes:
  • the setting unit 21 is configured to perform timing scheduling setting by using the Quartz Scheduler
  • the scheduling unit 22 is configured to initiate timing scheduling, stop timing scheduling, and pause timing scheduling by a unique identification key of the job.
  • the setting unit 21 of the cloud scheduling server uses the Quartz Scheduler open source timing scheduler to develop the scheduling server.
  • the scheduling server starts, the information of all the jobs can be obtained through the Zookeeper query; once the information of the job is obtained, the remote service can be referred to.
  • the local scheduling stub refers to the local interface of the Web Service, the local interface of the HttpInvoker local interface, RPC or CORBA local implementation; after the local call stub, when the specified scheduling time arrives, the remote call can be made.
  • the scheduling unit 22 of the cloud scheduling server provides the user with the functions of starting timing scheduling, stopping timing scheduling, and suspending timing scheduling, and these functions are scheduled by the key of the job.
  • the scheduling module provided by the first embodiment is further configured to perform scheduling by using a load balancing algorithm if the same job is detected by multiple servers in the cloud job server.
  • a load balancing algorithm can be used to make a reasonable call.
  • a random number load balancing algorithm can also be used. This algorithm can randomly distribute the load to different servers. It can also use a polling load balancing algorithm, which in turn distributes the load to different servers.
  • a custom weighted load balancing algorithm that requires the cloud job server to write its own weight information to the Zookeeper designated node when it starts, and the cloud scheduling server reads the content of the node and calculates the cloud when actually called. Different weights of each cloud job server are distributed to different cloud job servers.
  • the cloud scheduling server needs to connect to the same database server.
  • the Quartz Scheduler itself supports the cluster. After connecting to the same database server, each cloud scheduling server has a cluster. characteristic. Among them, you can increase or decrease the number of cloud scheduling servers at will, because the jobs are executed in the cloud job server and will not affect the execution of the jobs.
  • the Quartz Scheduler itself supports the cluster feature. As long as there is one server running in the cloud scheduling server, the entire system will be guaranteed to operate normally.
  • the cloud job server itself is registered on ZooKeeper, and the load balancing algorithm determines which server to execute on.
  • the addition of one server can improve the processing capacity of the entire system.
  • the reduction of one server only affects the operation. Jobs currently running on this server will not affect jobs that have not yet been triggered in the future.
  • the cloud scheduling server when detecting that the same job is simultaneously provided by multiple servers in the cloud job server, uses a load balancing algorithm for scheduling, thereby reducing the traffic volume and burden of the cloud job server, and the cloud job server Separation allows you to dynamically increase or decrease the number of servers in a cluster.
  • all or part of the steps of the above embodiments may also be implemented by using an integrated circuit. These steps may be separately fabricated into individual integrated circuit modules, or multiple modules or steps may be fabricated into a single integrated circuit module. achieve.
  • the devices/function modules/functional units in the above embodiments may be implemented by a general-purpose computing device, which may be centralized on a single computing device or distributed over a network of multiple computing devices.
  • each device/function module/functional unit in the above embodiment When each device/function module/functional unit in the above embodiment is implemented in the form of a software function module and sold or used as a stand-alone product, it can be stored in a computer readable storage medium.
  • the above mentioned computer readable storage medium may be a read only memory, a magnetic disk or an optical disk or the like.
  • the embodiments of the present invention can implement large-scale distributed cloud batch job scheduling, which can dynamically increase or decrease the number of servers, and maintain and deploy flexible.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer And Data Communications (AREA)
  • Debugging And Monitoring (AREA)

Abstract

一种作业调度方法，通过获取云作业服务器中部署的作业信息；根据获取的作业信息，远程调度云作业服务器上的作业，可实现大规模分布式云批量作业调度；可动态增加或减少服务器数量，维护和部署灵活。

Description

作业调度方法和云调度服务器 技术领域
本文涉及网络通讯领域，尤其涉及作业调度方法和云调度服务器。
背景技术
在相关技术的作业调度技术方案中，通常使用调度服务器集群(例如Java领域的Quartz Scheduler)进行批量调度，集群中的每一台服务器的内容是完全一致的，且被调度的作业也是运行在该服务器上。相关技术存在的缺点和问题如下所示：
一、调度服务器集群维护困难，要更新被调度的作业时，往往需要对集群中的所有服务器均进行停机并重启；
二、无法设置权重，无法将更高的负载分配给性能更好的服务器；
三、系统不区分调度服务端和调度客户端，无法任意增加或减少服务器数量。
四、每台调度服务器部署的作业需要完全一致，若有变动所有服务器都得变动。
发明内容
本发明实施例提供一种作业调度方法和云调度服务器，解决调度服务器集群维护和部署困难的问题。
本发明实施例提供的所述作业调度方法包括以下步骤：
获取云作业服务器中部署的作业信息；
根据获取的作业信息，远程调度云作业服务器上的作业。
可选地，所述获取云作业服务器中部署的作业信息的步骤包括：
监听ZooKeeper服务器的作业节点，所述云作业服务器注册在ZooKeeper服务器的作业节点上；
根据监听的ZooKeeper服务器的作业节点，获取云作业服务器中部署的作业信息。
可选地，所述根据获取的作业信息，远程调用云作业服务器上的作业的步骤之前还包括：
将作业和Web服务暴露为远程服务。
可选地，所述根根据获取的作业信息，远程调用云作业服务器上的作业的步骤包括：
采用开源定时调度器进行定时调度设置；
通过作业的唯一标识启动定时调度、停止定时调度和暂停定时调度。
可选地，所述根据获取的作业信息，远程调用云作业服务器上的作业的步骤包括：
如果检测同一作业由云作业服务器中的多台服务器同时提供时，则采用负载均衡算法进行调度。
本发明实施例还提供一种云调度服务器，所述云调度服务器包括：
获取模块，设置为获取云作业服务器中部署的作业信息；
调度模块，设置为根据获取的作业信息，远程调度云作业服务器上的作业。
可选地，所述获取模块包括：
监听单元，设置为监听ZooKeeper服务器的作业节点，所述云作业服务器注册在ZooKeeper服务器的作业节点上；
获取单元，设置为根据监听的ZooKeeper服务器的作业节点，获取云作业服务器中部署的作业信息。
可选地，所述云调度服务器还包括：
暴露模块，设置为将作业和Web服务暴露为远程服务。
可选地，所述调度模块包括：
设置单元，设置为采用开源定时调度器Quartz Scheduler进行定时调度设置；
调度单元，设置为通过作业的唯一标识启动定时调度、停止定时调度和暂停定时调度。
可选地，所述调度模块，还设置为如果检测同一作业由云作业服务器中的多台服务器同时提供时，则采用负载均衡算法进行调度。
本发明实施例还提供一种计算机可读存储介质，存储有程序指令，当该程序指令被执行时可实现本发明实施例提供的上述方法。
本发明实施例可实现大规模分布式云批量作业调度；可动态增加或减少服务器数量，维护和部署灵活。
附图概述
图1为本发明作业调度方法第一实施例的流程示意图；
图2为图1中所述获取云作业服务器中部署的作业信息的步骤的细化流程示意图；
图3为本发明作业调度方法第二实施例的流程示意图；
图4为图1中所述根据获取的作业信息，远程调用云作业服务器上的作业的步骤的细化流程示意图；
图5为本发明作业调度方法第三实施例的流程示意图；
图6为本发明云调度服务器第一实施例的功能模块示意图；
图7为图6中所述获取模块的功能模块示意图；
图8为本发明云调度服务器第二实施例的功能模块示意图；
图9为图6中所述调度模块的功能模块示意图。
本发明目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
本发明的实施方式
应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。
本发明提供一种作业调度方法，参照图1，图1为本发明作业调度方法第一实施例的流程示意图，在第一实施例中，所述作业调度方法包括以下步骤：
步骤S100、获取云作业服务器中部署的作业信息。
云调度服务器启动时，会连接动物园管理员(Zookeeper)服务器。ZooKeeper是一个分布式的，开放源码的分布式应用程序协调服务，包含一个简单的原语集，是Hadoop和Hbase的重要组件，提供Java语言和C语言的接口。Hadoop是一个由Apache基金会所开发的分布式系统基础架构，HBase(Hadoop Database)，是一个高可靠性、高性能、面向列、可伸缩的分布式存储系统，利用HBase技术可搭建起大规模结构化存储集群。其中，Zookeeper采用树形目录结构，使用Zookeeper的过程就是树节点的创建、删除和监听的过程。例如服务器A通过Zookeeper提供的客户端API(Application Programming Interface，应用程序编程接口)创建了一个持久化节点，并在该节点注册监听器。任何其他服务器在该节点下创建子节点以及删除子节点，服务器A均会收到通知。如果创建的节点类型为临时节点，则服务器与Zookeeper断开连接后(例如服务器冗机)，节点将自动被删除，服务器A会收到通知。如果将作业的名字作为节点名创建临时节点，利用该特性，监听器就可以感知整个云中的所有作业；开发调度作业并对其赋以唯一的名字，该名字就是该作业在云作业服务器中的唯一标识。所述Zookeeper服务器可以为Zookeeper服务器集群，所述云调度服务器可以为云调度服务器集群，所述云作业服务器可以为云作业服务器集群。Zookeeper服务器查询得到所有的作业调度节点，并在这些节点上添加监听器。因此，云调度服务器就可以通过Zookeeper服务器获取所有作业的信息，并且针对每一个作业，知道由哪些服务器提供了这些作业。
步骤S200、根据获取的作业信息，远程调度云作业服务器上的作业。
云调度服务器根据获取的作业信息，远程调度云作业服务器上的作业，调度作业是指，采用工作流引擎开发的不含人工任务的流程，每个流程具备 一个流程定义的key可以作为名字使用；调度作业还可以是，实现了用户自定义Java接口的一个实现类，采用该类名作为名字；其中，作业不局限于采用Java实现，也可以是数据库的存储过程，Linux下的Shell脚本，并可以通过Java接口的方式进行调用。作业名作为Zookeeper服务器的节点名称，在Zookeeper上创建持久化节点。如果云中的一台云作业服务器启动时，连接Zookeeper服务器；将该服务器上的所有作业，都以作业的名字在Zookeeper服务器上创建持久化节点，并以Zookeeper服务器的地址(IP端口号)作为节点名，将子节点创建到每一个持久化节点下；当有多台服务器尝试创建相同名字的持久化作业节点时，Zookeeper会保证其先后顺序，如果第一台进行创建的服务器会创建成功，之后所有的服务器创建服务器地址子节点均会成功。
本实施例提供的作业调度方法，可实现大规模分布式云批量作业调度；可动态增加或减少服务器数量，从而维护和部署灵活。
如图2所示，图2为图1中所述步骤S100的细化流程示意图，所述步骤S100
包括：
步骤S110、监听ZooKeeper服务器的作业节点，所述云作业服务器注册在ZooKeeper服务器的作业节点上。
云调度服务器启动时，会连接Zookeeper服务器，监听ZooKeeper服务器的作业节点，并查询得到所有的作业调度节点，并在这些节点上添加监听器。其中，云作业服务器自身是注册在Zookeeper上的作业节点上的。
步骤S120、根据监听的ZooKeeper服务器的作业节点，获取云作业服务器中部署的作业信息。
云调度服务器根据监听的ZooKeeper服务器的作业节点，就可以获取云作业服务器中部署的所有作业的信息，并且针对每一个作业，都知道有哪些服务器提供了这些作业；根据Zookeeper服务器临时节点的特性，当云中的某台作业执行服务器停止时，其在Zookeeper服务器上创建的临时节点将被 删除，此时云调度服务器将收到通知；云调度服务器需要缓存作业及其执行服务器的信息，并根据监听器收到的通知，添加或删除缓存信息。
本实施例提供的作业调度方法，通过ZooKeeper服务器的作业节点获取云作业服务器中部署的作业信息，并将云调度服务器和云作业服务器很好地分离，可操作性强、维护方便。
如图3所示，图3为本发明作业调度方法第二实施例的流程示意图，在第一实施例的基础上，所述步骤S200之前还包括：
步骤S200A、将作业和网络(Web)服务暴露为远程服务。
云调度服务器启动时，作业必须同时暴露为远程服务，以便可以被远程调用，暴露接口或者Web服务以便提供远程调用能力。所述暴露是指露在外面，所述暴露接口是指将接口露在外面，以供使用的意思。其中，通过Web服务的方式暴露为远程服务，是指通过HTTP通道发布为SOAP(Simple Object Access Protocol，简单对象访问协议)或RESTFul(Representational State Transfer Software Architecture Style，基于REST式的软件体系结构风格)这二种形式的Web服务。通过Spring Framework应用架构中的HttpInvoker(一个Spring远程调用协议)方式将其暴露为Http服务，Spring是Java世界里一个依赖注入框架，其提供了很多开箱即用的组建，HttpInvoker即Http调用器，通过该接口，可以方便的将服务发布为远程服务器，在客户端引用并调用。在本实施例中，暴露为远程服务并不局限于HTTP调用，还可以通过Socket以及RPC(Remote Procedure Call Protocol，远程过程调用协议)、CORBA(Common Object Request Broker Architecture，通用对象请求代理体系结构)等方式进行。
如图4所示，图4为图1中所述步骤S200的细化流程示意图，所述步骤S200
包括：
步骤S210、采用Quartz Scheduler进行定时调度设置。
云调度服务器采用Quartz Scheduler开源定时调度器,进行调度服务端的开发，调度服务器启动时，可以通过Zookeeper查询得到所有作业的信息；一旦获取作业的信息，就可以通过引用远程服务的方式获取本地调度存根，从而进行远程调度。本地调度存根是指Web Service本地接口，Http Invoker(HTTP调用器)本地接口、RPC(Remote Procedure Call Protocol，远程过程调用协议)或CORBA(Common ORB Architecture，通用对象请求代理体系结构)本地实现的本地调用接口等；有了本地调用存根后，在指定的调度时间到达时，将可以进行远程调用。
步骤S220、通过作业的唯一标识(key)启动定时调度、停止定时调度和暂停定时调度。
云调度服务器对用户提供启动定时调度、停止定时调度、暂停定时调度的功能，这些功能通过作业的key进行调度。
如图5所示，图5为本发明作业调度方法第三实施例的流程示意图，在第一实施例的基础上，所述步骤S200包括：
步骤S200B、如果检测同一作业由云作业服务器中的多台服务器同时提供时，则采用负载均衡算法进行调度。
云调度服务器进行远程调度时，如果同一作业同时由云中的很多作业执行服务器提供，此时就可以采用负载均衡算法进行合理调用。也可以采用随机数的负载均衡算法，此算法即可以将负载随机分配到不同服务器上；还可以采用轮询的负载均衡算法，此算法依次将负载分配到不同服务器上。自定义权重的负载均衡算法，此算法要求云作业服务器在启动时，将自身的权重信息写入到Zookeeper指定节点上，并由云调度服务器读取该节点的内容，并在实际调用时计算云中各个云作业服务器的不同权重，将负载分配到各个不同云作业服务器上。
云调度服务器需要连接同一台数据库服务器，Quartz Scheduler自身支持集群，连接到同一台数据库服务器后，其每一台云调度服务器就具备了集群特性。其中，可以随意增加或减少云调度服务器的数量，因为作业在云作业 服务器中执行，不会影响到作业的执行。同时Quartz Scheduler自身支持集群特性，只要云调度服务器中还有一台服务器在运行，将保证整个系统的正常运行。
云作业服务器自身是注册在Zookeeper上的，并且由负载均衡算法决定了在哪一台服务器上执行，增加一台服务器，可以提高整个系统的作业处理能力；而减少一台服务器，仅仅会影响到当前在该服务器上运行的作业，不会影响到未来还没有触发的作业。
本实施例提供的作业调度方法，如果检测同一作业由云作业服务器中的多台服务器同时提供时，则采用负载均衡算法进行调度，从而减轻了云作业服务器的业务量和负担，并且对云调度服务器和云作业服务器进行分离，可以在集群中动态增加或减少服务器数量。
如图6所示，图6为本发明云调度服务器第一实施例的功能模块示意图，在第一实施例中，所述云调度服务器包括：
获取模块10，设置为获取云作业服务器中部署的作业信息；以及
调度模块20，设置为根据获取的作业信息，远程调度云作业服务器上的作业。
云调度服务器启动时，会连接Zookeeper服务器。ZooKeeper是一个分布式的，开放源码的分布式应用程序协调服务，包含一个简单的原语集，是Hadoop和Hbase的重要组件，提供Java语言和C语言的接口。Hadoop是一个由Apache基金会所开发的分布式系统基础架构，HBase(Hadoop Database)，是一个高可靠性、高性能、面向列、可伸缩的分布式存储系统，利用HBase技术可搭建起大规模结构化存储集群。其中，Zookeeper采用树形目录结构，使用Zookeeper的过程就是树节点的创建、删除和监听的过程。例如服务器A通过Zookeeper提供的客户端API(Application Programming Interface，应用程序编程接口)创建了一个持久化节点，并在该节点注册监听器。任何其他服务器在该节点下创建子节点以及删除子节点，服务器A均会收到通知。如果创建的节点类型为临时节点，则服务器与Zookeeper断开连 接后(例如服务器冗机)，节点将自动被删除，服务器A会收到通知。如果将作业的名字作为节点名创建临时节点，利用该特性，监听器就可以感知整个云中的所有作业；开发调度作业并对其赋以唯一的名字，该名字就是该作业在云作业服务器中的唯一标识。所述Zookeeper服务器可以为Zookeeper服务器集群，所述云调度服务器可以为云调度服务器集群，所述云作业服务器可以为云作业服务器集群。Zookeeper服务器查询得到所有的作业调度节点，并在这些节点上添加监听器。因此，云调度服务器的获取模块10就可以通过Zookeeper服务器获取所有作业的信息，并且针对每一个作业，知道由哪些服务器提供了这些作业。
云调度服务器的调度模块20根据获取的作业信息，远程调度云作业服务器上的作业，调度作业是指，采用工作流引擎开发的不含人工任务的流程，每个流程具备一个流程定义的key可以作为名字使用；调度作业还可以是，实现了用户自定义Java接口的一个实现类，采用该类名作为名字；其中，作业不局限于采用Java实现，也可以是数据库的存储过程，Linux下的Shell脚本，并可以通过Java接口的方式进行调用。作业名作为Zookeeper服务器的节点名称，在Zookeeper上创建持久化节点。如果云中的某一台云作业服务器启动时，连接Zookeeper服务器；将该服务器上的所有作业，都以作业的名字在Zookeeper服务器上创建持久化节点，并以Zookeeper服务器的地址(IP端口号)作为节点名，将子节点创建到每一个持久化节点下；当有多台服务器尝试创建相同名字的持久化作业节点时，Zookeeper会保证其先后顺序，如果第一台进行创建的服务器会创建成功，之后所有的服务器创建服务器地址子节点均会成功。
本实施例提供的云调度服务器，可实现大规模分布式云批量作业调度；可动态增加或减少服务器数量，从而维护和部署灵活。
如图7所示，图7为图6中所述获取模块的功能模块示意图，所述获取模块10包括：
监听单元11，设置为监听ZooKeeper服务器的作业节点，所述云作业服务器注册在ZooKeeper服务器的作业节点上；
获取单元12，设置为根据监听的ZooKeeper服务器的作业节点，获取云作业服务器中部署的作业信息。
云调度服务器启动时，会连接Zookeeper服务器，监听单元11监听ZooKeeper服务器的作业节点，并查询得到所有的作业调度节点，并在这些节点上添加监听器。其中，云作业服务器自身是注册在Zookeeper上的作业节点上的。
云调度服务器的获取单元12根据监听的ZooKeeper服务器的作业节点，就可以获取云作业服务器中部署的所有作业的信息，并且针对每一个作业，都知道有哪些服务器提供了这些作业；根据Zookeeper服务器临时节点的特性，当云中的一台作业执行服务器停止时，其在Zookeeper服务器上创建的临时节点将被删除，此时云调度服务器将收到通知；云调度服务器需要缓存作业及其执行服务器的信息，并根据监听器收到的通知，添加或删除缓存信息。
本实施例提供的云调度服务器，通过ZooKeeper服务器的作业节点获取云作业服务器中部署的作业信息，并将云作业服务器很好地分离，可操作性强、维护方便。
如图8所示，图8为本发明云调度服务器第二实施例的功能模块示意图，在第一实施例中，所述云调度服务器还包括：
暴露模块30，设置为将作业和Web服务暴露为远程服务。
云调度服务器启动时，暴露模块30将作业必须同时暴露为远程服务，以便可以被远程调用；将暴露接口或者Web服务以便提供远程调用能力。其中，通过Web服务的方式暴露为远程服务，是指通过HTTP通道发布为SOAP或RESTFul这二种形式的Web服务。通过Spring Framework应用架构中的HttpInvoker(一个Spring远程调用协议)方式将其暴露为Http服务，Spring是Java世界里一个依赖注入框架，其提供了很多开箱即用的组建，HttpInvoker即Http调用器，通过该接口，可以方便的将服务发布为远程服务器，在客户端引用并调用。在本实施例中，暴露为远程服务并不局限于HTTP调用，还 可以通过Socket以及RPC、CORBA等方式进行。
如图9所示，图9为图6中所述调度模块的功能模块示意图，所述获取调度模块20包括：
设置单元21，设置为采用Quartz Scheduler进行定时调度设置；
调度单元22，设置为通过作业的唯一标识key启动定时调度、停止定时调度和暂停定时调度。
云调度服务器的设置单元21采用Quartz Scheduler开源定时调度器,进行调度服务端的开发，调度服务器启动时，可以通过Zookeeper查询得到所有作业的信息；一旦获取作业的信息，就可以通过引用远程服务的方式获取本地调度存根，从而进行远程调度。本地调度存根是指Web Service本地接口，HttpInvoker本地接口、RPC或CORBA本地实现的本地调用接口等；有了本地调用存根后，在指定的调度时间到达时，将可以进行远程调用。
云调度服务器的调度单元22对用户提供启动定时调度、停止定时调度、暂停定时调度的功能，这些功能通过作业的key进行调度。
进一步参见图6，第一实施例提供的调度模块，所述调度模块20，还用于如果检测同一作业由云作业服务器中的多台服务器同时提供时，则采用负载均衡算法进行调度。
云调度服务器的调度模块20进行远程调度时，如果同一作业同时由云中的很多作业执行服务器提供，此时就可以采用负载均衡算法进行合理调用。也可以采用随机数的负载均衡算法，此算法即可以将负载随机分配到不同服务器上；还可以采用轮询的负载均衡算法，此算法依次将负载分配到不同服务器上。自定义权重的负载均衡算法，此算法要求云作业服务器在启动时，将自身的权重信息写入到Zookeeper指定节点上，并由云调度服务器读取该节点的内容，并在实际调用时计算云中各个云作业服务器的不同权重，将负载分配到各个不同云作业服务器上。
云调度服务器需要连接同一台数据库服务器，Quartz Scheduler自身支持集群，连接到同一台数据库服务器后，其每一台云调度服务器就具备了集群 特性。其中，可以随意增加或减少云调度服务器的数量，因为作业在云作业服务器中执行，不会影响到作业的执行。同时Quartz Scheduler自身支持集群特性，只要云调度服务器中还有一台服务器在运行，将保证整个系统的正常运行。
云作业服务器自身是注册在Zookeeper上的，并且由负载均衡算法决定了在哪一台服务器上执行，增加一台服务器，可以提高整个系统的作业处理能力；而减少一台服务器，仅仅会影响到当前在该服务器上运行的作业，不会影响到未来还没有触发的作业。
本实施例提供的云调度服务器，如果检测同一作业由云作业服务器中的多台服务器同时提供时，则采用负载均衡算法进行调度，从而减轻了云作业服务器的业务量和负担，并且云作业服务器进行分离，可以在集群中动态增加或减少服务器数量。
本领域普通技术人员可以理解上述实施例的全部或部分步骤可以使用计算机程序流程来实现，所述计算机程序可以存储于一计算机可读存储介质中，所述计算机程序在相应的硬件平台上(如系统、设备、装置、器件等)执行，在执行时，包括方法实施例的步骤之一或其组合。
可选地，上述实施例的全部或部分步骤也可以使用集成电路来实现，这些步骤可以被分别制作成一个个集成电路模块，或者将它们中的多个模块或步骤制作成单个集成电路模块来实现。
上述实施例中的各装置/功能模块/功能单元可以采用通用的计算装置来实现，它们可以集中在单个的计算装置上，也可以分布在多个计算装置所组成的网络上。
上述实施例中的各装置/功能模块/功能单元以软件功能模块的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。上述提到的计算机可读取存储介质可以是只读存储器，磁盘或光盘等。
工业实用性
本发明实施例可实现大规模分布式云批量作业调度，可动态增加或减少服务器数量，维护和部署灵活。

Claims (11)

  1. 一种作业调度方法，包括以下步骤：
    获取云作业服务器中部署的作业信息；
    根据获取的作业信息，远程调度所述云作业服务器上的作业。
  2. 如权利要求1所述的作业调度方法，其中，所述获取云作业服务器中部署的作业信息的步骤包括：
    监听ZooKeeper服务器的作业节点，所述云作业服务器注册在ZooKeeper服务器的作业节点上；
    根据监听的ZooKeeper服务器的作业节点，获取云作业服务器中部署的作业信息。
  3. 如权利要求1或2所述的作业调度方法，其中，所述根据获取的作业信息，远程调用云作业服务器上的作业的步骤之前还包括：
    将作业和Web服务暴露为远程服务。
  4. 如权利要求1或2所述的作业调度方法，其中，所述根据获取的作业信息，远程调用云作业服务器上的作业的步骤包括：
    采用开源定时调度器进行定时调度设置；
    通过作业的唯一标识启动定时调度、停止定时调度和暂停定时调度。
  5. 如权利要求1或2所述的作业调度方法，其中，所述根据获取的作业信息，远程调用云作业服务器上的作业的步骤包括：
    如果检测同一作业由所述云作业服务器中的多台服务器同时提供时，则采用负载均衡算法进行调度。
  6. 一种云调度服务器，包括：
    获取模块，设置为获取云作业服务器中部署的作业信息；以及
    调度模块，设置为根据获取的作业信息，远程调度所述云作业服务器上的作业。
  7. 如权利要求6所述云调度服务器，其中，所述获取模块包括：
    监听单元，设置为监听ZooKeeper服务器的作业节点，所述云作业服务器注册在ZooKeeper服务器的作业节点上；以及
    获取单元，设置为根据监听的ZooKeeper服务器的作业节点，获取所述云作业服务器中部署的作业信息。
  8. 如权利要求6或7所述云调度服务器，所述云调度服务器还包括：
    暴露模块，设置为将作业和Web服务暴露为远程服务。
  9. 如权利要求6或7所述云调度服务器，其中，所述调度模块包括：
    设置单元，设置为采用开源定时调度器进行定时调度设置；以及
    调度单元，设置为通过作业的唯一标识启动定时调度、停止定时调度和暂停定时调度。
  10. 如权利要求6或7所述云调度服务器，其中，所述调度模块，还设置为如果检测同一作业由云作业服务器中的多台服务器同时提供时，则采用负载均衡算法进行调度。
  11. 一种计算机可读存储介质，存储有程序指令，当该程序指令被执行时可实现权利要求1-5任一项所述的方法。
PCT/CN2015/074641 2014-12-23 2015-03-19 作业调度方法和云调度服务器 WO2016101423A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201410815344.8 2014-12-23
CN201410815344.8A CN105791354A (zh) 2014-12-23 2014-12-23 作业调度方法和云调度服务器

Publications (1)

Publication Number Publication Date
WO2016101423A1 true WO2016101423A1 (zh) 2016-06-30

Family

ID=56149045

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/074641 WO2016101423A1 (zh) 2014-12-23 2015-03-19 作业调度方法和云调度服务器

Country Status (2)

Country Link
CN (1) CN105791354A (zh)
WO (1) WO2016101423A1 (zh)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106528297A (zh) * 2016-11-14 2017-03-22 山东浪潮云服务信息科技有限公司 一种管理任务的系统
CN106980678A (zh) * 2017-03-30 2017-07-25 温馨港网络信息科技（苏州）有限公司 基于zookeeper技术的数据分析方法及系统
CN107181806A (zh) * 2017-05-27 2017-09-19 北京思特奇信息技术股份有限公司 一种基于dubbo的分布式业务处理系统及方法
CN109862101A (zh) * 2019-02-13 2019-06-07 中国银行股份有限公司 跨平台应用启动方法、装置、计算机设备和存储介质
CN110780869A (zh) * 2019-10-31 2020-02-11 辽宁振兴银行股份有限公司 一种分布式批量调度
CN112835690A (zh) * 2021-04-07 2021-05-25 北京航天云路有限公司 一种可编排流程的定时作业执行的方法
CN113110922A (zh) * 2021-03-08 2021-07-13 北京世纪安图数码科技发展有限责任公司 一种基于Quartz与多线程的不动产登记方法
CN113672367A (zh) * 2021-08-16 2021-11-19 深圳壹账通智能科技有限公司 分布式定时任务调度方法及系统

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107766147A (zh) * 2016-08-23 2018-03-06 上海宝信软件股份有限公司 分布式的数据分析任务调度系统
CN106874090A (zh) * 2017-01-23 2017-06-20 北京思特奇信息技术股份有限公司 基于云系统的作业调度方法和系统
CN111163117B (zh) * 2018-11-07 2023-01-31 北京京东尚科信息技术有限公司 一种基于Zookeeper的对等式调度方法和装置
CN109697112B (zh) * 2018-12-24 2023-05-16 广东亿迅科技有限公司 分布式集约化一站式作业系统和实现方法
CN109525466B (zh) * 2019-01-03 2020-09-29 杭州云毅网络科技有限公司 数据节点监听方法、装置、电子设备及存储介质
CN110633914A (zh) * 2019-09-20 2019-12-31 中国银行股份有限公司 批量作业调度处理方法及装置
CN110764910A (zh) * 2019-10-23 2020-02-07 中国银行股份有限公司 批量作业调度处理方法及装置
CN113037839B (zh) * 2021-03-08 2022-11-29 中国工商银行股份有限公司 一种分布式批量框架通讯系统及方法
CN114363332B (zh) * 2021-12-27 2024-01-23 徐工汉云技术股份有限公司 一种基于分布式的网关远程自动化运维方法

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090248693A1 (en) * 2008-03-27 2009-10-01 Microsoft Corporation Managing data transfer between endpoints in a distributed computing environment
CN103530182A (zh) * 2013-10-22 2014-01-22 海南大学 一种作业调度方法和装置
CN103617305A (zh) * 2013-10-22 2014-03-05 芜湖大学科技园发展有限公司 一种自适应的电力仿真云计算平台作业调度算法

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101140528B (zh) * 2007-08-31 2013-03-20 中兴通讯股份有限公司 一种实现集群内定时任务负载分担的方法及装置
CN101702721B (zh) * 2009-10-26 2011-08-31 北京航空航天大学 一种多集群系统的可重组方法
CN101741904B (zh) * 2009-11-27 2013-01-02 中国林业科学研究院资源信息研究所 一种分布式空间计算服务节点的构建方法及网关装置
CN102033777B (zh) * 2010-09-17 2013-03-20 中国资源卫星应用中心 基于ice的分布式作业调度引擎
US9063939B2 (en) * 2011-11-03 2015-06-23 Zettaset, Inc. Distributed storage medium management for heterogeneous storage media in high availability clusters
CN102801766B (zh) * 2011-11-18 2015-01-07 北京安天电子设备有限公司 一种云服务器负载均衡及数据冗余备份的方法及系统
CN102932210B (zh) * 2012-11-23 2015-06-24 北京搜狐新媒体信息技术有限公司 一种PaaS云平台的节点监控方法和系统
CN103268245B (zh) * 2012-11-28 2016-04-13 北京建筑工程学院 一种气象数据快速处理流程化的方法
CN103744734B (zh) * 2013-12-24 2017-09-26 中国科学院深圳先进技术研究院 一种任务作业处理方法、装置及系统
CN103986748B (zh) * 2014-04-22 2019-01-25 数贸科技（北京）有限公司 实现服务化的方法和装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090248693A1 (en) * 2008-03-27 2009-10-01 Microsoft Corporation Managing data transfer between endpoints in a distributed computing environment
CN103530182A (zh) * 2013-10-22 2014-01-22 海南大学 一种作业调度方法和装置
CN103617305A (zh) * 2013-10-22 2014-03-05 芜湖大学科技园发展有限公司 一种自适应的电力仿真云计算平台作业调度算法

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106528297A (zh) * 2016-11-14 2017-03-22 山东浪潮云服务信息科技有限公司 一种管理任务的系统
CN106980678A (zh) * 2017-03-30 2017-07-25 温馨港网络信息科技（苏州）有限公司 基于zookeeper技术的数据分析方法及系统
CN107181806A (zh) * 2017-05-27 2017-09-19 北京思特奇信息技术股份有限公司 一种基于dubbo的分布式业务处理系统及方法
CN109862101A (zh) * 2019-02-13 2019-06-07 中国银行股份有限公司 跨平台应用启动方法、装置、计算机设备和存储介质
CN110780869A (zh) * 2019-10-31 2020-02-11 辽宁振兴银行股份有限公司 一种分布式批量调度
CN113110922A (zh) * 2021-03-08 2021-07-13 北京世纪安图数码科技发展有限责任公司 一种基于Quartz与多线程的不动产登记方法
CN113110922B (zh) * 2021-03-08 2024-04-12 北京世纪安图数码科技发展有限责任公司 一种基于Quartz与多线程的不动产登记方法
CN112835690A (zh) * 2021-04-07 2021-05-25 北京航天云路有限公司 一种可编排流程的定时作业执行的方法
CN112835690B (zh) * 2021-04-07 2024-04-05 北京航天云路有限公司 一种可编排流程的定时作业执行的方法
CN113672367A (zh) * 2021-08-16 2021-11-19 深圳壹账通智能科技有限公司 分布式定时任务调度方法及系统

Also Published As

Publication number Publication date
CN105791354A (zh) 2016-07-20

Similar Documents

Publication Publication Date Title
WO2016101423A1 (zh) 作业调度方法和云调度服务器
McGrath et al. Serverless computing: Design, implementation, and performance
US10904303B2 (en) Control message from streaming source to facilitate scaling
US9952896B2 (en) Asynchronous task management in an on-demand network code execution environment
US11321139B2 (en) Streaming traffic pattern for public cloud auto scaling
US11716264B2 (en) In situ triggered function as a service within a service mesh
CN109684036B (zh) 一种容器集群管理方法、存储介质、电子设备及系统
JP2020502643A (ja) オンデマンドコード実行能力を有する局所化されたデバイスコーディネータ
CN112804289B (zh) 一种资源同步方法、装置、设备及存储介质
JP2012230667A (ja) 高負荷のビジネスプロセスのスケーラビリティ
CN105592110B (zh) 一种资源调度方法及装置
WO2022257247A1 (zh) 数据处理方法、装置及计算机可读存储介质
CN109756461A (zh) 一种基于netty的远程过程调用方法
CN103546571A (zh) 一种平台即服务实现方法及装置
CN113794652B (zh) 数据处理方法、装置、电子设备及存储介质
CN108073414B (zh) 一种基于Jedis的将多线程并发请求合并批量提交并分发结果的实现方法
WO2019000790A1 (zh) 一种以同步方式进行远程过程调用的方法及装置
CN113535362A (zh) 一种分布式调度系统架构和微服务工作流调度方法
WO2024030328A1 (en) Interservice communication optimization for microservices
CN110365743B (zh) 一种基于Zookeeper实现的支持多种可自定义负载算法的负载均衡器实现方法
O'Sullivan et al. The cloud personal assistant for providing services to mobile clients
Cheng et al. A service-based fog execution environment for the IoT-aware business process applications
US11614957B1 (en) Native-hypervisor based on-demand code execution system
CN108062245A (zh) 一种应用定时任务统一管理系统及方法
CN112839065A (zh) 信息处理方法、装置、第一设备及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15871501

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15871501

Country of ref document: EP

Kind code of ref document: A1