CN115357403A - Micro-service system for task scheduling and task scheduling method - Google Patents
Micro-service system for task scheduling and task scheduling method Download PDFInfo
- Publication number
- CN115357403A CN115357403A CN202211283489.9A CN202211283489A CN115357403A CN 115357403 A CN115357403 A CN 115357403A CN 202211283489 A CN202211283489 A CN 202211283489A CN 115357403 A CN115357403 A CN 115357403A
- Authority
- CN
- China
- Prior art keywords
- task
- scheduling
- service
- module
- actuator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000000977 initiatory effect Effects 0.000 claims abstract description 8
- 230000008569 process Effects 0.000 claims description 21
- 238000012544 monitoring process Methods 0.000 claims description 19
- 230000007246 mechanism Effects 0.000 claims description 4
- 238000007726 management method Methods 0.000 description 22
- 230000006870 function Effects 0.000 description 12
- 238000012423 maintenance Methods 0.000 description 10
- 238000012545 processing Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 230000002159 abnormal effect Effects 0.000 description 4
- 238000002955 isolation Methods 0.000 description 4
- 230000000903 blocking effect Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 239000003292 glue Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 239000010453 quartz Substances 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- VYPSYNLAJGMNEJ-UHFFFAOYSA-N silicon dioxide Inorganic materials O=[Si]=O VYPSYNLAJGMNEJ-UHFFFAOYSA-N 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5022—Mechanisms to release resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
- G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
Abstract
A micro service system and a task scheduling method for task scheduling are provided, the system comprises: the authority authentication module identifies user login information according to the login request and grants service authority according to the user login information; the task scheduling service module is configured to: creating an actuator according to the user login information, configuring one or more actuator parameters matched with the user login information, and returning to an actuator list; selecting a scheduling task in the task list or automatically initiating a timing scheduling task, executing the selected scheduling task, sending an execution result to a message center, and updating and returning to the task list; and the service administration and configuration module stores the actuator parameters and the scheduling task parameters, so that the task scheduling service module can call the stored actuator parameters and the stored scheduling task parameters.
Description
Technical Field
The present invention relates to a micro service system and a control method thereof, and more particularly, to a micro service system and a task scheduling method for task scheduling.
Background
The micro-service architecture is an API-oriented serial centralized strong management and control service appearing on the system boundary, wherein the boundary is the boundary of an enterprise IT system and mainly plays a role in isolating external access from an internal system. Before the popularity of the micro-service concept, the entity of the API gateway has born a front-end system common in the fields of banks, securities, and the like, and the front-end system is also a main way for solving the problems of access authentication, message conversion, access statistics, and the like. The mobile application and the enterprise are interconnected, so that objects supported by the background service are expanded to multiple use scenes from the previous single Web application, and the requirements of each use scene on the background service are different. This not only increases the amount of response of the background service, but also increases the complexity of the background service. With the introduction of the micro-service architecture concept, the API gateway becomes a standard component of the micro-service architecture.
Goals and vision with microservice architecture:
1) Local modification, local update. When the operation and maintenance modifies a single application, the whole package may be stopped first and then modified, and the micro-service only needs to be modified and updated step by step;
2) Fault isolation, not global. The monolithic applications are run together so that if one module has a problem, the others will have problems. The fault isolation and the business sustainability of the micro-service are very high;
3) The resource utilization rate is high. The resource utilization rate of the single application is low, and the micro service is used, so that the resources can be distributed according to the requirement, and the resource utilization rate is very high;
4) The automatic operation and maintenance management platform realizes the automation of operation and maintenance;
5) Standardized construction, continuous integration, continuous delivery and the like are carried out on the basis of containers;
6) Performing service discovery and management on containers deployed in the microservices based on a standard tool;
7) The container is subjected to automatic telescopic management and automatic operation and maintenance management through an arrangement tool of the container;
8) The service inside the Pod is monitored through automatic operation and maintenance, and the corresponding responsible person is informed in time through early warning, so that the corresponding responsible person can receive the reason for the warning in the first time and feed back the result in time.
When the existing micro service architecture is applied, the requirements on resources and performance in the process of scheduling local CPU and memory resources are higher. On one hand, the background process needs to be terminated when the current application is finished to release the memory, so that the requirements on local CPU and memory resources are high; on the other hand, this also results in that the application process needs to be restarted when the next service is performed, and frequent initialization will limit the performance of the micro-service, such as class loading or just-in-time compilation during the running process, which will decrease the performance of the micro-service.
Taking distributed task scheduling service as an example, a set of independent task scheduling service exists in each of a plurality of micro-service systems in the same company, and each system performs task scheduling, so that the following main problems exist:
1. the wheel is repeatedly manufactured, each system in the same company develops respective task scheduling functions, and development cost is increased;
2. each system of the same company deploys respective task scheduling service, which wastes resources such as a CPU (central processing unit), a memory, storage and the like of a server;
3. task scheduling functions developed by developers of different systems have different quality, non-uniform quality and more problems;
4. the timing task functions of all systems of the same company cannot be monitored uniformly, and problems are difficult to find and troubleshoot in time.
On the other hand, when the micro service platform is utilized in the vehicle-related application, it needs to face a scheduling problem with high requirement, because the requirement of timely feedback of the vehicle application is high, and the existing general micro service platform needs to be adapted to the system architecture and the scheduling scheme.
Disclosure of Invention
Aiming at the prior art, the invention provides a micro-service system for task scheduling and a task scheduling method thereof, which can at least solve the problems of repeated task scheduling, high system resource requirement and the like.
In order to achieve the purpose, the invention adopts the following technical scheme:
a microservice system for task scheduling, comprising: the authority authentication module identifies user login information according to the login request and grants service authority according to the user login information; the task scheduling service module is configured to: creating an actuator according to the user login information, configuring one or more actuator parameters matched with the user login information, and returning to an actuator list; selecting a scheduling task in the task list or automatically initiating a timing scheduling task, executing the selected scheduling task, sending an execution result to a message center, and updating and returning the execution result to the task list; and the service administration and configuration module stores the actuator parameters and the scheduling task parameters, so that the task scheduling service module can call the stored actuator parameters and the stored scheduling task parameters.
As an implementation mode of the invention, the micro-service system also comprises a service monitoring module which monitors the authority authentication module and the task scheduling service module in real time and fuses the specific task scheduling service according to the trigger mechanism.
As an embodiment of the present invention, the microservice system further includes a load balancing module, where the load balancing module concurrently processes login requests and balances peak login requests.
As an implementation mode of the invention, the micro-service system further comprises a database cluster, and the task scheduling service module, the service administration and configuration module and the service monitoring module respectively call data of the database cluster.
As one embodiment of the invention, the task scheduling service module comprises a scheduling layer and an execution layer.
As an embodiment of the present invention, the scheduling layer includes: the task management unit is set as a creation executor and configures executor parameters matched with the user login information; the executor management unit is used for configuring a registration mode and a machine address list of an executor; and the log management unit is used for setting a scheduling log for configuring the actuator parameters.
In one embodiment of the invention, the execution layer invokes data within the database cluster based on the configuration of the scheduling layer.
As an implementation mode of the invention, for the manual execution of the task, a user firstly initiates the execution of the scheduling task, and the front end of the task scheduling service module selects the corresponding task in the task list and executes the task; after the task is executed, the front end of the task scheduling service module sends a task execution completion instruction to the back end.
As an implementation mode of the invention, for the automatic execution task, the rear end of the task scheduling service module directly and automatically initiates the timing scheduling task, executes the selected scheduling task, saves the execution log and the system log of the executed task, sends the execution result to a shared message notification system of one of the subsystems of the task scheduling service module, and updates and returns the task list.
In order to achieve the purpose, the invention also adopts the following technical scheme:
a task scheduling method using a micro service system includes: identifying user login information according to the login request, and granting service authority according to the user login information; creating an actuator according to the user login information, configuring one or more actuator parameters matched with the user login information, and returning to an actuator list; configuring a scheduling task, wherein the scheduling task is configured to select an actuator meeting configuration conditions, so as to select actuator parameters executed by the actuator, and return the actuator parameters to a task list; selecting a scheduling task in the task list or automatically initiating a timing scheduling task, executing the selected scheduling task, sending an execution result to a message center, and updating and returning the execution result to the task list.
As an implementation mode of the invention, the user login information is used for replacing the user mark, and whether the user has the distributed task scheduling service authority or not is judged according to the user mark.
According to the embodiment of the invention, the executor data and the scheduling task data are divided into two groups of independent data streams, and the two groups of independent data streams are respectively used for monitoring alarm, message notification and log recording.
As an embodiment of the present invention, executing the selected scheduled task includes manually executing the tasks: for the manual execution task, firstly, initiating an execution scheduling task, selecting a corresponding task from a task list, and executing the task; and after the task is executed, sending an instruction for completing the task execution.
As an embodiment of the present invention, executing the selected scheduled task includes automatically executing the tasks: for the automatic execution task, the timing scheduling task is directly and automatically initiated, the selected scheduling task is executed, the execution log and the system log of the executed task are saved, the execution result is sent to the message center, and the return task list is updated.
In the technical scheme, the dispatching efficiency of the system in the automobile related application can be improved, the dispatching requirement is met, and hardware resources are reduced.
Drawings
FIG. 1 is a schematic diagram of the architecture of the microservice system of the present invention;
FIG. 2 is a schematic diagram of a portion of the architecture of FIG. 1;
FIG. 3 is a timing diagram of the task scheduling method of the present invention;
FIG. 4 is a timing diagram of an authentication flow of the rights authentication module;
FIG. 5 is a data flow diagram of a task scheduling service.
Detailed Description
The technical solution in the embodiments of the present invention will be further clearly and completely described below with reference to the accompanying drawings and embodiments. It is obvious that the described embodiments are used for explaining the technical solution of the present invention, and do not mean that all embodiments of the present invention have been exhaustively exhausted.
Examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are illustrative and intended to be illustrative of the invention and are not to be construed as limiting the invention.
Referring to fig. 1, the present invention first discloses a micro-service system for task scheduling, which includes 7 main functional modules, which are respectively an authority authentication module IAM, a task scheduling service module Pod, a service administration and configuration module, a load balancing module (SLB/Ingress), a service monitoring module, an operation and maintenance module, a database cluster (infrastructure module), and the like. The service management and configuration module mainly comprises a service registration unit, a service discovery unit and a unified configuration unit, and the service monitoring module mainly comprises a fusing protection unit, a real-time monitoring service unit, a log service unit and a call chain tracking unit. As an embodiment of the present invention, the operation and maintenance module and the database cluster of the present invention may include various software platforms and database platforms, for example, the operation and maintenance module may include software platforms such as Git deployment, jenkins, docker, kubernets, etc., and the database cluster may include databases such as MySQL, redis, kafka, mongoDB, etc., which is not limited in this respect.
As shown in fig. 1, the authority authentication module identifies user login information according to a login request, each system user first inputs a user and a password thereof on the unified identity authentication interface of the authority authentication module, and the authority authentication module grants a service authority according to the user login information. The authority authentication module identifies the user authority, and if the user has the service authority of the distributed task scheduling service, the user can log in the distributed task scheduling service interface in a single point. The load balancing module concurrently processes the login requests, all login requests are subjected to load balancing through the load balancing module, and a high concurrent processing mechanism of the load balancing module ensures that all login requests are effectively processed, so that peak login requests are balanced.
The task scheduling service module is a core business function of the system shown in fig. 1, and is used for creating an executor, configuring task details, executing a task, recording a log, giving an abnormal alarm, displaying a report, and the like. The task scheduling service module includes a scheduling layer and an execution layer, as shown in fig. 1 and fig. 2, the scheduling layer mainly includes a task management unit, an executor management unit, a log management unit, and other service units, and the scheduling layer calls the execution layer in the task scheduling service module through a Remote Procedure Call (RPC). The execution layer mainly comprises functions of executor service, log file execution, ready callback, scheduling result queue feedback and the like. According to the configuration of the scheduling layer, the execution layer accesses a DB (database) layer of the database cluster, such as a DEV-DEV library, a TEST-TEST library, a PROD-PRODUCT library of MySQL database, and the like, through scheduling of the scheduling layer.
Referring to fig. 2, in the task scheduling service module, the task management unit creates an executor according to the user login information, configures one or more executor parameters matched with the user login information, and outputs and returns to the executor list. The task management unit sets task modes, and each task mode comprises a task list. And a dispatcher Job Handler in the task management unit is used for selecting a scheduling task or automatically initiating a timing scheduling task in the task list, executing the selected scheduling task, sending an execution result to a message center, and updating and returning the execution result to the task list.
With continued reference to fig. 2, in the task scheduling service module, the executor management unit is configured to control functions of a registration mode of the executor, an executor name (AppName), a machine address list of the executor, and the like; the log management unit is configured to control scheduling logs of actuator parameters, record various log data such as GLUE logs and system logs; and the other service units are configured to run the report forms, send out failure alarms when the running fails, and process data such as task dependence conditions and the like.
With continued reference to fig. 1, the task scheduling service module is connected to the configuration module through its subsystem and service administration, and the subsystem mainly includes a third party service system and a public message notification system. The service administration and configuration module is a micro-service registration center and a configuration file unified management center, and mainly comprises a service registration unit, a service discovery unit and a unified configuration unit. Wherein the service registration unit is configured to register one or more services matching the user login information; the service discovery unit is configured to monitor an executor and a task list of the task scheduling service module so as to discover a specific task; the unified configuration unit stores the actuator parameters and the scheduling task parameters, so that the task scheduling service module can call the stored actuator parameters and the stored scheduling task parameters.
As shown in fig. 1 and fig. 2, the task scheduling service module uses a database-based distributed function of Quartz, and in order to ensure "lightweight" of the system and reduce the cost of learning and deployment, the service administration and configuration module performs task registration discovery in a DB manner. With this arrangement, once a new actuator machine goes online or offline, the task will be reassigned the next time it is scheduled. The executor in the task scheduling service module can have rich routing strategies when being deployed, and the method comprises the following steps: first, last, polled, random, coherent HASH, least frequently used, least recently used, failover, busy transfer, and the like. The one-time complete task flow comprises two stages of scheduling (scheduling center) + executing (actuator). If the 'failover' occurs in the scheduling stage, when an actuator cluster is deployed, if a certain actuator fails, the strategy supports automatic switching to a normal actuator machine and completion of a scheduling request process. If the 'failure retry' occurs in two stages of 'scheduling and executing', the task scheduling service module supports the user-defined task failure retry times, and when the task fails, the task will be actively retried according to the preset failure retry times.
On the other hand, the processing strategy when the task scheduling misses the trigger time is as follows:
the possible reasons are as follows: restarting the service; the scheduling thread is blocked and the thread is exhausted; the last scheduling is continuously blocked, and the next scheduling is missed;
and (3) processing strategies:
overdue for more than 5s: neglecting this time, and calculating the next triggering time at the current time;
within 5s of expiration: and immediately triggering once, and starting to calculate the next triggering time at the current time.
Intensive scheduling or time-consuming tasks of the task scheduling service module may cause task blocking, and the scheduling layer may be triggered repeatedly with a small probability. For the above situation, the task can be avoided by combining a "single machine routing policy (such as the first station, consistent hash)" + "blocking policy (such as the single machine serial, and subsequent scheduling discard)", and finally, the repeated execution of the task is avoided.
When the executor is in abnormal conditions such as network jitter callback failure or downtime, the task scheduling result is lost. Since the task management unit relies on the executor callback to sense the scheduling result, the scheduling log is always in a "running" state.
Aiming at the problem, the dispatching center provides built-in components for processing, and the logic is as follows: if the scheduling record stays in the 'running' state for more than 10min and the corresponding executor fails to register online, the local scheduling initiative is marked to fail.
With reference to fig. 1, the service monitoring module monitors the authority authentication module and the task scheduling service module in real time, and mainly includes a fusing protection unit, a real-time monitoring service unit, a log service unit, and a call chain tracking unit. The fusing protection unit sets a degradation strategy and a fusing strategy of the micro-service, and the specific task scheduling service is degraded or even fused by triggering the relevant strategy according to the triggering mechanism.
The task scheduling service module, the service administration and configuration module and the service monitoring module can call data of the database cluster respectively.
Referring to fig. 3, in addition to the above-mentioned micro service system, the present invention also discloses a task scheduling method using the micro service system, and fig. 3 is a timing chart of executing task scheduling by the method.
The main control logic of the task scheduling method is to set a task automatically triggered at a certain time point, the task can be circularly executed on a time rule, and the method mainly has the following characteristics:
time driving: generally, the method can be driven by time, and is timed, fixed-point and fixed-time;
batch treatment: the method has the advantages that the method is more effective in batch processing of the accumulated data, has more advantages than a message middleware under the condition of not needing real-time performance, and some business logics can only be processed in batches;
non-real-time property: the timing task does not require real-time performance, is generally different from the interaction of a C-end user, and is more used for processing service data;
isolation/specificity: the method can be separated from other systems, only focuses on the processing of the service data, and does not influence the operation of the user and the performance of the user system.
As shown in fig. 3, when the user logs in, the authority authentication module identifies the user login information according to the login request, and grants the service authority according to the user login information. And after the login is finished, the authority authentication module jumps the task to the front end of the task scheduling service module. After the task jumps to the scheduling layer, the front end of the task scheduling service module interacts the code (code) obtained by the task application with the rear end (authentication center) of the task scheduling service module to obtain a token (token), and meanwhile, the rear end of the task scheduling service module can obtain user information from the authority authentication module according to the token (token) and can judge whether the user has distributed task scheduling service authority or not according to the user mark. After the user information is acquired, the back end of the task scheduling service module returns the menu and the user information, and thus the user login process is completed.
With continued reference to FIG. 3, after the user completes the login, the task scheduling service module may create an executor from the user login information. One or more actuator parameters matched with the user login information are configured at the front end of the task scheduling service module, and the actuator parameters are stored at the rear end of the task scheduling service module. And after the storage is finished, the back end of the task scheduling service module returns the executor list and the storage content.
After the configuration of the executor is completed, the task scheduling service module can continue to configure the scheduling task, the front-end scheduling task configuration of the task scheduling service module selects the executor meeting the configuration conditions, so that the executor parameters executed by the executor are selected, and the corresponding task parameters are sent to the rear end of the task scheduling service module. And the rear end of the task scheduling service module stores the task parameters and returns to the task list.
With continued reference to fig. 3, the task scheduling method of the present invention includes two ways of performing tasks manually and automatically.
For the manual task execution mode, a user firstly initiates and executes a scheduling task, and the front end of the task scheduling service module selects a corresponding task from the task list and executes the task. After the task is executed, the front end of the task scheduling service module sends a task execution completion instruction to the back end. At this time, the back end of the task scheduling service module saves the execution log and the system log of the executed task, sends the execution result to a common message notification system (message center) which is one of the subsystems of the task scheduling service module, and returns the task list.
For the automatic task executing mode, the back end of the task scheduling service module directly and automatically initiates a timing scheduling task, executes the selected scheduling task, saves an execution log and a system log of the executed task, sends an execution result to a common message notification system (message center) of one of the subsystems of the task scheduling service module, and updates and returns a task list.
As can be seen from fig. 1, fig. 2, and fig. 3, the task scheduling method of the present invention mainly utilizes the task scheduling service module, the service monitoring module, and the operation and maintenance module, and on one hand, can perform distributed operation on the micro-service, and deliver the micro-service to the service administration and configuration module for unified permission management. On the other hand, the local memory resources are monitored through the service monitoring module, the service process is temporarily stored or called out through the operation and maintenance module, the service process does not need to be stopped or initialized for many times, and the micro-service performance is effectively improved.
In addition, the invention can also monitor the information data exchange of the service end by setting the service monitoring module, ensure the processing performance in the interaction process of the service process and the Internet, and collect the processing process information of the service process by establishing a feedback system so as to adjust the task scheduling service module, or feedback through a gateway adjusting micro-service system according to the received gateway information to request to forward or feed back abnormal information.
Referring to fig. 4, fig. 4 shows a login and authentication flow of the authority authentication module IAM for the user identity.
As shown in fig. 4, the authority authentication module sends the request to a Software Development Kit (SDK) through an oauth/callback protocol, and the Software Development Kit (SDK) stores information such as a token (token) and forwards the information such as the token (token) to the front end of the task scheduling service module. In this process, a Software Development Kit (SDK) redirects to the front-end index address of the task scheduling service module and concatenates the applied code (code) at the end of the address.
The front end of the task scheduling service module calls an oauth protocol/token (token) interface of a Software Development Kit (SDK) according to the code (code) of the application to obtain a jwt token (jwtToken), and the Software Development Kit (SDK) returns the jwt token (jwtToken) to the front end of the task scheduling service module.
The front end of the task scheduling service module locally stores a jwt token (jwtToken), and a rear end interface of the task scheduling service module is called to obtain a menu tree and user information. The back end of the task scheduling service module obtains relevant information from a Software Development Kit (SDK) through a thread variable (thread local) and sends the relevant information to the front end of the task scheduling service module. And finally, the front end of the task scheduling service module displays the corresponding menu tree and the user information on a front end page.
The control sequence of fig. 4 is further described below by one embodiment.
The first interface of the authority authentication module: and acquiring a code (code) of the application through the interface, interactively acquiring a token (token) with the authentication center, and requesting the token (token) interface by using an http mode.
The request mode comprises the following steps: post
Request protocol: http protocol
Request parameters:
and returning parameters:
the second interface of the authority authentication module: refreshing access _ token according to refresh _ token instruction
The request mode comprises the following steps: post
Request protocol: http protocol
Request parameters:
returning parameters:
the third interface of the authority authentication module: requesting to obtain user information from authentication according to token (token)
The request mode comprises the following steps: get to
Request protocol: http protocol
Request parameters:
and returning parameters:
and analyzing the returned data to obtain user information, and matching the account data of the platform by the application system according to the field to ensure that the user passes the authentication.
The fourth interface of the authority authentication module: checking whether the token is valid or not to the authentication center according to the token
If the application system needs to perform a unified logout, each time an interface is requested, it needs to check whether the token (token) is valid. The specific interfaces are as follows:
the request mode comprises the following steps: get to
Request protocol: http protocol
Request parameters:
returning parameters: and (4) success.
Referring to fig. 5, when the task scheduling service module configures an executor and schedules a task, the executor data and the scheduling task data are divided into two independent data streams. Two independent data streams can be used for monitoring alarm, message notification, log recording and other functions. In other words, the data sources are mainly two parts, one part is user executor data, and the second part is task data. The data output is in three directions, namely a monitoring alarm function, a message notification center and a log recording function.
In summary, the invention discloses a micro-service system capable of being used for task scheduling and a corresponding task scheduling method thereof, wherein tasks of a third-party platform are subjected to distributed scheduling management through a distributed task scheduling service module, and are uniformly managed by the task scheduling service module, all the third-party platforms needing task scheduling service can be uniformly accessed, uniformly authorized, uniformly monitored and uniformly alarmed, the workload of the third-party platforms is reduced, the safety of the task scheduling service is enhanced, the usability is enhanced, and the purposes of efficiency improvement and cost reduction are achieved.
The system and the method of the invention can at least bring the following beneficial effects:
1. unified task access management: the task scheduling service module is unified to perform operations such as creation of actuators and creation and stop of tasks on all accessed systems, so that the management pressure of a third-party system is reduced, and the third-party system can be smoothly accessed by introducing a JAR (Java object manager) packet of task scheduling service;
2. unified user authority control: the unified authority authentication module is used for carrying out authority control, menu authority control and data isolation display on all accessed third-party system access users;
3. unified monitoring: all accessed third-party system logs are uniformly stored and uniformly displayed, and the states of all tasks are uniformly displayed, so that the workload of a third-party access system is saved;
4. unified warning: all abnormal alarms generated by the accessed third-party system carry out unified short message alarm and mail alarm through the notification service, and each system is not required to process alarm information respectively.
The apparatus of the present invention may be a computer program product, wherein the computer program product, when run on a computer, causes the computer to perform some or all of the steps of the method as in the above method embodiments.
The apparatus of the present invention may be an application publishing platform for publishing a computer program product, wherein the computer program product, when run on a computer, causes the computer to perform some or all of the steps of a method as in the above method embodiments.
In various embodiments of the present invention, it should be understood that the sequence numbers of the above-mentioned processes do not imply an inevitable order of execution, and the execution order of the processes should be determined by their functions and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
The various systems and elements of the invention, if implemented as software functional elements, may be stored in a computer accessible memory. With this understanding, part or all of the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a memory and includes several requests to enable one or more computer devices (such as a personal computer, a server, or a network device, and may specifically be a processor in the computer device) to execute part or all of the steps of the above methods according to the embodiments of the present invention.
Those skilled in the art will appreciate that all or a portion of the steps of the various embodiments recited herein may be performed by associated hardware as instructed by a computer program that may be stored centrally or distributed on one or more computer devices, such as on a readable storage medium. Such computer devices may include Read-Only Memory (ROM), random Access Memory (RAM), programmable Read-Only Memory (PROM), erasable Programmable Read-Only Memory (EPROM), one-time Programmable Read-Only Memory (OTPROM), electrically Erasable Programmable Read-Only Memory (EEPROM), compact Disc Read-Only Memory (CD-ROM) or other optical disk storage, magnetic disk storage, tape storage, or any other medium from which a computer can Read or store data.
It should be understood by those skilled in the art that the above embodiments are only for illustrating the present invention and are not to be used as a limitation of the present invention, and that changes and modifications to the above described embodiments are within the scope of the claims of the present invention as long as they are within the spirit and scope of the present invention.
Claims (14)
1. A microservice system for task scheduling, comprising:
the authority authentication module identifies user login information according to the login request and grants service authority according to the user login information;
a task scheduling service module configured to:
creating an actuator according to the user login information, configuring one or more actuator parameters matched with the user login information, and returning to an actuator list;
selecting a scheduling task in the task list or automatically initiating a timing scheduling task, executing the selected scheduling task, sending an execution result to a message center, and updating and returning to the task list;
and the service administration and configuration module stores the actuator parameters and the scheduling task parameters, so that the task scheduling service module can call the stored actuator parameters and the stored scheduling task parameters.
2. The micro-service system for task scheduling according to claim 1, further comprising a service monitoring module, wherein the service monitoring module monitors the authority authentication module and the task scheduling service module in real time, and fuses a specific task scheduling service according to a trigger mechanism.
3. The microservice system for task scheduling of claim 1, further comprising a load balancing module that concurrently processes login requests and balances peak login requests.
4. The microservice system for task scheduling of claim 1 further comprising a database cluster, the task scheduling service module, the service administration and configuration module, and the service monitoring module each invoking data of the database cluster.
5. The microservice system for task scheduling of claim 1, wherein the task scheduling service module comprises a scheduling layer and an execution layer, and wherein the task scheduling service module is configured to execute tasks manually or automatically.
6. The microservice system for task scheduling of claim 5, wherein the scheduling layer comprises:
the task management unit is set to create an executor and configure executor parameters matched with user login information;
the system comprises an executor management unit, a register unit and a machine address list, wherein the executor management unit is used for configuring a register mode and a machine address list of an executor;
a log management unit configured to configure a dispatch log of the actuator parameters.
7. The microservice system for task scheduling according to claim 5, wherein the execution layer invokes data within the database cluster according to the configuration of the scheduling layer.
8. The microservice system for task scheduling as claimed in claim 5, wherein for a manual execution task, a user first initiates the execution of the scheduled task, and the front end of the task scheduling service module selects a corresponding task in the task list and executes the task; after the task is executed, the front end of the task scheduling service module sends a task execution completion instruction to the back end.
9. The microservice system for task scheduling according to claim 5, wherein for an auto-execute task, the backend of the task scheduling service module directly and automatically initiates a scheduled task, executes the selected scheduled task, saves the execution log and the system log of the executed task, sends the execution result to the shared message notification system of one of the subsystems of the task scheduling service module, and updates the returned task list.
10. A task scheduling method using a micro service system is characterized by comprising the following steps:
identifying user login information according to the login request, and granting service authority according to the user login information;
creating an actuator according to the user login information, configuring one or more actuator parameters matched with the user login information, and returning to an actuator list;
configuring a scheduling task, wherein the scheduling task is configured to select an actuator meeting configuration conditions, so as to select actuator parameters executed by the actuator and return to a task list;
and selecting a scheduling task in the task list or automatically initiating a timing scheduling task, executing the selected scheduling task, sending an execution result to a message center, and updating and returning to the task list.
11. The method for task scheduling with a microservice system of claim 10, wherein:
and exchanging the user login information for a user mark, and judging whether the user has the distributed task scheduling service authority or not according to the user mark.
12. The task scheduling method using a microservice system according to claim 11, wherein:
dividing the data of the actuator and the data of the scheduling task into two groups of independent data streams, and respectively utilizing the two groups of independent data streams to carry out monitoring alarm, message notification and log recording.
13. The method of task scheduling with a microservice system of claim 10, wherein the executing the selected scheduled task comprises manually executing the task:
for the manual execution task, firstly, initiating an execution scheduling task, selecting a corresponding task from a task list, and executing the task; and after the task is executed, sending an instruction for completing the task execution.
14. The method for task scheduling with a microservice system of claim 10, wherein the executing the selected scheduled task comprises automatically executing the tasks:
for the automatic execution task, the timing scheduling task is directly and automatically initiated, the selected scheduling task is executed, the execution log and the system log of the executed task are saved, the execution result is sent to the message center, and the return task list is updated.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211283489.9A CN115357403A (en) | 2022-10-20 | 2022-10-20 | Micro-service system for task scheduling and task scheduling method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211283489.9A CN115357403A (en) | 2022-10-20 | 2022-10-20 | Micro-service system for task scheduling and task scheduling method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115357403A true CN115357403A (en) | 2022-11-18 |
Family
ID=84008701
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211283489.9A Pending CN115357403A (en) | 2022-10-20 | 2022-10-20 | Micro-service system for task scheduling and task scheduling method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115357403A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111382051A (en) * | 2018-12-29 | 2020-07-07 | 北京数聚鑫云信息技术有限公司 | API automatic test method and test device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180321996A1 (en) * | 2017-05-04 | 2018-11-08 | Microsoft Technology Licensing, Llc | Micro- service framework derived from third-party apps |
| WO2020162680A1 (en) * | 2019-02-08 | 2020-08-13 | 아콘소프트 주식회사 | Microservice system and method |
| CN112052081A (en) * | 2020-08-13 | 2020-12-08 | 烽火通信科技股份有限公司 | Task scheduling method and device and electronic equipment |
| CN112564916A (en) * | 2020-12-01 | 2021-03-26 | 上海艾融软件股份有限公司 | Access client authentication system applied to micro-service architecture |
| CN113535362A (en) * | 2021-07-26 | 2021-10-22 | 北京计算机技术及应用研究所 | Distributed scheduling system architecture and micro-service workflow scheduling method |
| US20220067044A1 (en) * | 2019-05-14 | 2022-03-03 | Huawei Technologies Co., Ltd. | Microservice component-based database system and related method |
-
2022
- 2022-10-20 CN CN202211283489.9A patent/CN115357403A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180321996A1 (en) * | 2017-05-04 | 2018-11-08 | Microsoft Technology Licensing, Llc | Micro- service framework derived from third-party apps |
| WO2020162680A1 (en) * | 2019-02-08 | 2020-08-13 | 아콘소프트 주식회사 | Microservice system and method |
| US20220067044A1 (en) * | 2019-05-14 | 2022-03-03 | Huawei Technologies Co., Ltd. | Microservice component-based database system and related method |
| CN112052081A (en) * | 2020-08-13 | 2020-12-08 | 烽火通信科技股份有限公司 | Task scheduling method and device and electronic equipment |
| CN112564916A (en) * | 2020-12-01 | 2021-03-26 | 上海艾融软件股份有限公司 | Access client authentication system applied to micro-service architecture |
| CN113535362A (en) * | 2021-07-26 | 2021-10-22 | 北京计算机技术及应用研究所 | Distributed scheduling system architecture and micro-service workflow scheduling method |
Non-Patent Citations (1)
| Title |
|---|
| 田晶晶: "基于微服务的数据交换与共享平台的研究与实现", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111382051A (en) * | 2018-12-29 | 2020-07-07 | 北京数聚鑫云信息技术有限公司 | API automatic test method and test device |
| CN111382051B (en) * | 2018-12-29 | 2023-04-18 | 北京数聚鑫云信息技术有限公司 | API automatic test method and test device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8484659B2 (en) | Distributed processing of binary objects via message queues including a failover safeguard | |
| EP3837604B1 (en) | In situ triggered function as a service within a service mesh | |
| US10454795B1 (en) | Intermediate batch service for serverless computing environment metrics | |
| US20060095914A1 (en) | System and method for job scheduling | |
| CN110289965B (en) | Application program service management method and device | |
| CN111711697A (en) | Message pushing method, device, equipment and storage medium | |
| CN112799786A (en) | Exit method, device, equipment and storage medium of micro-service instance | |
| CN111258723B (en) | Transaction processing method, device, system, medium and equipment of distributed system | |
| CN110673933A (en) | ZooKeeper-based distributed asynchronous queue implementation method, device, equipment and medium | |
| US20230046979A1 (en) | Microservice call method and apparatus, device, and medium | |
| CN111858007A (en) | Task scheduling method and device based on message middleware | |
| US20230275976A1 (en) | Data processing method and apparatus, and computer-readable storage medium | |
| CN112433863A (en) | Micro-service calling method and device, terminal equipment and storage medium | |
| CN115357403A (en) | Micro-service system for task scheduling and task scheduling method | |
| CN115695139A (en) | Method for enhancing micro-service system architecture based on distributed robust | |
| CN112787999A (en) | Cross-chain calling method, device, system and computer readable storage medium | |
| CN113342503B (en) | Real-time progress feedback method, device, equipment and storage medium | |
| CN111835797A (en) | Data processing method, device and equipment | |
| US11093309B1 (en) | Communication hub for information technology (IT) services | |
| US8458716B2 (en) | Enterprise resource planning with asynchronous notifications of background processing events | |
| US20230147232A1 (en) | Database connection refresh for server instances and live database updates | |
| CN116319810A (en) | Flow control method, device, equipment, medium and product of distributed system | |
| CN113542409B (en) | Management system and processing method for instances of RocktMQ message queues | |
| US20220276901A1 (en) | Batch processing management | |
| CN113553152A (en) | Job scheduling method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20221118 |