CN117632409A - Method and device for scheduling data acquisition timing tasks - Google Patents
Method and device for scheduling data acquisition timing tasks Download PDFInfo
- Publication number
- CN117632409A CN117632409A CN202311467047.4A CN202311467047A CN117632409A CN 117632409 A CN117632409 A CN 117632409A CN 202311467047 A CN202311467047 A CN 202311467047A CN 117632409 A CN117632409 A CN 117632409A
- Authority
- CN
- China
- Prior art keywords
- data acquisition
- task
- acquisition timing
- timing task
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 72
- 230000014509 gene expression Effects 0.000 claims description 22
- 240000008042 Zea mays Species 0.000 claims description 17
- 235000005824 Zea mays ssp. parviglumis Nutrition 0.000 claims description 17
- 235000002017 Zea mays subsp mays Nutrition 0.000 claims description 17
- 235000005822 corn Nutrition 0.000 claims description 17
- 238000012545 processing Methods 0.000 claims description 17
- 230000004048 modification Effects 0.000 claims description 16
- 238000012986 modification Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 13
- 230000000977 initiatory effect Effects 0.000 claims description 5
- 230000008569 process Effects 0.000 description 14
- 238000013480 data collection Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 4
- 238000012217 deletion Methods 0.000 description 3
- 230000037430 deletion Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000010453 quartz Substances 0.000 description 2
- VYPSYNLAJGMNEJ-UHFFFAOYSA-N silicon dioxide Inorganic materials O=[Si]=O VYPSYNLAJGMNEJ-UHFFFAOYSA-N 0.000 description 2
- 241000283973 Oryctolagus cuniculus Species 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention provides a method and a device for scheduling a data acquisition timing task, wherein the method comprises the following steps: under the condition that the completion of the starting of the timing task scheduling service is determined, acquiring task parameters of a newly-built data acquisition timing task; under the condition that the number of the existing data acquisition timing tasks in execution does not exceed a first preset value, establishing a Redis lock corresponding to the new data acquisition timing task by taking the identification information of the new data acquisition timing task as a key value; and under the condition that the Redis lock corresponding to the newly-built data acquisition timing task is successfully established, starting the execution of the newly-built data acquisition positioning task. The method and the device for scheduling the data acquisition timing task can simply and efficiently add the new data acquisition timing task, can flexibly configure the execution period of the new data acquisition timing task, can dynamically acquire the rail transit operation data, and can better meet the acquisition requirement of the rail transit operation data.
Description
Technical Field
The invention relates to the technical field of rail transit, in particular to a method and a device for scheduling data acquisition timing tasks.
Background
In rail traffic operation, data acquisition is a very important task. By collecting and analyzing operation data generated in the rail transit operation process, the vehicle running state, passenger flow, ticket information and the like can be monitored in real time, and further data support and operation optimization suggestions can be provided for traffic management departments.
In the related art, operation data generated in the track traffic operation process can be collected based on a timing task scheduling scheme.
However, due to the huge and complex track traffic network and the large number of vehicles, the conditions of vehicle scheduling, data transmission and the like related to data acquisition are complex, so that the traditional timing task scheduling scheme is difficult to meet the acquisition requirement of track traffic operation data.
Disclosure of Invention
The invention provides a data acquisition timing task scheduling method and device, which are used for solving the defect that the traditional timing task scheduling scheme in the prior art is difficult to meet the acquisition requirement of rail transit operation data, and realizing better meeting of the acquisition requirement of the rail transit operation data.
The invention provides a method for scheduling a data acquisition timing task, which comprises the following steps:
under the condition that the completion of the starting of the timing task scheduling service is determined, acquiring task parameters of a newly-built data acquisition timing task, wherein the task parameters comprise identification information and execution period information of the task;
Under the condition that the number of the existing data acquisition timing tasks in execution does not exceed a first preset value, establishing a Redis lock corresponding to the new data acquisition timing task by taking the identification information of the new data acquisition timing task as a key value;
and under the condition that the Redis lock corresponding to the newly-built data acquisition timing task is successfully established, starting the execution of the newly-built data acquisition positioning task.
According to the method for scheduling the data acquisition timing task, which is provided by the invention, the specific steps of starting the task scheduling service comprise the following steps:
loading task parameters of all unexecuted existing data acquisition timing tasks;
judging whether the number of the executed existing data acquisition timing tasks exceeds a second preset value, and under the condition that the number of the executed existing data acquisition timing tasks does not exceed the second preset value, creating a Redis lock corresponding to any unexecuted existing data acquisition timing task by taking identification information of any unexecuted existing data acquisition timing task as a key value;
under the condition that the Redis lock corresponding to any unexecuted existing data acquisition timing task is successfully established, starting the execution of any unexecuted existing data acquisition timing task;
And returning to the step of judging whether the number of the executed existing data acquisition timing tasks exceeds a second preset value, until the number of the executed existing data acquisition timing tasks exceeds the second preset value, and confirming that the timing task scheduling service is started to be completed.
According to the method for scheduling the data acquisition timing task provided by the invention, the starting of the execution of the newly-built data acquisition positioning task comprises the following steps:
generating a corresponding turn expression of the new data acquisition timing task based on task parameters of the new data acquisition timing task, and creating a task thread corresponding to the new data acquisition timing task by using a schedule interface and a threadPoolTaskscheduler;
and adding the task thread corresponding to the new data acquisition timing task to a target thread pool so that the target thread pool can execute the new data acquisition positioning task at regular time based on the corresponding corn expression of the new data acquisition timing task.
The method for scheduling the data acquisition timing task provided by the invention further comprises the following steps:
under the condition that the completion of the starting of the timing task scheduling service is determined, acquiring a modification instruction of a target data acquisition timing task, wherein the target data acquisition timing task is an existing data acquisition timing task in execution needing modification, and the modification instruction carries task parameters of the modified data acquisition timing task;
Deleting a task thread corresponding to the target data acquisition timing task under the condition that the task thread corresponding to the target data acquisition timing task exists in the target thread pool, and creating a Redis lock corresponding to the modified data acquisition timing task by taking the identification information of the modified data acquisition timing task as a key value;
under the condition that the Redis lock corresponding to the modified data acquisition timing task is successfully established, a corn expression corresponding to the modified data acquisition timing task is generated based on task parameters of the modified data acquisition timing task, a task thread corresponding to the modified data acquisition timing task is established by using a ScheduedFuture interface and a ThreadPoolTaskScheduler scheduler, and then the task thread corresponding to the modified data acquisition timing task is added to a target thread pool so that the target thread pool can execute the new data acquisition positioning task at regular time based on the corn expression corresponding to the modified data acquisition timing task.
According to the method for scheduling data acquisition timing tasks provided by the invention, when the number of executed existing data acquisition timing tasks does not exceed the second preset value, identification information of any unexecuted existing data acquisition timing task is used as a key value, and after a Redis lock corresponding to any unexecuted existing data acquisition timing task is created, the method further comprises:
And under the condition that the Redis lock corresponding to any unexecuted existing data acquisition timing task is not successfully established, eliminating any unexecuted existing data acquisition timing task from all unexecuted existing data acquisition timing tasks, and returning to the step of judging whether the number of the unexecuted existing data acquisition timing tasks exceeds a second preset value or not until the Redis lock corresponding to any unexecuted existing data acquisition timing task is successfully established.
The method for scheduling the data acquisition timing task provided by the invention further comprises the following steps:
under the condition that the timing task scheduling service is determined to be started to be completed, receiving a target instruction of a target data acquisition timing task, wherein the target data acquisition timing task is an existing data acquisition timing task in execution, and the target instruction comprises any one of a deleting instruction, a checking instruction and a stopping instruction;
and executing the target instruction on the task thread corresponding to the target data acquisition timing task under the condition that the target thread Chi Cun is determined to be the task thread corresponding to the target data acquisition timing task.
The method for scheduling the data acquisition timing task provided by the invention further comprises the following steps:
Under the condition that the starting completion of the timing task scheduling service is determined, receiving a manual operation instruction of a target data acquisition timing task, wherein the target data acquisition timing task is an existing data acquisition timing task in execution which needs manual operation;
and under the condition that the target thread Chi Cun is determined to be the task thread corresponding to the target data acquisition timing task, executing the target data acquisition timing task based on an execution instruction input by a user.
The invention also provides a device for scheduling the data acquisition timing task, which comprises the following steps:
the data acquisition module is used for acquiring task parameters of the newly-built data acquisition timing task under the condition that the timing task scheduling service is determined to be started, wherein the task parameters comprise identification information and execution period information of the task;
the data processing module is used for creating a Redis lock corresponding to the new data acquisition timing task by taking the identification information of the new data acquisition timing task as a key value under the condition that the number of the existing data acquisition timing tasks in execution does not exceed a first preset value;
and the task starting module is used for starting the execution of the new data acquisition positioning task under the condition that the Redis lock corresponding to the new data acquisition timing task is successfully established.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the data acquisition timing task scheduling method according to any one of the above when executing the program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a data acquisition timing task scheduling method as described in any of the above.
The invention also provides a computer program product comprising a computer program which when executed by a processor implements a method of scheduling data acquisition timing tasks as described in any of the above.
According to the method and the device for scheduling the data acquisition timing tasks, under the condition that the starting of the timing task scheduling service is determined to be completed, if the task parameters of the new data acquisition positioning tasks are acquired, whether the number of the existing data acquisition timing tasks in execution exceeds a first preset value is judged, under the condition that the number of the existing data acquisition timing tasks in execution does not exceed the first preset value, identification information of the new data acquisition timing tasks is used as a key value, redis locks corresponding to the new data acquisition timing tasks are created, and under the condition that the Redis locks corresponding to the new data acquisition timing tasks are determined to be successfully created, the execution of the new data acquisition positioning tasks is started, the new data acquisition timing tasks can be added more simply and more efficiently, the execution period of the new data acquisition timing tasks can be configured more flexibly, rail traffic operation data can be acquired more flexibly, and the acquisition requirements of the rail traffic operation data can be met better.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a method for scheduling data acquisition timing tasks provided by the invention;
FIG. 2 is a second flow chart of a method for scheduling data acquisition timing tasks according to the present invention;
FIG. 3 is a schematic flow chart of a task scheduling service starting in the data acquisition timing task scheduling method provided by the invention;
FIG. 4 is a third flow chart of a method for scheduling data acquisition timing tasks according to the present invention;
FIG. 5 is a schematic diagram of a data acquisition timing task scheduling device provided by the invention;
fig. 6 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the description of the present application, the terms "first," "second," and the like are used for distinguishing between similar objects and not for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged, as appropriate, such that embodiments of the present application may be implemented in sequences other than those illustrated or described herein, and that the objects identified by "first," "second," etc. are generally of a type and not limited to the number of objects, e.g., the first object may be one or more. Further, in the description of the present application, "and/or" means at least one of the connected objects, and the character "/", generally means a relationship in which the associated objects are one kind of "or".
It should be noted that the timed task scheduling scheme refers to a mechanism or system for executing tasks according to a plan. The conventional timed task scheduling scheme includes: an operating system level scheduler, a timed task framework of a programming language, and a third party scheduling service.
Among them, most operating systems have schedulers built therein, for example: task schedulers in Windows operating systems and cron in Linux operating systems. The scheduler may allow a user to set timed tasks and specify the execution time and frequency of the tasks;
Many programming languages provide a framework for handling timed tasks, such as: a quantiz framework in the Java programming language, an APScheduler framework in the Python programming language, a node-cron framework in the node. Js programming language, a distributed timed task scheduling platform ElasticJob, xxl-job, and the like. The developer can use the timing task framework to set the timing task and the execution logic thereof;
there are also third party services specifically designed for timed task scheduling, such as: AWS Lambda, google Cloud Scheduler and Azure Logic Apps. The third party service can help the user easily set and manage the timing tasks and provide a highly reliable execution environment.
However, due to the huge and complex track traffic network, the number of vehicles is large, and the conditions of vehicle scheduling, data transmission and the like related to data acquisition are complex. The above-mentioned timing task scheduling scheme has many problems when applied to data acquisition in rail traffic operation, for example: the Quartz frame in Java programming language is in a single-machine mode and cannot be uniformly scheduled in micro services; elasticJob, xxl-job and other distributed timing task scheduling platforms, which are complicated to deploy due to the fact that the number of dependent three-party applications is large, so that the use cost of the distributed timing task scheduling platform is high; moreover, the task period cannot be dynamically configured by the conventional timing task scheduling scheme, so that the track traffic operation data is difficult to flexibly and dynamically collect based on the conventional timing task scheduling scheme, and the collection requirement of the track traffic operation data is further difficult to meet.
In this regard, the invention provides a data acquisition timing task scheduling method for the technical field of rail transit, which can better solve the scheduling problem of the distributed data acquisition timing problem of the rail transit operation data acquisition by combining the characteristics of Redis locks with the SchedudFuture interface in Jave, further can dynamically acquire the rail transit operation data more flexibly, and can better meet the acquisition requirement of the rail transit operation data.
Fig. 1 is a schematic flow chart of a data acquisition timing task scheduling method provided by the invention. The data acquisition timing task scheduling method of the present invention is described below with reference to fig. 1. As shown in fig. 1, the method includes: step 101, under the condition that the completion of the starting of the timing task scheduling service is determined, acquiring task parameters of a new data acquisition timing task, wherein the task parameters comprise identification information and execution period information of the task;
it should be noted that, the execution body of the embodiment of the present invention is a data acquisition timing task scheduling device.
Optionally, the data acquisition timing task scheduling device in the embodiment of the invention can be a coordinator serving timing task scheduling. Where coordinator refers to a component or tool for coordinating and managing various services in a distributed system. In a distributed system, due to the increased number of services and complex dependencies between services, a mechanism is required to coordinate and manage these services to ensure that the system can operate properly.
The coordinator of the timed task scheduling service may also be used to coordinate communications between the timed task scheduling service and other services.
Specifically, the data acquisition timing task scheduling method provided by the invention can schedule the data acquisition timing task under the condition that the timing task scheduling service is started. The scheduling may include any one of adding a new data acquisition timing task, deleting an existing data acquisition timing task, checking an existing data acquisition timing task, stopping an existing data acquisition timing task, starting an existing data acquisition timing task, and manually executing an existing data acquisition timing task.
It should be noted that, in the embodiment of the present invention, the timing task scheduling service may use a Spring closed micro-service framework. The Spring Cloud is an open source framework for constructing a distributed system and a micro-service architecture, and provides a set of tools and components for assisting developers in constructing, deploying and managing micro-services in the distributed system based on Spring Boot.
Fig. 2 is a second flowchart of a data acquisition timing task scheduling method according to the present invention. As shown in fig. 2, in the case that the user needs to add a new data acquisition timing task, the user may input task parameters of the new data acquisition timing task through an operation on the web interface.
The task parameters may include identification information and execution cycle information of a task; the identification information of the task can carry out unique identification on the task, and the identification information of the task can be an identification card identification number (Identity Document, ID) of the task; the task parameter may further include name information of the task.
After the user inputs the task parameters of the newly-built data acquisition timing task through the operation of the web interface, the task parameters of the newly-built data acquisition timing task can be forwarded to the control center as web request information through the gateway.
The control center can verify task parameters of the newly-built data acquisition timing task.
And under the condition that the task parameters of the new data acquisition timing task are not verified, the controller center can return prompt information for prompting that the task parameters of the new data acquisition timing task are failed to be verified to the web interface.
Under the condition that the task parameters of the newly-built data acquisition timing task pass the verification, the control center can send the task parameters of the newly-built data acquisition timing task to a data acquisition timing task scheduling device (a coordinator of timing task scheduling service) in a broadcast mode through the RabbitMQ. Wherein RabbitMQ is an open source message queue middleware that implements Advanced Message Queue Protocol (AMQP) and provides a reliable messaging mechanism. RabbitMQ is developed by Rabbit Technologies Ltd and written in Erlang language, has the characteristics of cross-platform, high performance, expandability and the like, and is widely applied to distributed systems, micro-service architecture and asynchronous communication scenes.
The data acquisition timing task scheduling device can receive task parameters of newly-built data acquisition timing tasks.
It should be noted that, after the control center may send the task parameters of the new data acquisition timing task to the data acquisition timing task scheduling device in a broadcast manner through the rabitimq, the control center may also return a prompt message for prompting that the task parameters of the new data acquisition timing task are successfully checked to the web interface.
102, under the condition that the number of the existing data acquisition timing tasks in execution does not exceed a first preset value, establishing a Redis lock corresponding to the new data acquisition timing task by taking the identification information of the new data acquisition timing task as a key value;
as shown in fig. 2, after the data acquisition timing task scheduling device receives the task parameters of the newly-built data acquisition timing task, it may be determined whether the number of the existing data acquisition timing tasks in execution exceeds the first preset value.
It should be noted that, in the embodiment of the present invention, the first preset value may be determined based on priori knowledge and/or actual conditions, and the value of the first preset value in the embodiment of the present invention is not specifically limited.
Under the condition that the number of the existing data acquisition timing tasks in execution exceeds a first preset value, the fact that the data acquisition timing tasks in execution reach the upper limit of computing resources at the moment is indicated, the data acquisition timing tasks cannot be newly added, and the data acquisition positioning tasks are newly added and fail.
Under the condition that the number of the existing data acquisition timing tasks in execution does not exceed a first preset value, the data acquisition timing tasks can be newly added, the identification information of the newly-built data acquisition timing tasks can be used as a Key value (Key value), and a Redis lock corresponding to the newly-built data acquisition timing tasks is created.
It should be noted that, the Redis lock is a distributed lock mechanism implemented based on a Redis database, and the Redis lock may be used to implement mutual exclusive access to a shared resource between multiple applications or threads, for example, guarantee an exclusive operation on a certain resource in a distributed environment, such as distributed task scheduling, cache update, and so on.
And step 103, under the condition that the Redis lock corresponding to the newly-built data acquisition timing task is successfully established, starting the execution of the newly-built data acquisition positioning task.
As shown in fig. 2, when it is determined that the creation of the dis lock corresponding to the new data acquisition timing task is unsuccessful, it is indicated that the new data acquisition timing task is loaded by other timing task scheduling services, and the data acquisition positioning task newly fails.
And under the condition that the Redis lock corresponding to the newly-built data acquisition timing task is successfully established, starting the execution of the newly-added data acquisition positioning task.
As an alternative embodiment, initiating execution of the new data acquisition positioning task includes: based on task parameters of the new data acquisition timing task, generating a corn expression corresponding to the new data acquisition timing task, and creating a task thread corresponding to the new data acquisition timing task by using a scheduledFutureinterface and a thread task scheduler;
it should be noted that the Corn expression (also called a Quartz expression) is a time expression for specifying when to execute a task or trigger event. The Corn expression may consist of 6 fields, each field representing a portion of time, including seconds, minutes, hours, days, months, weeks. The syntax of the Corn expression is very flexible and can meet various complex time requirements, such as execution at certain time points on a day of the week, at intervals, etc. Corn expressions are widely used in the context of task scheduling and timing trigger events.
It should be noted that schedule future is an interface in Java for representing a deferred task or a timed task. The scheduledFutureinterface is an extension of the scheduledExecutorService interface definition to return results. In the case of a scheduledFutureinterface for timing tasks, the timing execution task may be created by the scheduleAtFixedRate or scheduleWithFixedDelay method in scheduledExecutorService, again returning a scheduledFutureobject. The schedule future interface may be used to view the status of the task, the results, and cancel execution of the task.
The threadpoolmask scheduler is a task scheduler in the Spring Framework for executing asynchronous or timed tasks in the background thread pool. The threadpoolatskscheduler scheduler is an implementation class of the TaskScheduler interface. The ThreadPoolTaskScheduler scheduler may provide asynchronous execution, timed tasks, thread pool management, etc.
In the embodiment of the invention, a task thread corresponding to the newly built data acquisition timing task can be created by using a schedule future interface and a threadPoolTaskscheduler scheduler.
And adding the task thread corresponding to the new data acquisition timing task into the target thread pool so that the target thread pool can execute the new data acquisition positioning task at regular time based on the turn expression corresponding to the new data acquisition timing task.
It should be noted that, in the embodiment of the present invention, the target thread pool may be a Runnable thread pool.
The Java provides Executos class to create and manage thread pools, and different types of thread pools can be created through the Executos class, wherein one of the thread pools is a Runneable thread pool. The thread pool can be used for effectively managing the life cycle of a plurality of threads, reusing the thread resources, controlling the number of concurrently executed threads and avoiding the exhaustion of system resources caused by excessive threads. A Runneable thread pool is a thread pool used to perform Runneable tasks. Runnable is an interface in Java that represents tasks that can be executed by threads, and has only one run () method for defining the specific logic of the task.
It should be noted that, when judging whether the number of the tasks in the existing data acquisition timing in execution exceeds the first preset value, it may be judged whether the number of task threads in the Runnable thread pool exceeds the first preset value.
According to the embodiment of the invention, under the condition that the starting completion of the timing task scheduling service is determined, if the task parameters of the new data acquisition positioning task are acquired, whether the number of the existing data acquisition timing tasks in execution exceeds a first preset value is judged, under the condition that the number of the existing data acquisition timing tasks in execution does not exceed the first preset value, the identification information of the new data acquisition timing tasks is used as a key value, the Redis lock corresponding to the new data acquisition timing tasks is created, and under the condition that the Redis lock corresponding to the new data acquisition timing tasks is determined to be successfully created, the execution of the new data acquisition positioning task is started, the new data acquisition timing tasks can be added more simply and more efficiently, the execution period of the new data acquisition timing tasks can be more flexibly configured, the track traffic operation data can be more flexibly and dynamically acquired, and the acquisition requirements of the track traffic operation data can be better met.
The embodiment of the invention can better solve the scheduling problem of the distributed data acquisition timing problem of the rail transit operation data acquisition by combining the characteristics of the Redis lock with the schedule future interface in Jave.
Based on the content of the above embodiments, specific steps for starting the task scheduling service include: loading task parameters of all unexecuted existing data acquisition timing tasks;
fig. 3 is a schematic flow chart of starting task scheduling service in the data acquisition timing task scheduling method provided by the invention. As shown in fig. 3, when the task scheduling service is started, a service signal of the task scheduling service may be registered on the open source platform nano so that the task scheduling service can communicate with other micro services.
It should be noted that, the Nacos is an open-source distributed service discovery, configuration and management platform, which provides core functions such as a registry, a configuration center and a namespace, and is helpful for quickly constructing a micro-service architecture.
Specifically, after registering the service signal of the task scheduling service on the open source platform nano, the data acquisition timing task scheduling device (coordinator of the timing task scheduling service) may be initialized.
After initializing the data acquisition timing task scheduling device, the data acquisition timing task scheduling device may load task parameters of all the unexecuted existing data acquisition timing tasks from the task_info table.
The data acquisition timing task scheduling device loads task parameters of all unexecuted existing data acquisition timing tasks, and can initialize the task parameters of each unexecuted existing data acquisition timing task on the server.
Judging whether the number of the executed existing data acquisition timing tasks exceeds a second preset value, and under the condition that the number of the executed existing data acquisition timing tasks does not exceed the second preset value, establishing a Redis lock corresponding to any unexecuted existing data acquisition timing task by taking identification information of any unexecuted existing data acquisition timing task as a key value;
specifically, the data acquisition timing task scheduling device loads task parameters of all the unexecuted existing data acquisition timing tasks, and can also judge whether the number of the executed existing data acquisition timing tasks exceeds a second preset value.
It should be noted that, in the embodiment of the present invention, the second preset value may be determined based on priori knowledge and/or actual conditions, and the value of the second preset value in the embodiment of the present invention is not specifically limited.
It is understood that the first preset value and the second preset value in the embodiments of the present invention may be the same or different.
It should be noted that, when judging whether the number of the tasks in the existing data acquisition timing in execution exceeds the second preset value, it may be judged whether the number of task threads in the Runnable thread pool exceeds the second preset value.
And under the condition that the number of the existing data acquisition timing tasks in execution exceeds a second preset value, the data acquisition timing tasks in execution at the moment reach the upper limit of the computing resources, and the completion of the starting of the task scheduling service can be confirmed.
Under the condition that the number of the executed existing data acquisition timing tasks does not exceed a second preset value, the identification information of any one of the non-started existing data acquisition positioning tasks can be used as a Key value (Key value), and a Redis lock corresponding to any one of the non-executed existing data acquisition timing tasks can be created.
It should be noted that, in the embodiment of the present invention, when the number of existing data acquisition timing tasks in execution does not exceed the second preset value, the identification information of a certain non-started existing data acquisition positioning task may be sequentially set as a Key value (Key value), so as to create a dis lock corresponding to the certain non-executed existing data acquisition timing task.
Under the condition that the Redis lock corresponding to any unexecuted existing data acquisition timing task is successfully established, starting the execution of any unexecuted existing data acquisition timing task;
specifically, when it is determined that the Redis lock corresponding to the any one of the non-executed existing data acquisition timing tasks is successfully created, a corresponding corn expression of the any one of the non-executed existing data acquisition timing tasks may be generated based on task parameters of the any one of the non-executed existing data acquisition timing tasks, and a task thread corresponding to the any one of the non-executed existing data acquisition timing tasks may be created by using a schedule function interface and a thread task scheduler, so that the task thread corresponding to the any one of the non-executed existing data acquisition timing tasks may be added to the run thread pool, and the target thread pool may periodically execute the any one of the non-executed existing data acquisition timing tasks based on the corresponding corn expression of the any one of the non-executed existing data acquisition timing tasks, thereby starting execution of the any one of the non-executed existing data acquisition timing tasks.
As an optional embodiment, in a case that the number of the executed existing data acquisition timing tasks does not exceed the second preset value, using identification information of any unexecuted existing data acquisition timing task as a key value, after creating a dis lock corresponding to any unexecuted existing data acquisition timing task, the method further includes: and under the condition that the Redis lock corresponding to any unexecuted existing data acquisition timing task is not successfully established, eliminating any unexecuted existing data acquisition timing task from all unexecuted existing data acquisition timing tasks, and returning to the step of judging whether the number of the unexecuted existing data acquisition timing tasks exceeds a second preset value or not until the Redis lock corresponding to any unexecuted existing data acquisition timing task is successfully established.
Specifically, under the condition that the Redis lock creation corresponding to any one of the unexecuted existing data acquisition timing tasks is determined to be unsuccessful, it is indicated that any one of the unexecuted existing data acquisition timing tasks is loaded by other timing task scheduling services, any one of the unexecuted existing data acquisition timing tasks can be removed from all the unexecuted existing data acquisition timing tasks, and the step of judging whether the number of the unexecuted existing data acquisition timing tasks exceeds a second preset value is returned until the Redis lock creation corresponding to any one of the unexecuted existing data acquisition positioning tasks is determined to be successful.
And returning to the step of judging whether the number of the executed existing data acquisition timing tasks exceeds a second preset value or not, until the number of the executed existing data acquisition timing tasks exceeds the second preset value, and confirming that the starting of the timing task scheduling service is completed.
Specifically, after the execution of any one of the non-executed existing data acquisition timing tasks is started, any one of the non-executed existing data acquisition timing tasks is changed into the executing existing data acquisition timing task. Accordingly, the number of existing data acquisition timing tasks in execution increases by 1.
As shown in fig. 3, after the execution of any of the non-executed existing data acquisition timing tasks is started, a step of determining whether the number of the executing existing data acquisition timing tasks exceeds a second preset value may be returned until the number of the executing existing data acquisition timing tasks exceeds the second preset value, and completion of the startup of the timing task scheduling service is confirmed.
According to the embodiment of the invention, whether the number of the executed existing data acquisition timing tasks exceeds the second preset value is judged, when the number of the executed existing data acquisition timing tasks does not exceed the second preset value, identification information of any unexecuted existing data acquisition timing task is used as a key value, a Redis lock corresponding to any unexecuted existing data acquisition timing task is created, when the Redis lock corresponding to any unexecuted existing data acquisition timing task is determined to be successfully created, after the execution of any unexecuted existing data acquisition timing task is started, the step of judging whether the number of the executed existing data acquisition timing tasks exceeds the second preset value is returned until the number of the executed existing data acquisition timing tasks exceeds the second preset value, the completion of the start of a timing task scheduling service is confirmed, the execution of the existing data acquisition timing task can be started more timely and more efficiently, calculation resources can be more fully utilized, and the repeated execution or execution of the existing data acquisition timing tasks can be more effectively avoided.
Based on the content of each embodiment, the method for scheduling the data acquisition timing task further comprises: under the condition that the timing task scheduling service is determined to be started to be completed, acquiring a modification instruction of a target data acquisition timing task, wherein the target data acquisition timing task is an existing data acquisition timing task in execution needing modification, and the modification instruction carries task parameters of the modified data acquisition timing task;
specifically, in the embodiment of the invention, the existing data acquisition timing task in execution which needs to be modified can be determined as the target data acquisition timing task.
Fig. 4 is a third flow chart of a method for scheduling data acquisition timing tasks according to the present invention. As shown in fig. 4, in the case that the user needs to modify the target data acquisition timing task, the user may input a modification instruction of the target data acquisition timing task carrying the task parameter of the modified data acquisition timing task through an operation on the web interface.
It should be noted that, the data processing and data forwarding process of the modification instruction of the target data acquisition timing task may refer to the data processing and data forwarding process of the task parameter of the newly built data acquisition timing task, which is not described in detail in the embodiment of the present invention.
The data acquisition timing task scheduling device may receive a modification instruction of the target data acquisition timing task.
Under the condition that a task thread corresponding to a target data acquisition timing task exists in a target thread pool, deleting the task thread corresponding to the target data acquisition timing task, and creating a Redis lock corresponding to the modified data acquisition timing task by taking the identification information of the modified data acquisition timing task as a key value;
specifically, after the data acquisition timing task scheduling device receives the modification instruction of the target data acquisition timing task, it can determine whether a task thread corresponding to the target data acquisition timing task exists in the Runneable thread pool.
And under the condition that the task thread corresponding to the target data acquisition timing task does not exist in the Runneable thread pool, the error occurs, and the target data acquisition positioning task is failed to be modified.
Under the condition that task threads corresponding to the target data acquisition timing task exist in the Runneable thread pool, the task threads corresponding to the target data acquisition timing task can be deleted from the Runneable thread pool, and a Redis lock corresponding to the modified data acquisition timing task is created by taking the identification information of the modified data acquisition timing task as a key value.
Under the condition that the Redis lock corresponding to the modified data acquisition timing task is successfully established, a corn expression corresponding to the modified data acquisition timing task is generated based on task parameters of the modified data acquisition timing task, a task thread corresponding to the modified data acquisition timing task is established by using a Schedulfurure interface and a ThreadPoolTaskScheduler scheduler, and further the task thread corresponding to the modified data acquisition timing task is added to a target thread pool so that the target thread pool can execute a new data acquisition positioning task at regular time based on the corn expression corresponding to the modified data acquisition timing task.
According to the embodiment of the invention, under the condition that the starting completion of the timing task scheduling service is determined, if the modification instruction of the target data acquisition positioning task is acquired, whether the task thread corresponding to the target data acquisition positioning task exists in the target thread pool is judged, under the condition that the task thread corresponding to the target data acquisition positioning task exists in the target thread pool, the task thread corresponding to the target data acquisition positioning task is deleted from the target thread pool, the modified data acquisition positioning task is used as a new data acquisition timing task, the execution of the modified data acquisition positioning task is started, the modification of the existing data acquisition timing task in execution can be completed under the condition that the timing task scheduling service is not required to be restarted, the dynamic configuration of the execution period of the existing data acquisition timing task in execution can be realized more flexibly, and the flexibility of dynamically acquiring the rail transit operation data can be further improved.
Based on the content of each embodiment, the method for scheduling the data acquisition timing task further comprises: under the condition that the completion of the starting of the timing task scheduling service is determined, receiving a target instruction of a target data acquisition timing task, wherein the target data acquisition timing task is an existing data acquisition timing task in execution, and the target instruction comprises any one of a deleting instruction, a checking instruction and a stopping instruction;
and executing a target instruction on the task thread corresponding to the target data acquisition timing task under the condition that the task thread corresponding to the target data acquisition timing task exists in the target thread pool.
Specifically, in the embodiment of the invention, the existing data acquisition timing task in execution which needs to be deleted, checked or stopped can be determined as the target data acquisition timing task.
As shown in fig. 4, in the case where the user needs to delete the target data collection timing task, the user may input a deletion instruction of the target data collection timing task through an operation on the web interface.
It should be noted that, the data processing and data forwarding process of the stop instruction of the target data acquisition timing task may refer to the data processing and data forwarding process of the task parameter of the newly built data acquisition timing task, which is not described in detail in the embodiment of the present invention.
The data acquisition timing task scheduling device may receive a deletion instruction of the target data acquisition timing task.
After receiving the deleting instruction of the target data acquisition timing task, the data acquisition timing task scheduling device can judge whether a task thread corresponding to the target data acquisition timing task exists in the Runneable thread pool.
And under the condition that the task thread corresponding to the target data acquisition timing task does not exist in the Runneable thread pool, the error occurs, and the deletion of the target data acquisition positioning task fails.
Under the condition that task threads corresponding to the target data acquisition timing tasks exist in the Runneable thread pool, the task threads corresponding to the target data acquisition timing tasks can be deleted from the Runneable thread pool.
It can be understood that after deleting the task thread corresponding to the target data acquisition timing task from the Runnable thread pool, the number of existing data acquisition timing tasks in execution is reduced by 1.
As shown in fig. 4, in a case where the user needs to view the target data collection timing task, the user may input a viewing instruction of the target data collection timing task through an operation on the web interface.
It should be noted that, the data processing and data forwarding process of the checking instruction of the target data acquisition timing task can refer to the data processing and data forwarding process of the task parameter of the newly built data acquisition timing task, which is not described in detail in the embodiment of the present invention.
The data acquisition timing task scheduling device can receive a viewing instruction of a target data acquisition timing task.
After receiving the checking instruction of the target data acquisition timing task, the data acquisition timing task scheduling device can judge whether a task thread corresponding to the target data acquisition timing task exists in the Runneable thread pool.
And under the condition that a task thread corresponding to the target data acquisition timing task does not exist in the Runneable thread pool, the error occurs, and the target data acquisition positioning task fails to check.
Under the condition that a task thread corresponding to the target data acquisition timing task exists in the Runneable thread pool, the data acquisition timing task scheduling device can perform information interaction with the control center, so that the control center loads task data of the target data acquisition timing task in the task_info table, and returns the task data of the target data acquisition timing task to the web interface for a user to view.
As shown in fig. 4, in the case where the user needs to stop the target data collection timing task, the user can input a stop instruction of the target data collection timing task through an operation on the web interface.
It should be noted that, the data processing and data forwarding process of the stop instruction of the target data acquisition timing task may refer to the data processing and data forwarding process of the task parameter of the newly built data acquisition timing task, which is not described in detail in the embodiment of the present invention.
The data acquisition timing task scheduling device may receive a stop instruction of the target data acquisition timing task.
After receiving the stop instruction of the target data acquisition timing task, the data acquisition timing task scheduling device can judge whether a task thread corresponding to the target data acquisition timing task exists in the Runneable thread pool.
And under the condition that the task thread corresponding to the target data acquisition timing task does not exist in the Runneable thread pool, the error occurs, and the target data acquisition positioning task fails to stop.
And stopping the task thread corresponding to the target data acquisition timing task under the condition that the task thread corresponding to the target data acquisition timing task exists in the Runneable thread pool.
It can be understood that after the task thread corresponding to the target data acquisition timing task is stopped from the Runnable thread pool, the number of existing data acquisition timing tasks in execution is reduced by 1.
Based on the content of each embodiment, the method for scheduling the data acquisition timing task further comprises: under the condition that the starting completion of the timing task scheduling service is determined, a manual operation instruction of a target data acquisition timing task is received, wherein the target data acquisition timing task is an existing data acquisition timing task in execution requiring manual operation;
And under the condition that the task thread corresponding to the target data acquisition timing task exists in the target thread pool, executing the target data acquisition timing task based on an execution instruction input by a user.
Specifically, in the embodiment of the invention, the existing data acquisition timing task in execution requiring manual operation can be determined as the target data acquisition timing task.
As shown in fig. 4, in the case where the user needs to manually operate the target data collection timing task, the user can input a manual operation instruction of the target data collection timing task through an operation on the web interface.
It should be noted that, the data processing and data forwarding process of the manual operation instruction of the target data acquisition timing task may refer to the data processing and data forwarding process of the task parameter of the newly built data acquisition timing task, which is not described in detail in the embodiment of the present invention.
The data acquisition timing task scheduling device can receive manual operation instructions of target data acquisition timing tasks.
After the data acquisition timing task scheduling device receives the manual operation instruction of the target data acquisition timing task, whether a task thread corresponding to the target data acquisition timing task exists in the Runneable thread pool or not can be judged.
And under the condition that a task thread corresponding to the target data acquisition timing task does not exist in the Runneable thread pool, the error occurs, and the manual operation of the target data acquisition positioning task fails.
Under the condition that task threads corresponding to the target data acquisition timing tasks exist in the Runneable thread pool, the target data acquisition timing tasks can be executed based on execution instructions input by a user.
Based on the content of each embodiment, the method for scheduling the data acquisition timing task further comprises: under the condition that the completion of the starting of the timing task scheduling service is determined, a starting execution instruction of a target data acquisition timing task is received, wherein the target data acquisition timing task is an unexecuted existing data acquisition timing task which needs to be executed;
under the condition that a task thread corresponding to a target data acquisition timing task exists in a target thread pool, establishing a Redis lock corresponding to the modified data acquisition timing task by taking identification information of the target data acquisition timing task as a key value;
under the condition that the Redis lock corresponding to the target data acquisition timing task is successfully established, a corn expression corresponding to the modified data acquisition timing task is generated based on task parameters of the target data acquisition timing task, a task thread corresponding to the target data acquisition timing task is established by using a Schedulfurure interface and a ThreadPoolTaskScheduler scheduler, and further the task thread corresponding to the target data acquisition timing task is added to a target thread pool so that the target thread pool can execute the target data acquisition timing task at fixed time based on the corn expression corresponding to the target data acquisition timing task.
Specifically, in the embodiment of the invention, the unexecuted existing data acquisition timing task which needs to be executed can be determined as the target data acquisition timing task.
Under the condition that a user needs to execute the target data acquisition timing task, the user can input an execution starting instruction of the target data acquisition timing task through operation on a web interface.
It should be noted that, the data processing and data forwarding process of the start execution instruction of the target data acquisition timing task may refer to the data processing and data forwarding process of the task parameter of the newly built data acquisition timing task, which is not described in detail in the embodiment of the present invention.
The data acquisition timing task scheduling device can receive a start execution instruction of a target data acquisition timing task.
After receiving the start execution instruction of the target data acquisition timing task, the data acquisition timing task scheduling device can judge whether a task thread corresponding to the target data acquisition timing task exists in the Runneable thread pool.
And under the condition that a task thread corresponding to the target data acquisition timing task does not exist in the Runneable thread pool, the occurrence of errors is indicated, and the target data acquisition positioning task is failed to start and execute.
In the case that a task thread corresponding to the target data acquisition timing task exists in the Runneable thread pool, specific steps for starting and executing the target data acquisition timing task are shown in fig. 2.
According to the embodiment of the invention, if the target instruction of the target data acquisition positioning task is acquired under the condition that the starting of the timing task scheduling service is determined to be completed, whether the task thread corresponding to the target data acquisition positioning task exists in the target thread pool is judged, and under the condition that the task thread corresponding to the target data acquisition positioning task exists in the target thread pool, the target instruction is executed for the task thread corresponding to the target data acquisition positioning task exists in the target thread pool, so that the existing data acquisition timing task in execution can be scheduled more flexibly, and the flexibility of dynamically acquiring the rail transit operation data can be further improved.
It should be noted that, during the execution of the timing task, corresponding logs are generated, and the execution conditions of the task are recorded, including failure, exception, success, and failure and exception reasons. The information can be stored in the database after the task is executed, so that the user can check the information conveniently.
When inquiring logs, the logs can be arranged in descending order by taking a single task as an aggregation condition, so that the logs of each task can be more clearly displayed, and a user can quickly master the running state of the task.
Fig. 5 is a schematic structural diagram of a data acquisition timing task scheduling device provided by the invention. The data acquisition timing task scheduling device provided by the invention is described below with reference to fig. 5, and the data acquisition timing task scheduling device described below and the data acquisition timing task scheduling method provided by the invention described above can be referred to correspondingly. As shown in fig. 5, the apparatus includes: a data acquisition module 501, a data processing module 502 and a task initiation module 503.
The data acquisition module 501 is configured to acquire task parameters of a new data acquisition timing task when it is determined that the timing task scheduling service is started, where the task parameters include identification information and execution cycle information of the task;
the data processing module 502 is configured to create a dis lock corresponding to the newly-built data acquisition timing task by using the identification information of the newly-built data acquisition timing task as a key value when the number of existing data acquisition timing tasks in execution does not exceed a first preset value;
the task starting module 503 is configured to start execution of the new data acquisition positioning task when it is determined that the Redis lock corresponding to the new data acquisition timing task is successfully created.
Specifically, the data acquisition module 501, the data processing module 502, and the task initiation module 503 are electrically connected.
According to the data acquisition timing task scheduling device, under the condition that the timing task scheduling service is determined to be started to be completed, if task parameters of a new data acquisition positioning task are acquired, whether the number of the existing data acquisition timing tasks in execution exceeds a first preset value is judged, under the condition that the number of the existing data acquisition timing tasks in execution does not exceed the first preset value, identification information of the new data acquisition timing tasks is used as a key value, redis locks corresponding to the new data acquisition timing tasks are created, and under the condition that the Redis locks corresponding to the new data acquisition timing tasks are determined to be successfully created, execution of the new data acquisition positioning task is started, the new data acquisition timing tasks can be added more simply and more efficiently, the execution period of the new data acquisition timing tasks can be configured more flexibly, rail traffic operation data can be acquired more flexibly, and the acquisition requirements of rail traffic operation data can be met better.
Fig. 6 illustrates a physical schematic diagram of an electronic device, as shown in fig. 6, which may include: processor 610, communication interface (Communications Interface) 620, memory 630, and communication bus 640, wherein processor 610, communication interface 620, and memory 630 communicate with each other via communication bus 640. The processor 610 may invoke logic instructions in the memory 630 to perform a data acquisition timing task scheduling method comprising: under the condition that the completion of the starting of the timing task scheduling service is determined, acquiring task parameters of a newly-built data acquisition timing task, wherein the task parameters comprise identification information and execution period information of the task; under the condition that the number of the existing data acquisition timing tasks in execution does not exceed a first preset value, establishing a Redis lock corresponding to the new data acquisition timing task by taking the identification information of the new data acquisition timing task as a key value; and under the condition that the Redis lock corresponding to the newly-built data acquisition timing task is successfully established, starting the execution of the newly-built data acquisition positioning task.
Further, the logic instructions in the memory 630 may be implemented in the form of software functional units and stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, where the computer program product includes a computer program, where the computer program can be stored on a non-transitory computer readable storage medium, and when the computer program is executed by a processor, the computer can execute the data acquisition timing task scheduling method provided by the above methods, and the method includes: under the condition that the completion of the starting of the timing task scheduling service is determined, acquiring task parameters of a newly-built data acquisition timing task, wherein the task parameters comprise identification information and execution period information of the task; under the condition that the number of the existing data acquisition timing tasks in execution does not exceed a first preset value, establishing a Redis lock corresponding to the new data acquisition timing task by taking the identification information of the new data acquisition timing task as a key value; and under the condition that the Redis lock corresponding to the newly-built data acquisition timing task is successfully established, starting the execution of the newly-built data acquisition positioning task.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the data acquisition timing task scheduling method provided by the above methods, the method comprising: under the condition that the completion of the starting of the timing task scheduling service is determined, acquiring task parameters of a newly-built data acquisition timing task, wherein the task parameters comprise identification information and execution period information of the task; under the condition that the number of the existing data acquisition timing tasks in execution does not exceed a first preset value, establishing a Redis lock corresponding to the new data acquisition timing task by taking the identification information of the new data acquisition timing task as a key value; and under the condition that the Redis lock corresponding to the newly-built data acquisition timing task is successfully established, starting the execution of the newly-built data acquisition positioning task.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. The method for scheduling the data acquisition timing task is characterized by comprising the following steps of:
under the condition that the completion of the starting of the timing task scheduling service is determined, acquiring task parameters of a newly-built data acquisition timing task, wherein the task parameters comprise identification information and execution period information of the task;
under the condition that the number of the existing data acquisition timing tasks in execution does not exceed a first preset value, establishing a Redis lock corresponding to the new data acquisition timing task by taking the identification information of the new data acquisition timing task as a key value;
and under the condition that the Redis lock corresponding to the newly-built data acquisition timing task is successfully established, starting the execution of the newly-built data acquisition positioning task.
2. The method of claim 1, wherein the specific step of initiating the task scheduling service comprises:
loading task parameters of all unexecuted existing data acquisition timing tasks;
judging whether the number of the executed existing data acquisition timing tasks exceeds a second preset value, and under the condition that the number of the executed existing data acquisition timing tasks does not exceed the second preset value, creating a Redis lock corresponding to any unexecuted existing data acquisition timing task by taking identification information of any unexecuted existing data acquisition timing task as a key value;
Under the condition that the Redis lock corresponding to any unexecuted existing data acquisition timing task is successfully established, starting the execution of any unexecuted existing data acquisition timing task;
and returning to the step of judging whether the number of the executed existing data acquisition timing tasks exceeds a second preset value, until the number of the executed existing data acquisition timing tasks exceeds the second preset value, and confirming that the timing task scheduling service is started to be completed.
3. The method for scheduling data acquisition timing tasks according to claim 1, wherein said initiating execution of the newly created data acquisition positioning task comprises:
generating a corresponding turn expression of the new data acquisition timing task based on task parameters of the new data acquisition timing task, and creating a task thread corresponding to the new data acquisition timing task by using a schedule interface and a threadPoolTaskscheduler;
and adding the task thread corresponding to the new data acquisition timing task to a target thread pool so that the target thread pool can execute the new data acquisition positioning task at regular time based on the corresponding corn expression of the new data acquisition timing task.
4. The method for scheduling data acquisition timing tasks as claimed in claim 3, further comprising:
under the condition that the completion of the starting of the timing task scheduling service is determined, acquiring a modification instruction of a target data acquisition timing task, wherein the target data acquisition timing task is an existing data acquisition timing task in execution needing modification, and the modification instruction carries task parameters of the modified data acquisition timing task;
deleting a task thread corresponding to the target data acquisition timing task under the condition that the task thread corresponding to the target data acquisition timing task exists in the target thread pool, and creating a Redis lock corresponding to the modified data acquisition timing task by taking the identification information of the modified data acquisition timing task as a key value;
under the condition that the Redis lock corresponding to the modified data acquisition timing task is successfully established, a corn expression corresponding to the modified data acquisition timing task is generated based on task parameters of the modified data acquisition timing task, a task thread corresponding to the modified data acquisition timing task is established by using a ScheduedFuture interface and a ThreadPoolTaskScheduler scheduler, and then the task thread corresponding to the modified data acquisition timing task is added to a target thread pool so that the target thread pool can execute the new data acquisition positioning task at regular time based on the corn expression corresponding to the modified data acquisition timing task.
5. The method for scheduling data acquisition timing tasks according to claim 2, wherein when the number of executed existing data acquisition timing tasks does not exceed the second preset value, using identification information of any unexecuted existing data acquisition timing task as a key value, after creating a dis lock corresponding to the any unexecuted existing data acquisition timing task, the method further comprises:
and under the condition that the Redis lock corresponding to any unexecuted existing data acquisition timing task is not successfully established, eliminating any unexecuted existing data acquisition timing task from all unexecuted existing data acquisition timing tasks, and returning to the step of judging whether the number of the unexecuted existing data acquisition timing tasks exceeds a second preset value or not until the Redis lock corresponding to any unexecuted existing data acquisition timing task is successfully established.
6. The method for scheduling data acquisition timing tasks as claimed in claim 3, further comprising:
under the condition that the timing task scheduling service is determined to be started to be completed, receiving a target instruction of a target data acquisition timing task, wherein the target data acquisition timing task is an existing data acquisition timing task in execution, and the target instruction comprises any one of a deleting instruction, a checking instruction and a stopping instruction;
And executing the target instruction on the task thread corresponding to the target data acquisition timing task under the condition that the target thread Chi Cun is determined to be the task thread corresponding to the target data acquisition timing task.
7. The method for scheduling data acquisition timing tasks as claimed in claim 3, further comprising:
under the condition that the starting completion of the timing task scheduling service is determined, receiving a manual operation instruction of a target data acquisition timing task, wherein the target data acquisition timing task is an existing data acquisition timing task in execution which needs manual operation;
and under the condition that the target thread Chi Cun is determined to be the task thread corresponding to the target data acquisition timing task, executing the target data acquisition timing task based on an execution instruction input by a user.
8. A data acquisition timing task scheduling device, comprising:
the data acquisition module is used for acquiring task parameters of the newly-built data acquisition timing task under the condition that the timing task scheduling service is determined to be started, wherein the task parameters comprise identification information and execution period information of the task;
the data processing module is used for creating a Redis lock corresponding to the new data acquisition timing task by taking the identification information of the new data acquisition timing task as a key value under the condition that the number of the existing data acquisition timing tasks in execution does not exceed a first preset value;
And the task starting module is used for starting the execution of the new data acquisition positioning task under the condition that the Redis lock corresponding to the new data acquisition timing task is successfully established.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the data acquisition timing task scheduling method of any one of claims 1 to 7 when the program is executed by the processor.
10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the data acquisition timing task scheduling method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311467047.4A CN117632409A (en) | 2023-11-06 | 2023-11-06 | Method and device for scheduling data acquisition timing tasks |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311467047.4A CN117632409A (en) | 2023-11-06 | 2023-11-06 | Method and device for scheduling data acquisition timing tasks |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117632409A true CN117632409A (en) | 2024-03-01 |
Family
ID=90015487
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311467047.4A Pending CN117632409A (en) | 2023-11-06 | 2023-11-06 | Method and device for scheduling data acquisition timing tasks |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117632409A (en) |
-
2023
- 2023-11-06 CN CN202311467047.4A patent/CN117632409A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6499986B2 (en) | Fault tolerant batch processing | |
| EP2972821B1 (en) | Application compatibility checking in a distributed computing environment | |
| CN110597531B (en) | Distributed module upgrading method and device and storage medium | |
| CN113569987A (en) | Model training method and device | |
| CA2904253C (en) | Computer system using in-service software upgrade | |
| CN108429787B (en) | Container deployment method and device, computer storage medium and terminal | |
| CN113609019B (en) | Pipelining method and device for cross-cluster deployment of container cloud application | |
| US20210224076A1 (en) | Techniques for managing dependencies of an orchestration service | |
| CN110611707A (en) | Task scheduling method and device | |
| CN112817710A (en) | Timed task processing method, timed task processing system, computer equipment and computer storage medium | |
| CN111884834A (en) | Zookeeper-based distributed system upgrading method and system and computer equipment | |
| CN112925648B (en) | Business strategy issuing method and device | |
| US9760364B2 (en) | Checks for software extensions | |
| CN114006815B (en) | Automatic deployment method and device for cloud platform nodes, nodes and storage medium | |
| US8135393B2 (en) | System and method for transactional application lifecycle management for mobile devices | |
| CN114787836A (en) | System and method for remotely executing one or more arbitrarily defined workflows | |
| CN111522623A (en) | Multi-process running system of modular software | |
| CN117632409A (en) | Method and device for scheduling data acquisition timing tasks | |
| CN110188008B (en) | Job scheduling master-slave switching method and device, computer equipment and storage medium | |
| CN115794416B (en) | Data hot switching method and device based on Storm and Nacos | |
| CN112486631B (en) | Method for constructing virtual machine mirror image | |
| CN115454450B (en) | Method and device for resource management of data job, electronic equipment and storage medium | |
| CN112825044B (en) | Task execution method, device and computer storage medium | |
| US11907364B2 (en) | Managing incompliances identified at instances of cloud applications | |
| CN116501477B (en) | Automatic data processing method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |