CN117453365A - Task scheduling method and distributed task scheduling system of Internet of things equipment - Google Patents

Task scheduling method and distributed task scheduling system of Internet of things equipment Download PDF

Info

Publication number
CN117453365A
CN117453365A CN202311529718.5A CN202311529718A CN117453365A CN 117453365 A CN117453365 A CN 117453365A CN 202311529718 A CN202311529718 A CN 202311529718A CN 117453365 A CN117453365 A CN 117453365A
Authority
CN
China
Prior art keywords
task
information
queue
lock
executed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311529718.5A
Other languages
Chinese (zh)
Inventor
何振勇
段小勇
廖润迪
陈超
方国才
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Qianhai Yanxiang Asia Pacific Electronic Equipment Technology Co ltd
Original Assignee
Shenzhen Qianhai Yanxiang Asia Pacific Electronic Equipment Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Qianhai Yanxiang Asia Pacific Electronic Equipment Technology Co ltd filed Critical Shenzhen Qianhai Yanxiang Asia Pacific Electronic Equipment Technology Co ltd
Priority to CN202311529718.5A priority Critical patent/CN117453365A/en
Publication of CN117453365A publication Critical patent/CN117453365A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G06F9/4887—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues involving deadlines, e.g. rate based, periodic
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication
    • G06F9/542—Event management; Broadcasting; Multicasting; Notifications
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication
    • G06F9/546—Message passing systems or structures, e.g. queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/52—Indexing scheme relating to G06F9/52
    • G06F2209/521—Atomic
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/54—Indexing scheme relating to G06F9/54
    • G06F2209/547—Messaging middleware
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2209/00—Indexing scheme relating to G06F9/00
    • G06F2209/54—Indexing scheme relating to G06F9/54
    • G06F2209/548—Queue

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Telephonic Communication Services (AREA)

Abstract

The application provides a task scheduling method and a distributed task scheduling system of Internet of things equipment, wherein the method comprises the following steps: the kafka cluster module receives and stores task information to be scheduled, which carries task execution time information, from an internet of things platform; the kafka consumer instance acquires task information to be scheduled from the kafka cluster module and sends the task information to the task scheduler instance; the task scheduler instance generates task information to be executed according to the task execution time information, and adds the task information to be executed to a lock-free task queue through a task enqueue interface; the task notification example obtains the task information to be executed from the lock-free task queue through the task dequeue interface and sends the task information to at least one corresponding target internet of things device, so that each target internet of things device executes the task according to the task information to be executed. The task scheduling method and the distributed task scheduling system of the Internet of things equipment can improve timeliness of task distribution and task scheduling efficiency.

Description

Task scheduling method and distributed task scheduling system of Internet of things equipment
Technical Field
The application relates to the technical field of the internet of things, in particular to a task scheduling method and a distributed task scheduling system of internet of things equipment.
Background
Under the background of continuous development of the internet of things technology, the types of internet of things equipment are continuously increased, and the equipment comprises various types of equipment such as industry, intelligent home, water conservancy, electric power and the like, and the types are various. In the internet of things, reading status data periodically from an internet of things device is an indispensable operation.
The existing implementation mode is generally a mode of task scheduling, the internet of things equipment reports data according to tasks, and the task distribution is generally realized by adopting a locking mechanism in the existing scheme for effectively coordinating the scheduling of the tasks in the face of a large number of data reporting tasks corresponding to a large number of internet of things equipment. However, when a lock mechanism is adopted to implement task distribution scheduling, performance problems such as lock contention and database operation delay may exist, so that task scheduling efficiency is low, and the method is not suitable for situations where a large number of tasks need to be scheduled. Therefore, a task scheduling scheme of the internet of things equipment is needed, and aims to coordinate a large number of tasks to be scheduled, improve timeliness of task distribution and task scheduling efficiency, and improve coordination and order of scheduling of the large number of tasks.
Disclosure of Invention
The embodiment of the application provides a task scheduling method and a distributed task scheduling system of Internet of things equipment, which can coordinate a large number of tasks to be scheduled, improve timeliness of task distribution and task scheduling efficiency, and improve coordination and order of scheduling of the large number of tasks.
In a first aspect, an embodiment of the present application provides a task scheduling method of an internet of things device, which is applied to a distributed task scheduling system, where the distributed task scheduling system includes: a plurality of task scheduler instances, a lock-free task queue, a kafka cluster module, a plurality of kafka consumer instances, and a plurality of task notification instances; the task enqueuing interface corresponding to the lock-free task queue adopts comparison aiming at a tail pointer and exchange atomic operation to carry out enqueuing control of the task; the task dequeue interface corresponding to the lock-free task queue adopts comparison for head pointers and exchange atomic operation to perform dequeue control of tasks, and the method comprises the following steps:
the kafka cluster module receives and stores task information to be scheduled from an internet of things platform; the task to be scheduled carries task execution time information;
the kafka consumer instance acquires the task information to be scheduled from the kafka cluster module and sends the task information to be scheduled to the task scheduler instance;
the task scheduler instance generates task information to be executed corresponding to the task information to be scheduled according to the task execution time information, and adds the task information to be executed to the lock-free task queue through the task enqueue interface;
The task notification instance obtains the task information to be executed from the lock-free task queue through the task dequeue interface, and sends the task information to be executed to at least one corresponding target internet of things device, so that each target internet of things device executes the task according to the task information to be executed.
In a second aspect, embodiments of the present application provide a distributed task scheduling system including a plurality of task scheduler instances, a lock-free task queue, a kafka cluster module, a plurality of kafka consumer instances, and a plurality of task notification instances; the task enqueuing interface corresponding to the lock-free task queue adopts comparison aiming at a tail pointer and exchange atomic operation to carry out enqueuing control of the task; the task dequeue interface corresponding to the lock-free task queue adopts comparison aiming at a head pointer and exchange atomic operation to perform dequeue control of the task;
the kafka cluster module is used for: receiving and storing task information to be scheduled from an internet of things platform; the task to be scheduled carries task execution time information;
the kafka consumer example is for: acquiring the task information to be scheduled from the kafka cluster module, and sending the task information to be scheduled to the task scheduler instance;
The task scheduler instance is to: generating task information to be executed corresponding to the task information to be scheduled according to the task execution time information, and adding the task information to be executed to the lock-free task queue through the task enqueue interface;
the task notification instance is configured to: and acquiring the task information to be executed from the lock-free task queue through the task dequeue interface, and sending the task information to be executed to at least one corresponding target internet of things device so that each target internet of things device executes the task according to the task information to be executed.
The embodiment of the application provides a task scheduling method and a distributed task scheduling system of Internet of things equipment, wherein the method is applied to the distributed task scheduling system, and the distributed task scheduling system comprises the following steps: a plurality of task scheduler instances, a lock-free task queue, a kafka cluster module, a plurality of kafka consumer instances, and a task notification instance; the task enqueuing interface corresponding to the lock-free task queue adopts comparison aiming at the tail pointer and exchange atomic operation to carry out enqueuing control of the task; the dequeue control method of the task is characterized in that dequeue control of the task is carried out by a task dequeue interface corresponding to the lock-free task queue through comparison of head pointers and exchange of atomic operation, and the method comprises the following steps: the kafka cluster module receives and stores task information to be scheduled from the internet of things platform; the task to be scheduled carries task execution time information; the kafka consumer instance acquires task information to be scheduled from the kafka cluster module and sends the task information to be scheduled to the task scheduler instance; the task scheduler instance generates task information to be executed corresponding to the task information to be scheduled according to the task execution time information, and adds the task information to be executed to a lock-free task queue through a task enqueue interface; the task notification example obtains task information to be executed from the lock-free task queue through the task dequeue interface, and sends the task information to be executed to at least one corresponding target internet of things device, so that each target internet of things device executes the task according to the task information to be executed. In the application, a kafka cluster module is adopted as a message middleware to accept task traffic generated by the Internet of things platform, traffic peak clipping is realized, and a plurality of kafka consumer examples are used for acquiring task information to be scheduled from the kafka cluster module and giving the task information to a task scheduler example to carry out actual task scheduling; for actual task scheduling, the method adopts a plurality of task scheduler instances to be responsible for task scheduling according to task execution time information, adds tasks to be executed to the lock-free task queue, and uses the plurality of task notification instances as consumers of the lock-free task queue to acquire the tasks to be executed through task dequeue interfaces corresponding to the lock-free task queue and send the tasks to at least one corresponding target internet of things device, so that the task distribution efficiency is improved; for the task queue, the task enqueuing interface and the task dequeuing interface corresponding to the task queue without locking are used for enqueuing control and task dequeuing control of corresponding tasks by comparing and exchanging atomic operations, on one hand, the task queue without locking can reduce the acquisition and adding time of the tasks, which is beneficial to timely distributing the tasks to the Internet of things equipment, and on the other hand, the task queue control and dequeuing control by comparing and exchanging atomic operations can avoid competition caused when a plurality of task notification examples acquire the tasks and a plurality of task scheduler examples add the tasks, so that the ordered distribution of the tasks under a plurality of examples is realized. In summary, the task scheduling method and the distributed task scheduling system of the internet of things equipment provided by the application can coordinate a large number of tasks to be scheduled, improve timeliness of task distribution and task scheduling efficiency, and improve coordination and order of scheduling of a large number of tasks.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is an application scenario schematic diagram of a task scheduling method of an internet of things device according to an embodiment of the present application;
fig. 2 is a flow chart of a task scheduling method of an internet of things device according to an embodiment of the present application;
FIG. 3 is a schematic processing logic diagram of a task enqueuing interface corresponding to a lock-free task queue according to an embodiment of the present application;
FIG. 4 is a schematic processing logic diagram of a task dequeue interface corresponding to a lock-free task queue according to an embodiment of the present disclosure;
fig. 5 is a schematic block diagram of a distributed task scheduling system provided in an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
It should be understood that the terms "comprises" and "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes such combinations.
Under the background of continuous development of the internet of things technology, the types of internet of things equipment are continuously increased, and the equipment comprises various types of equipment such as industry, intelligent home, water conservancy, electric power and the like, and the types are various. In the internet of things, reading status data periodically from an internet of things device is an indispensable operation.
The existing implementation mode is generally a mode of task scheduling, the internet of things equipment reports data according to tasks, and the task distribution is generally realized by adopting a locking mechanism in the existing scheme for effectively coordinating the scheduling of the tasks in the face of a large number of data reporting tasks corresponding to a large number of internet of things equipment. However, when a lock mechanism is used to implement task distribution scheduling, performance issues such as lock contention, database operation delay, and database loading may exist.
Lock contention: a lock contention situation may occur when multiple instances of an acquisition task attempt to acquire a lock at the same time. If the lock is competitive, it may cause some instances to wait for the lock to be released for a long time, reducing the efficiency of task scheduling.
Delay problem: the acquisition and release of locks can introduce delays in database operations. The acquisition and release of the lock require additional time, and when the acquisition and release operation time of the lock is long under the condition of frequent task scheduling operation, the time delay of task scheduling is greatly increased, and the timely execution of the task is affected.
The task scheduling efficiency is low, and the method is especially not suitable for the situation that a large number of tasks need to be scheduled. Therefore, a task scheduling scheme of the internet of things equipment is needed, and aims to improve task scheduling efficiency, coordinate a large number of tasks to be scheduled and improve coordination and order of scheduling of the large number of tasks.
Referring to fig. 1, fig. 1 is an application scenario schematic diagram of a task scheduling method of an internet of things device according to an embodiment of the present application. The distributed task scheduling system includes a plurality of task scheduler instances, a lock-free task queue, a kafka cluster module, a plurality of kafka consumer instances, and a plurality of task notification instances. As shown in fig. 1, the internet of things platform is in communication connection with the kafka cluster, a user submits task information to be scheduled (for example, the current temperature data is uploaded by the device a every 1 hour) to the kafka cluster module at the internet of things platform, the task information to be scheduled is acquired from the kafka cluster module by the kafka consumer instance and submitted to the task scheduler instance, and the task scheduler instance is responsible for actual task scheduling. Specifically, the task scheduler instance generates task information to be executed (for example, uploads current temperature data information) corresponding to the task information to be scheduled every 1 hour, and adds the task information to the lock-free task queue through the task enqueuing interface, and the task notification instance is responsible for acquiring the task information to be executed from the lock-free task queue through the task dequeuing interface and sending the task information to the internet of things equipment A which needs to execute the task information to be executed.
Fig. 2 is a flow chart of a task scheduling method of an internet of things device according to an embodiment of the present application, where the method is applied to a distributed task scheduling system, and the distributed task scheduling system includes: the task scheduling method of the Internet of things device specifically comprises the following steps S110 to S190.
Step S110, a kafka cluster module receives and stores task information to be scheduled from an Internet of things platform; the task to be scheduled carries task execution time information.
The task information to be scheduled is task information to be scheduled and executed; specifically, the task information to be scheduled may specifically include task content, task execution time information corresponding to the task content, and a task execution object, where the task content indicates a task to be executed, the task execution time information indicates a time corresponding to the task content to be executed, and the task execution object indicates a target internet of things device that needs to execute the task content. In the implementation, if the task needs to be executed at regular time, the task execution time information may be specifically a Cron expression.
In this embodiment, the task information to be scheduled may be submitted by the user on the platform of the internet of things to reach the kafka cluster module, where the kafka cluster module is used as an middleware for storing the task information to be scheduled.
In this embodiment, because the number of the devices of the internet of things is huge, the number of tasks of the devices of the internet of things is also huge, and in order to be able to accept the task flow from the platform of the internet of things, the kafak cluster is used as a message middleware, so that peak clipping is realized, and system breakdown caused by simultaneous generation of a large number of tasks is avoided. Specifically, the kafka cluster module includes a plurality of kafka instances (i.e., broaders), each of which is used for processing read and write requests of task information to be scheduled, and storing the task information to be scheduled, so as to process a large number of message streams. The kafka instances run on servers, each running one or more kafka instances, and the servers may be distributed in different locations.
Step S120, the kafka consumer instance acquires task information to be scheduled from the kafka cluster module, writes the task information to be scheduled into the task information storage module, and sends the task information to be scheduled to the task scheduler instance.
Wherein the kafka consumer instance is used for consuming the message-task information to be scheduled stored in the kafka cluster, and one device can run one or more kafka consumer instances when the kafka consumer instance is implemented. In this embodiment, the distributed task scheduling system includes a plurality of kafka consumer instances, where each kafka consumer instance may be independently used to obtain task information to be scheduled from the kafka cluster module and is responsible for writing the task information to be scheduled into the task information storage module, and sending the task information to be scheduled to the task scheduler instance.
The task information storage module is used for storing data related to tasks of the Internet of things equipment in a lasting mode, and is convenient for managing and analyzing the tasks of the Internet of things equipment. After the kafka consumer example acquires the task to be scheduled, the task to be scheduled is stored in the task information storage module, and the task to be scheduled is sent to the task scheduler example to carry out specific task scheduling.
Step S130, the task scheduler instance generates task information to be executed corresponding to the task information to be scheduled according to the task execution time information, writes a task scheduling record corresponding to the task information to be executed into a task information storage module, associates the task scheduling record with the corresponding task information to be scheduled, and adds the task information to be executed to a lock-free task queue through a task enqueue interface.
The to-be-executed content indicates task content to be executed by the internet of things device. In this embodiment, the number of the task scheduler instances is multiple, and each task scheduler instance is configured to generate corresponding task information to be executed according to the task information to be scheduled, write a task scheduling record corresponding to the task information to be executed into the task information storage module, associate the task scheduling record with the corresponding task information to be scheduled, and add the task to be executed to the lock-free task queue. In particular implementations, one or more of the task scheduler instances described above may be running on a device.
The task scheduling record is record information corresponding to the task information to be executed, and may include a generation time of the task to be executed, an identification information of a task scheduler instance of the task to be executed, an execution object corresponding to the task to be executed, and a task execution result corresponding to the task to be reported.
Specifically, the task information to be scheduled is already stored in the task information storage module by the kafka consumer, after which the task scheduler instance generates task information to be executed according to task execution time information in the task information to be scheduled, and stores the task information to be executed in the task information storage module and associates the task information to be scheduled with the corresponding task information to be scheduled.
In a specific implementation, the task scheduler instance may include a Cron expression analysis module, configured to analyze a Cron expression in task execution time information, implement timing triggering, and generate task information to be executed corresponding to task information to be scheduled.
In this embodiment, after the task information to be executed is generated, the task information is submitted to a lock-free task queue, and the lock-free task queue distributes tasks in order based on a first-in first-out principle. Specifically, in the lock-free task queue, the lock operation is not added to the task acquisition operation and the task adding operation, and the task acquisition operation and the task adding operation are allowed to be performed only after the lock is strived for. Therefore, in the embodiment, when the task scheduler instances add the task information to be executed to the lock-free task queue, the locks do not need to be obtained first, so that lock competition and extra time expenditure in the lock distribution process are avoided, and the enqueuing speed of the task information to be executed is improved.
In the application, in order to avoid that a plurality of task scheduler instances write the task information to be executed into the same storage position of the lock-free task queue at the same time and improve the order of task enqueuing, the task enqueuing interface corresponding to the lock-free task queue adopts comparison aiming at a tail pointer and exchange atomic operation to carry out the enqueuing control of the task. Specifically, the compare-and-swap atomic operation is an atomic operation including a comparison operation and a swap operation, and can be executed certainly when the comparison operation satisfies a condition. In a queue, a head pointer usually points to the first element of the queue, a tail pointer points to the next element to be inserted in the queue, when an element is enqueued, the element is put in the tail pointer position and the tail pointer is modified to point to the next position, so that the tail pointer needs to be correctly modified to ensure the ordered storage of the queue. In the lock-free task queue provided by the application, the queue control is performed by adopting the comparison and exchange atomic operation aiming at the tail pointer in the task queue interface for task queue, so that when each piece of task information to be executed is enqueued, the tail pointer can be normally modified, the orderly queue control is realized, the task coverage problem caused by that a plurality of tasks are simultaneously placed at one position when the tail pointer is abnormally modified is avoided, and the orderly task queue control under multiple instances is realized.
The task enqueuing interface may specifically refer to a task enqueuing method corresponding to a lock-free task queue, the task scheduler instance may implement addition of task information to be executed by calling the task enqueuing method corresponding to the lock-free task queue, the enqueuing of the task enqueuing method may at least include the task information to be executed, and the task enqueuing method specifically performs addition of the task information to be executed to the lock-free task queue.
Specifically, in an embodiment, the lock-free task queue is a circular queue, and at this time, as shown in fig. 3, the processing steps corresponding to the task enqueue interface corresponding to the lock-free task queue include the following steps S210 to S250.
Step S210, a current head pointer position value and a current tail pointer position value of the lock-free task queue are obtained.
Wherein the current head pointer position value (e.g., denoted as current_head) refers to the value of the head pointer (the pointed position) in the current lock-free task queue (when step S210 is executed), and is specifically the value of the head pointer variable; the above-mentioned current tail pointer position value (for example, denoted as current_tail) refers to the value of the tail pointer (the position pointed to) in the current lock-free task queue (when step S210 is performed), which is specifically the value of the tail pointer variable.
Step S220, the position value of the next tail pointer is obtained according to the position value of the current head pointer and the size of the lock-free task queue.
The size of the lock-free task queue may be specifically the total number of storage units of the lock-free task queue; the next tail pointer position value is a value of a tail pointer (pointed position) of the lock-free task queue after one element is enqueued, which is calculated according to the current head pointer position value and the size of the lock-free task queue. In the queue, the tail pointer always points to the position of the next inserted element, if a new element is to be inserted, the tail pointer needs to be modified to point to the next position, and the value of the position is the position value of the next tail pointer.
Specifically, in the circular queue, the calculation formula of the next tail pointer position value next_tail is:
next_tail= (current_tail+1)% size of lock-free task queue
Step S230, judging whether the position value of the next tail pointer is equal to the position value of the current head pointer, if not, executing the following steps S240 to S250, if yes, returning to execute the step S210.
In the circular queue, the head pointer points to the first element in the queue (i.e., the first element in the dequeue order), the tail pointer points to the next storage location in the queue, and in order to distinguish between a full queue and a empty queue, an empty storage location is typically set in the circular queue, thereby determining whether the circular queue, the lock-free task queue, is full by determining whether the next tail pointer location value is equal to the current head pointer location value.
In this embodiment, the next tail pointer position value is equal to the current head pointer position value, which indicates that the queue is full and the task to be executed cannot be inserted continuously, the above step S210 may be executed again, that is, the task enqueuing steps from step S210 to step S250 are executed again until the following step S250 can be executed, and writing of the task information to be executed is completed.
In some embodiments, after determining whether the next tail pointer position value is equal to the current head pointer position value in the above step S230, if the next tail pointer position value is equal to the current head pointer position value, a notification of "the queue is full and the operation is failed" may be returned to the task scheduler instance, which is the caller of the task enqueuing interface, and the task scheduler instance may then recall the task enqueuing interface to attempt to write the task information to be executed into the lock-free task queue.
Step S240, the current tail pointer is modified and pointed to the next tail pointer position value when the current tail pointer is the same as the current tail pointer position value through comparison and exchange atomic operation.
The current tail pointer refers to (the position pointed to by) the tail pointer of the current queue (when step S240 is executed), which is a pointer variable.
Specifically, in the above step, the comparing and exchanging atomic operations first compares whether the (value of the) current tail pointer is the same as the current tail pointer position value obtained when step S210 is performed, and if so, modifies the tail pointer of the current queue to point to the next tail pointer position value, i.e. makes the tail pointer of the queue point to the next storage location.
And step S250, when the modification of the current tail pointer is successful, writing the task information to be executed into a storage position corresponding to the current tail pointer position value in the lock-free task queue.
In this embodiment, on the basis of successful modification of the tail pointer, the task information to be executed is written into the storage location corresponding to the current tail pointer location value obtained in step S210 in the lock-free task queue.
In this embodiment, the task enqueuing interface adopts to modify the tail pointer first, and then adds the task information to be executed to the position before the tail pointer to realize the enqueuing control of the task, so as to ensure the successful modification of the tail pointer.
For example, according to the above steps S210 to S250, assuming that the task scheduler instance a has not yet had access to write the task information to be executed into the queue after performing the comparison and exchange atomic operation in the above step S240, the task scheduler instance B starts to call the above task enqueue interface, and the task scheduler instance B may also successfully perform the comparison and exchange atomic operation. In this case, the ordered enqueuing of the task information to be performed is not affected for the following reasons:
first, the task scheduler instance A performs a compare and swap atomic operation, successfully updating the current tail pointer tail from the current tail pointer position value current_tail to the next tail pointer position value next_tail.
Then, task scheduler instance B begins to invoke the task enqueue interface: at this time, the current tail pointer position value acquired by the task scheduler instance B is already next_tail, the calculated next tail pointer position value is next_tail+1, and an attempt is made to perform a compare and replace atomic operation. Since the current tail pointer acquired by the task scheduler instance B is already next_tail, it is actually an attempt to update the current tail pointer from next_tail to next_tail+1, which does not affect the task scheduler instance a to add the task information a to be executed to the position corresponding to the current tail pointer position value current_tail, and the task information B to be executed corresponding to the subsequent task scheduler instance B is also added to the next_tail position in the queue, and is not placed in the same position as the task information a to be executed.
Therefore, under the condition that the tail pointer can be successfully modified by adopting comparison and exchange atomic operation, the method and the device for processing the task information insert the task information to be executed into the storage position in front of the position of the tail pointer, so that the interference that other examples simultaneously add the task information to be executed into the lock-free task queue is avoided, task loss is avoided, and orderly enqueuing of tasks on the lock-free basis is realized.
To more clearly illustrate the above steps, exemplary, the brief implementation code of the task enqueue interface (e.g., enqueue) under python may be:
step S140, the task notification example obtains task information to be executed from the lock-free task queue through the task dequeue interface, and sends the task information to be executed to at least one corresponding target Internet of things device, so that each target Internet of things device executes the task according to the task information to be executed.
The target internet of things device is an internet of things device which needs to execute the task information to be executed, and the number of the target internet of things devices can be one or more.
In this embodiment, the number of the task notification instances is multiple, and each task notification instance is a consumer of the lock-free task queue, and is responsible for acquiring task information to be executed from the lock-free task queue, and notifying the corresponding target internet of things device.
The task dequeue interface is a task dequeue method corresponding to the lock-free task queue, and the task notification example can be to obtain the task information to be executed in the lock-free task queue by calling the task dequeue method corresponding to the lock-free task queue, and the task dequeue method specifically executes and takes out the task information to be executed in the lock-free task queue.
In this embodiment, in order to avoid that multiple task notifier instances simultaneously acquire task messages to be executed in the same storage location of the lock-free task queue, the task dequeue interface corresponding to the lock-free task queue performs dequeue control of tasks by comparing head pointers and exchanging atomic operations. Specifically, in the queue, the head pointer points to the first element of the queue, the tail pointer points to the next element to be inserted position of the queue, when the elements are dequeued, the elements are dequeued from the head of the queue, and the head pointer is modified to point to the next position, so that the head pointer needs to be correctly modified to ensure the orderly dequeuing of the storage content of the queue. In the above lock-free task queue, the queue-out interface for dequeuing task information to be executed uses comparison and exchange atomic operation to perform dequeuing control on the head pointer, so that when each task information to be executed is dequeued, the head pointer can be modified normally, ordered dequeuing control is realized, a plurality of different task notification instances are prevented from acquiring the same task information to be executed when the head pointer is modified abnormally, and ordered dequeuing control under multiple instances is realized.
Specifically, in an embodiment, the lock-free task queue is a circular queue, and at this time, as shown in fig. 4, the processing steps corresponding to the task dequeue interface corresponding to the lock-free task queue include the following steps S310 to S360.
Step S310, a current head pointer position value and a current tail pointer position value of the lock-free task queue are obtained.
The step S310 is the same as the step S210. Specifically, the current head pointer position value (for example, denoted as current_head) refers to the value of the head pointer (the position pointed to) in the current lock-free task queue (when step S310 is performed), which is specifically the value of the head pointer variable; the above-mentioned current tail pointer position value (for example, denoted as current_tail) refers to the value of the tail pointer (the position pointed to) in the current lock-free task queue (when step S310 is performed), which is specifically the value of the tail pointer variable.
Step S320, determining whether the current head pointer position value is equal to the current tail pointer position value, if not, executing the following steps S330 to S350, and if yes, executing the following step S360.
In the circular queue, the head pointer points to the first element in the queue (i.e., the first element in the dequeue order), the tail pointer points to the next storage location in the queue, and in order to distinguish between full and empty situations in the circular queue, an empty storage location is typically set in the circular queue, thereby determining whether the circular queue, i.e., the lock-free task queue, is empty by determining whether the current tail pointer location value is equal to the current head pointer location value.
Step S330, the next head pointer position value is obtained according to the current head pointer position value and the size of the lock-free task queue.
The size of the lock-free task queue may be specifically the total number of storage units of the lock-free task queue; the next head pointer position value is a value of a head pointer (pointed position) of the lock-free task queue after one element is dequeued according to the current head pointer position value and the current tail pointer position value. Since the head pointer always points to the position of the first element in the queue, if the element needs to be acquired from the queue, the head pointer needs to be modified to point to the next position, and the value of the position is the position value of the next head pointer.
Specifically, in the circular queue, the calculation formula of the next head pointer position value next_head is:
next_head= (current_head+1)% size of lock-free task queue
Step S340, modifying and pointing the current head pointer to the next head pointer position value when the current head pointer is the same as the current head pointer position value by comparing and exchanging atomic operations.
The current head pointer refers to (the position pointed to by) the head pointer of the current queue (when step S340 is performed), which is a pointer variable.
Specifically, in the step S340, the comparing and exchanging atomic operations first compare whether the (value of the) current head pointer is the same as the current head pointer position value obtained when the step S310 is performed, and if so, modify the head pointer of the current queue to point to the next head pointer position value, i.e. make the head pointer of the queue point to the next storage location.
And step 350, when the current head pointer is successfully modified, reading the task information to be executed in the storage position corresponding to the current head pointer position value in the lock-free task queue.
In this embodiment, on the basis of successful modification of the head pointer, the task information to be executed is read from the storage location corresponding to the current head pointer position value obtained in step S210 in the lock-free task queue.
In some embodiments, after step S350, the storage location corresponding to the current head pointer position value may be set to be empty, and the storage content of the storage location may be emptied.
In this embodiment, the task dequeue interface also uses to modify the head pointer first, and then obtains the task information to be executed in the storage position before the head pointer after the modification is successful, so as to ensure the successful modification of the head pointer.
For example, according to the steps S310 to S350, assuming that the task scheduler instance a has not yet obtained the task information to be executed from the lock-free task queue after executing the comparison and exchange atomic operation in the step S340, the task scheduler instance B starts to schedule the task dequeue interface, and the task scheduler instance B may also successfully execute the comparison and exchange atomic operation. In this case, the information of the task to be performed is not affected to be dequeued in order for the following reason:
first, task scheduler instance a performs a compare and swap atomic operation, successfully updating the current head pointer head from the current head pointer position value current_head to the next head pointer position value next_head.
Then, task scheduler instance B begins to invoke the task dequeue interface: it obtains the current head pointer position value that is already next_head, calculates the next head pointer position value that is next_head+1, and tries to perform the compare and replace atomic operation. Since the current head pointer acquired by the task scheduler instance B is already next_head, it actually tries to update the current head pointer from next_head to next_head+1, which does not affect the task scheduler instance a to acquire the task information to be executed from the position corresponding to the current_head in the lock-free task queue, and the task scheduler instance B subsequently acquires the task information to be executed from the position corresponding to the next_head in the lock-free task queue, so that the same task information to be executed is not acquired.
Step S360, the cast queue is empty and abnormal.
In this embodiment, if the current head pointer position value is equal to the current tail pointer position value, it indicates that the lock-free task queue is empty, and the cast-out queue is empty.
To more clearly illustrate the above steps, exemplary, the brief implementation code of the task enqueue interface (dequeue) under python may be:
therefore, in this embodiment, under the condition that the head pointer can be successfully modified by adopting the comparison and exchange atomic operation, the information of the task to be executed is read from the storage position before the position of the head pointer, so that the interference of the other examples in acquiring the information of the task to be executed is avoided, the repeated acquisition of the information of the task to be executed is avoided, and the ordered dequeuing of the tasks on the basis of no lock is realized.
In order to facilitate notification of the target internet of things device, in an embodiment, the step S140 may be specifically implemented by:
the task notification instance obtains task information to be executed from a lock-free task queue through a task dequeue interface and sends the task information to a target theme in a task notification module based on the MQTT; the target topics are preset topics corresponding to the task information to be executed;
and forwarding the task information to be executed to each target internet of things device subscribed to the target topics through the task notification module based on the MQTT, wherein each internet of things device is subscribed to one or more preset topics in the task notification module based on the MQTT.
Specifically, the preset topics may be divided and determined according to actual situations, each piece of internet of things equipment may subscribe to one or more preset topics in advance, and when the task notification instance sends task information to be executed to a certain preset topic in the task notification module based on MQTT (Message Queuing Telemetry Transport), the task notification module based on the MQTT forwards the task information to be executed to all pieces of internet of things equipment subscribed to the preset topic based on a subscription mechanism.
In this embodiment, when notifying the target internet of things device, the task notification instance is to send the task information to be executed to the target topic in the MQTT-based task notification module, and the MQTT-based task notification module forwards the task information to the internet of things device, i.e., the target internet of things device, subscribed to the target topic in the MQTT-based task notification module, so that compared with real-time communication, the notification of the task information to be executed is performed based on a message subscription manner, and the task notification efficiency of the internet of things device can be ensured, and meanwhile, the sender and the receiver of the message are decoupled, so that the bandwidth requirement on the system is reduced.
Step S150, the task notification example receives task execution result information reported by the target Internet of things equipment according to the task information to be executed, and sends the task execution result information to the kafka cluster module.
In this embodiment, after receiving the task information to be executed, the target internet of things device executes the task according to the task information to be executed, and sends the task execution result information to the task notification instance, where the task notification instance sends the task execution result information to the kafka cluster module.
In specific implementation, the target internet of things device can send the task execution result information to the corresponding result topic in the message notification module based on the MQTT, and the message notification module based on the MQTT forwards the task execution result information to the task notification instance subscribed to the result topic.
In specific implementation, a task topic and a task execution result topic can be set in the kafka cluster module, on the basis, the kafka consumer sends the task execution result information to the task execution result topic in the kafka cluster module, and task information to be scheduled generated by the internet of things platform can be sent to the task topic in the kafka cluster module.
Step S160, the kafka cluster module receives and stores task execution result information.
In this embodiment, the task execution result information is stored in the kafka cluster module to perform traffic clipping.
Step S170, the kafka consumer instance acquires task execution result information from the kafka cluster module and writes the task execution result information into a corresponding task scheduling record in the task information storage module.
Specifically, the task information storage module has stored a task scheduling record corresponding to task information to be executed written by the task scheduler instance, and for task execution result information corresponding to the task to be executed, the task scheduling record in the task information storage module is written by the kafka consumer instance.
Step S180, the kafka cluster module receives and stores deleting information of target task information to be scheduled from the Internet of things platform.
The deleting information of the target task information to be scheduled is used for indicating to delete the target task information to be scheduled, namely stopping starting the target task information to be scheduled. In specific implementation, a user can input a deleting instruction aiming at target task information to be scheduled in the sent task information to be scheduled in the internet of things platform, and the internet of things platform further sends deleting information aiming at the target task information to be scheduled to the kafka cluster module, wherein the kafka cluster module serves as a middleware to receive and store the deleting information of the task information to be scheduled.
Step S190, the kafka consumer instance acquires the deletion information from the kafka cluster module, deletes the target task information to be scheduled in the task information storage module according to the deletion information, and sends the stopping scheduling information for the target task information to be scheduled to the task scheduler instance for scheduling the target task information to be scheduled.
In this embodiment, the kafka consumer instance acquires deletion information from the kafka cluster module, after that, deletes target task information to be scheduled in the task information storage module, and sends stop scheduling information for the target task information to be scheduled to the task scheduler instance that schedules the target task information to be scheduled, and the task scheduler instance does not generate execution task information corresponding to the target task information to be scheduled any more, i.e. stops task scheduling of the target task information to be scheduled.
In summary, the embodiment of the application provides a task scheduling method and a distributed task scheduling system of an internet of things device, where the method is applied to the distributed task scheduling system, and the distributed task scheduling system includes: a plurality of task scheduler instances, a lock-free task queue, a kafka cluster module, a plurality of kafka consumer instances, and a task notification instance; the task enqueuing interface corresponding to the lock-free task queue adopts comparison aiming at the tail pointer and exchange atomic operation to carry out enqueuing control of the task; the dequeue control method of the task is characterized in that dequeue control of the task is carried out by a task dequeue interface corresponding to the lock-free task queue through comparison of head pointers and exchange of atomic operation, and the method comprises the following steps: the kafka cluster module receives and stores task information to be scheduled from the internet of things platform; the task to be scheduled carries task execution time information; the kafka consumer instance acquires task information to be scheduled from the kafka cluster module and sends the task information to be scheduled to the task scheduler instance; the task scheduler instance generates task information to be executed corresponding to the task information to be scheduled according to the task execution time information, and adds the task information to be executed to a lock-free task queue through a task enqueue interface; the task notification example obtains task information to be executed from the lock-free task queue through the task dequeue interface, and sends the task information to be executed to at least one corresponding target internet of things device, so that each target internet of things device executes the task according to the task information to be executed. In the application, a kafka cluster module is adopted as a message middleware to accept task traffic generated by the Internet of things platform, traffic peak clipping is realized, and a plurality of kafka consumer examples are used for acquiring task information to be scheduled from the kafka cluster module and giving the task information to a task scheduler example to carry out actual task scheduling; for actual task scheduling, the method adopts a plurality of task scheduler instances to be responsible for task scheduling according to task execution time information, adds task information to be executed to the lock-free task queue, and uses the plurality of task notification instances as consumers of the lock-free task queue to acquire the task to be executed through a task dequeue interface corresponding to the lock-free task queue and send the task to at least one corresponding target internet of things device, so that the task distribution efficiency is improved; for the task queue, the task enqueuing interface and the task dequeuing interface corresponding to the task queue without locking are used for enqueuing control and task dequeuing control of corresponding tasks by comparing and exchanging atomic operations, on one hand, the task queue without locking can reduce the acquisition and adding time of the tasks, which is beneficial to timely distributing the tasks to the Internet of things equipment, and on the other hand, the task queue control and dequeuing control by comparing and exchanging atomic operations can avoid competition caused when a plurality of task notification examples acquire the tasks and a plurality of task scheduler examples add the tasks, so that the ordered distribution of the tasks under a plurality of examples is realized. In summary, the task scheduling method and the distributed task scheduling system of the internet of things equipment provided by the application can coordinate a large number of tasks to be scheduled, improve timeliness of task distribution and task scheduling efficiency, and improve coordination and order of scheduling of a large number of tasks.
Therefore, on one hand, the kafka cluster and the kafka consumer are adopted to realize peak clipping of the task generating end, and on the other hand, the lock-free task queue is adopted to improve the order and speed of task enqueuing and task dequeuing, so that the task consumption speed is improved, the adaptation of mass tasks from the task generating end and the task consuming end is realized, and the method is an effective solution for coordinating a large amount of task information to be scheduled generated by the Internet of things platform.
Referring to fig. 5, fig. 5 is a schematic block diagram of a distributed task scheduling system 500 provided in the embodiment of the present application, where the distributed task scheduling system 500 includes a kafka cluster module 501, a plurality of kafka consumer instances 502, a plurality of task scheduler instances 503, a lock-free task queue 504, and a plurality of task notification instances 505; the task enqueuing interface corresponding to the lock-free task queue 504 adopts comparison aiming at a tail pointer and exchange atomic operation to carry out enqueuing control of the task; the task dequeue interface corresponding to the lock-free task queue 504 performs dequeue control of tasks by comparing head pointers and exchanging atomic operations; wherein,
The kafka cluster module is used for: receiving and storing task information to be scheduled from an internet of things platform; the task to be scheduled carries task execution time information;
the kafka consumer example is for: acquiring the task information to be scheduled from the kafka cluster module, and sending the task information to be scheduled to the task scheduler instance;
the task scheduler instance is to: generating task information to be executed corresponding to the task information to be scheduled according to the task execution time information, and adding the task information to be executed to the lock-free task queue through the task enqueue interface;
the task notification instance is configured to: and acquiring the task information to be executed from the lock-free task queue through the task dequeue interface, and sending the task information to be executed to at least one corresponding target internet of things device so that each target internet of things device executes the task according to the task information to be executed.
It will be clear to those skilled in the art that, for convenience and brevity of description, reference may be made to the corresponding process in the foregoing method embodiment for the specific working process of the above-described system, which is not described herein again. Those of ordinary skill in the art will appreciate that the elements, modules, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the elements, modules, and algorithm steps of the examples are generally described in terms of functionality in the foregoing description to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the several embodiments provided in this application, it should be understood that the disclosed methods and systems may be implemented in other ways. For example, the system embodiments described above are merely illustrative, and for example, the modules, units are merely logical functional partitions, and there may be additional partitions in actual implementation, or modules, units with the same functionality may be integrated into one module, unit, or component, e.g., multiple modules, units, or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices, or elements, or may be an electrical, mechanical, or other form of connection.
The modules and units described as separate units may or may not be physically separate, and units shown as modules and units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purposes of the embodiments of the present application.
In addition, each functional module and unit in each embodiment of the present application may be integrated into one processing module and unit, or each module and unit may exist alone physically, or two or more modules and units may be integrated into one module and unit. The integrated modules and units can be realized in a hardware form or a software functional unit form.
The integrated modules, units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a storage medium. Based on such understanding, the technical solution of the present application is essentially or a part contributing to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, including several instructions to cause a computer device (which may be a personal computer, a background server, or a network device, etc.) to perform all or part of the steps of the methods of the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, an optical disk, or other various media capable of storing program codes.
The foregoing is merely a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any equivalent modifications or substitutions will be apparent to those skilled in the art within the scope of the present application, and these modifications or substitutions should be covered in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (10)

1. The task scheduling method of the Internet of things equipment is characterized by being applied to a distributed task scheduling system, wherein the distributed task scheduling system comprises the following components: a plurality of task scheduler instances, a lock-free task queue, a kafka cluster module, a plurality of kafka consumer instances, and a plurality of task notification instances; the task enqueuing interface corresponding to the lock-free task queue adopts comparison aiming at a tail pointer and exchange atomic operation to carry out enqueuing control of the task; the task dequeue interface corresponding to the lock-free task queue adopts comparison for head pointers and exchange atomic operation to perform dequeue control of tasks, and the method comprises the following steps:
the kafka cluster module receives and stores task information to be scheduled from an internet of things platform; the task to be scheduled carries task execution time information;
The kafka consumer instance acquires the task information to be scheduled from the kafka cluster module and sends the task information to be scheduled to the task scheduler instance;
the task scheduler instance generates task information to be executed corresponding to the task information to be scheduled according to the task execution time information, and adds the task information to be executed to the lock-free task queue through the task enqueue interface;
the task notification instance obtains the task information to be executed from the lock-free task queue through the task dequeue interface, and sends the task information to be executed to at least one corresponding target internet of things device, so that each target internet of things device executes the task according to the task information to be executed.
2. The method according to claim 1, wherein the lock-free task queue is a circular queue, and the processing step corresponding to the task enqueuing interface includes:
acquiring a current head pointer position value and a current tail pointer position value of the lock-free task queue;
acquiring a next tail pointer position value according to the current head pointer position value and the size of the lock-free task queue;
Modifying the current tail pointer to point to the next tail pointer position value by comparing and swapping atomic operations when the current tail pointer is the same as the current tail pointer position value when the next tail pointer position value is not the same as the current head pointer position value;
and when the current tail pointer is successfully modified, writing the task information to be executed into a storage position corresponding to the current tail pointer position value in the lock-free task queue.
3. The method of claim 2, wherein after the obtaining the next tail pointer position value based on the current head pointer position value and the size of the lock-free task queue, the method further comprises:
and when the next tail pointer position value is equal to the current head pointer position value, returning to execute the current head pointer position value and the current tail pointer position value of the obtained lock-free task queue.
4. The method of claim 1, wherein the lock-free task queue is a circular queue, and the processing step corresponding to the task dequeue interface includes:
acquiring a current head pointer position value and a current tail pointer position value of the lock-free task queue;
When the current head pointer position value is not equal to the current tail pointer position value, acquiring a next head pointer position value according to the current head pointer position value and the size of the lock-free task queue;
modifying the current head pointer to point to the next head pointer position value by comparing and swapping atomic operations when the current head pointer is the same as the current head pointer position value;
and when the current head pointer is successfully modified, reading the task information to be executed in a storage position corresponding to the current head pointer position value in the lock-free task queue.
5. The method of claim 4, wherein after the obtaining the current head pointer position value and the current tail pointer position value of the lock-free task queue, the method further comprises:
and when the current head pointer position value is equal to the current tail pointer position value, discarding the queue as empty exception.
6. The method of claim 1, wherein the distributed task scheduling system further comprises a task information storage module, wherein the kafka consumer instance, after obtaining the task information to be scheduled from the kafka cluster module, the method further comprises:
The kafka consumer instance writes the task information to be scheduled into the task information storage module;
after the task scheduler instance generates the task information to be executed corresponding to the task information to be scheduled according to the task execution time information, the method further comprises:
and the task scheduler instance writes a task scheduling record corresponding to the task information to be executed into the task information storage module and associates the task scheduling record with the corresponding task information to be scheduled.
7. The method of claim 6, wherein after the task notification instance obtains the task information to be performed from the lock-free task queue through the task dequeue interface and sends the task information to be performed to the corresponding at least one target internet of things device, the method further comprises:
the task notification example receives task execution result information reported by the target internet of things equipment according to the task information to be executed, and sends the task execution result information to the kafka cluster module;
the kafka cluster module receives and stores the task execution result information;
the kafka consumer instance acquires the task execution result information from the kafka cluster module and writes the task execution result information into the corresponding task scheduling record in the task information storage module.
8. The method of claim 6, wherein after the task scheduler instance writes the task schedule record corresponding to the task information to be performed in the task information storage module, the method further comprises:
the kafka cluster module receives and stores deleting information of target task information to be scheduled from an internet of things platform;
the kafka consumer instance obtains the deletion information from the kafka cluster module;
and deleting the target task information to be scheduled in the task information storage module according to the deleting information by the kafka consumer instance, and sending stopping scheduling information aiming at the target task information to be scheduled to the task scheduler instance for scheduling the target task information to be scheduled.
9. The method of claim 1, wherein the task notification instance obtains the task information to be performed from the lock-free task queue through the task dequeue interface, and sends the task information to be performed to at least one corresponding target internet of things device, so that each target internet of things device performs a task according to the task information to be performed, including:
The task notification instance obtains the task information to be executed from the lock-free task queue through the task dequeue interface and sends the task information to a target theme in a task notification module based on an MQTT; the target theme is a preset theme corresponding to the task information to be executed;
forwarding the task information to be executed to each target internet of things device subscribed to the target theme through the task notification module based on the MQTT, wherein each internet of things device is subscribed to one or more preset themes in the task notification module based on the MQTT.
10. A distributed task scheduling system, comprising a plurality of task scheduler instances, a lock-free task queue, a kafka cluster module, a plurality of kafka consumer instances, and a plurality of task notification instances; the task enqueuing interface corresponding to the lock-free task queue adopts comparison aiming at a tail pointer and exchange atomic operation to carry out enqueuing control of the task; the task dequeue interface corresponding to the lock-free task queue adopts comparison aiming at a head pointer and exchange atomic operation to perform dequeue control of the task;
the kafka cluster module is used for: receiving and storing task information to be scheduled from an internet of things platform; the task to be scheduled carries task execution time information;
The kafka consumer example is for: acquiring the task information to be scheduled from the kafka cluster module, and sending the task information to be scheduled to the task scheduler instance;
the task scheduler instance is to: generating task information to be executed corresponding to the task information to be scheduled according to the task execution time information, and adding the task information to be executed to the lock-free task queue through the task enqueue interface;
the task notification instance is configured to: and acquiring the task information to be executed from the lock-free task queue through the task dequeue interface, and sending the task information to be executed to at least one corresponding target internet of things device so that each target internet of things device executes the task according to the task information to be executed.
CN202311529718.5A 2023-11-15 2023-11-15 Task scheduling method and distributed task scheduling system of Internet of things equipment Pending CN117453365A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311529718.5A CN117453365A (en) 2023-11-15 2023-11-15 Task scheduling method and distributed task scheduling system of Internet of things equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311529718.5A CN117453365A (en) 2023-11-15 2023-11-15 Task scheduling method and distributed task scheduling system of Internet of things equipment

Publications (1)

Publication Number Publication Date
CN117453365A true CN117453365A (en) 2024-01-26

Family

ID=89579803

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311529718.5A Pending CN117453365A (en) 2023-11-15 2023-11-15 Task scheduling method and distributed task scheduling system of Internet of things equipment

Country Status (1)

Country Link
CN (1) CN117453365A (en)

Similar Documents

Publication Publication Date Title
US5428781A (en) Distributed mechanism for the fast scheduling of shared objects and apparatus
Sprunt et al. Aperiodic task scheduling for hard-real-time systems
US8516054B2 (en) Message handling
US7219198B2 (en) Facilitating communication within shared memory environments using lock-free queues
US7707464B2 (en) Timeout request scheduling using grouping and nonsynchronized processing to enhance performance
CA2536037A1 (en) Fast and memory protected asynchronous message scheme in a multi-process and multi-thread environment
CN110231995B (en) Task scheduling method, device and storage medium based on Actor model
US20030233392A1 (en) Method and system for managing the execution of threads and the processing of data
CN101470636B (en) Message read-write method and apparatus
CN111949497B (en) Message queue system and message processing method based on same
CN109842621A (en) A kind of method and terminal reducing token storage quantity
EP2171934B1 (en) Method and apparatus for data processing using queuing
US7950011B2 (en) Leveraging advanced queues to implement event based job scheduling
WO2022211724A1 (en) Method and apparatus for transmitting messages, and device and storage medium thereof
US20060143150A1 (en) Messaging system and method of operation thereof
CN116633875B (en) Time order-preserving scheduling method for multi-service coupling concurrent communication
CN111190541B (en) Flow control method of storage system and computer readable storage medium
CN111225063B (en) Data exchange system and method for static distributed computing architecture
CN117453365A (en) Task scheduling method and distributed task scheduling system of Internet of things equipment
CN116225651A (en) Processor scheduling method, device, equipment and machine-readable storage medium
CN116107952A (en) Multi-core heterogeneous communication method and vehicle
CN114416717A (en) Data processing method and architecture
CN116225665B (en) Queue scheduling method and device
CN117591318A (en) Delay queue implementation method based on Kafka
CN115858133B (en) Batch data processing method and device, electronic equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination