CN110309024A - Data processing system and its method for executing data processing task - Google Patents
Data processing system and its method for executing data processing task Download PDFInfo
- Publication number
- CN110309024A CN110309024A CN201910327896.7A CN201910327896A CN110309024A CN 110309024 A CN110309024 A CN 110309024A CN 201910327896 A CN201910327896 A CN 201910327896A CN 110309024 A CN110309024 A CN 110309024A
- Authority
- CN
- China
- Prior art keywords
- data processing
- task
- queue
- echo signal
- signal amount
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3089—Monitoring arrangements determined by the means or processing involved in sensing the monitored data, e.g. interfaces, connectors, sensors, probes, agents
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Debugging And Monitoring (AREA)
Abstract
A kind of method that the embodiment of the present invention provides data processing system and its executes data processing task, belongs to field of communication technology.Wherein, system includes controlling equipment and data processing platform (DPP), and method includes: that controlling equipment obtains target timing task, generates goal task queue collection corresponding with target timing task, and goal task queue, which is concentrated, to be included at least pending queue and execute queue；Controlling equipment is based on the timing of target timing task and generates data processing task, and semaphore corresponding with data processing task is added to pending queue；When detecting echo signal amount in pending queue, echo signal amount is transferred to execution queue by data processing platform (DPP), and is executed target data corresponding with echo signal amount and handled task；If it is abnormal that target data handles task execution, echo signal amount is added to pending queue by controlling equipment again.Using the present invention, can be re-executed automatically to abnormal data processing task is executed without manual intervention.
Description
Technical field
The present invention relates to field of communication technology, in particular to a kind of data processing system and its data processing task is executed
Method.
Background technique
Data processing platform (DPP) can analyze the extensive application data that service application generates in the process of running, business
The operation maintenance personnel of application can be applied come quick positioning service according to the data processed result that data processing platform (DPP) is fed back and run
The anomalous event occurred in the process, and then tuning processing is carried out to the runnability of service application.
The prior art can make data processing platform (DPP) periodically generate data processing task by the way that timed task is arranged.It is existing
The mode for having data processing platform (DPP) to generally use batch processing calculating handles the application data of lasting generation, specifically,
Data processing platform (DPP) can periodically generate the data processing task of needle application data based on timed task, then for every
One operation process of a data processing task creation is to service application in time window corresponding with each data processing task
All application data generated are handled.For example, the operation maintenance personnel of service application in data processing platform (DPP) be provided with " from when
Carve 8:00 rise, every three minutes detection service application whether be once operating abnormally " timed task, as arrival time 8:03
When, data processing platform (DPP) generates a data processing task and creates corresponding operation process, passes through the operation process analysis industry
Business judges whether service application was once operating abnormally using the application data generated from moment 8:00 to moment 8:03, works as arrival
When moment 8:06, data processing platform (DPP) generates another data processing task and creates corresponding operation process, passes through the operation
The application data that process analysis service application is generated from moment 8:03 to moment 8:06 judge whether service application once ran
It is abnormal, and so on.
The inventor of present patent application executes data by studying the existing data processing platform (DPP) calculated using batch processing
The process of processing task finds that the existing technology has at least the following problems: cannot using the data processing platform (DPP) that batch processing calculates
It is re-executed automatically for the data processing task for executing exception (including executing time-out and executing failure), needs operation maintenance personnel master
Dynamic inquiry with the presence or absence of the data processing task for executing time-out or failure, and to execute the data processing task of time-out or failure into
Row manual intervention re-executes, manual maintenance low efficiency and at high cost, moreover, when executing a large amount of short time window (usually
No more than ten minutes) corresponding data processing task when, data processing platform (DPP) needs frequently creation and deletes a large amount of to execute data
The operation process of processing task expends data processing platform (DPP) resource and influences the stability of data processing platform (DPP).
Summary of the invention
A kind of method for being designed to provide data processing system and its execute data processing task of the application, to solve
Part or all of problem of the existing technology.
To achieve the above object, on the one hand the application provides a kind of method for executing data processing task, and this method is answered
For data processing system, data processing system includes controlling equipment and data processing platform (DPP), this method comprises: controlling equipment obtains
Target timing task is taken, goal task queue collection corresponding with target timing task is generated, goal task queue concentration is at least wrapped
It includes pending queue and executes queue；Controlling equipment is based on the timing of target timing task and generates data processing task, and will be with number
Pending queue is added to according to the corresponding semaphore of processing task；When detecting echo signal amount in pending queue, number
Echo signal amount is transferred to execution queue according to processing platform, and executes target data processing corresponding with echo signal amount and appoints
Business；If it is abnormal that target data handles task execution, echo signal amount is added to pending queue by controlling equipment again.
Further, system further includes management equipment；The step of before controlling equipment acquisition target timing task includes: pipe
It manages equipment and is based on CRON expression formula configuration target timing task, and CRON expression formula is added to task allocation list, so that scheduling
Equipment reads target timing task from task allocation list.
In one embodiment, this method further include: data processing platform (DPP) is according to echo signal amount in pending queue
Put in order from pending queue obtain echo signal amount；If detecting, the execution of the target data processing task is preferential
The modification request of grade, the controlling equipment modify the echo signal amount putting in order in the pending queue, so that
The data processing platform (DPP) obtains the echo signal amount according to modified put in order from the pending queue.
In one embodiment, goal task queue concentration further includes unsuccessfully queue；If target data handles task execution
It is abnormal, if controlling equipment includes: target data processing task the step of echo signal amount is then added to pending queue again
Failure is executed, echo signal amount is transferred to unsuccessfully queue by data processing platform (DPP)；When failure queue in detect echo signal
When amount, echo signal amount is transferred to pending queue by controlling equipment.
In one embodiment, goal task queue concentration further includes overtime queue；If target data handles task execution
It is abnormal, if controlling equipment includes: that echo signal amount is executing the step of echo signal amount is then added to pending queue again
Storage time in queue is more than default storage time, and echo signal amount is transferred to overtime queue by controlling equipment；When in time-out
When detecting echo signal amount in queue, echo signal amount is transferred to pending queue by controlling equipment.
In one embodiment, goal task queue concentration further includes filing queue；This method further include: data processing is flat
Platform records data processing task in semaphore and runs succeeded or execute the implementing result of failure, and will have recorded implementing result
Semaphore is added to filing queue；Controlling equipment records the implementing result that data processing task executes time-out in semaphore, and
The semaphore for having recorded implementing result is added to filing queue.
Further, data processing system further includes archival device；Controlling equipment or data processing platform (DPP), which will carry, to be executed
As a result the step of semaphore is added to after filing queue includes: when detecting that data processing task is corresponding from filing queue
Semaphore when, archival device obtain semaphore, read implementing result, and by implementing result be stored in external data base appoint
Business implementing result collection.
Further, data processing system further includes warning device；Implementing result is stored in external data by archival device
The step of after task action result collection in library includes: when the processing task from task action result centralized detecting to target data
Implementing result be warning device alarm when executing failure or executing the number of time-out and be greater than default frequency of abnormity.
In one embodiment, data processing platform (DPP) executes data processing task using streaming computing；This method is also wrapped
Include: data processing platform (DPP), which is detected in pending queue by Resident Process corresponding with target timing task according to predetermined period, is
It is no that there are semaphores.
To achieve the above object, on the other hand the application provides a kind of data processing system, the data processing system packet
Include controlling equipment and data processing platform (DPP), wherein controlling equipment generates and target timing task for obtaining target timing task
Corresponding goal task queue collection, goal task queue concentrate and include at least pending queue, execute queue and failure queue；It adjusts
Degree equipment is also used to generate data processing task based on the timing of target timing task, and will signal corresponding with data processing task
Amount is added to pending queue；When detecting echo signal amount in pending queue, data processing platform (DPP) is used for target
Semaphore is transferred to execution queue, and executes target data corresponding with echo signal amount and handle task；If target data is handled
Task execution is abnormal, and controlling equipment is also used to echo signal amount being added to pending queue again.
Further, the system is for realizing the above-mentioned method for executing data processing task.
Therefore the present invention is compared to the prior art, and first, the present invention is by scheduling signals amount come scheduling signals amount pair
The data processing task answered stores semaphore by task queue collection, reduces the coupling of controlling equipment and data processing platform (DPP)
It is right, enhance controlling equipment and the independently operated ability of data processing platform (DPP).Second, a variety of teams concentrated by task queue
Column classify to the execution state of data processing task, and reschedule the data processing task for executing time-out or executing failure
Corresponding semaphore, and then the data processing task is re-executed automatically, avoid manual intervention.Third, will by archival device
The implementing result of data processing task unifies persistent storage in external data base, the operation maintenance personnel convenient for service application consult and
Analysis of history implementing result.4th, the implementing result stored in external data base is periodically obtained by warning device, it can be to holding
The abnormal data processing task of row is alarmed.5th, data processing platform (DPP) can be passed through at Resident Process using streaming computing
Data are managed, without frequently creating and deleting a large amount of operation processes for executing data processing task, data processing system is improved and exists
Stability in operation and maintenance.
Detailed description of the invention
To describe the technical solutions in the embodiments of the present invention more clearly, make required in being described below to embodiment
Attached drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention, for
For those of ordinary skill in the art, without creative efforts, it can also be obtained according to these attached drawings other
Attached drawing.
Fig. 1 is a kind of flow chart of method for executing data processing task provided in an embodiment of the present invention；
Fig. 2 is a kind of transfer stream of the corresponding semaphore of data processing task to run succeeded provided in an embodiment of the present invention
Journey schematic diagram.
Fig. 3 is a kind of transfer stream of corresponding semaphore of data processing task for executing failure provided in an embodiment of the present invention
Journey schematic diagram；
Fig. 4 is a kind of transfer stream of corresponding semaphore of data processing task for executing time-out provided in an embodiment of the present invention
Journey schematic diagram
Fig. 5 be another embodiment of the present invention provides a kind of data processing system general frame schematic diagram.
Specific embodiment
Below in conjunction with attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that institute
The embodiment of description is only a part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, originally
Field those of ordinary skill every other embodiment obtained without creative efforts, belongs to the present invention
The range of protection.
The embodiment of the invention provides a kind of method for executing data processing task, this method is applied to data processing system
System, this method can by data processing system controlling equipment and data processing platform (DPP) realize jointly.Wherein, controlling equipment is used
In scheduling data processing task, plan the execution time of each data processing task, controlling equipment can be each timed task
Generate corresponding task queue collection, controlling equipment can also be generated according to target timing task timing data processing task and with
The corresponding semaphore of each data processing task, and semaphore is stored in corresponding task queue and is concentrated.It should be noted that
Controlling equipment can be distributed apparatus, may include multiple dispatching host machines, and multiple dispatching host machines can enable distributed lock clothes
Business is locked according to predetermined period request, and any time can only have a dispatching host machine to get lock, by the tune for getting lock
Degree host realizes the function of controlling equipment, other dispatching host machines for realizing controlling equipment failover backup.Data processing is flat
Platform can be concentrated from task queue and obtain semaphore, and execute data processing task corresponding with the semaphore got；Data
The corresponding semaphore of data processing task for executing failure can also be added to again corresponding task queue collection by processing platform,
To allow data processing platform (DPP) to re-execute corresponding data processing task.It should be noted that in data processing platform (DPP)
May include data transmitting/receiving equipment, data transmitting/receiving equipment be used to obtain from other equipment for execute data processing task to from
Data are managed, and task action result is sent to other equipment.
Fig. 1 is a kind of flow chart of method for executing data processing task provided in an embodiment of the present invention.
S101, controlling equipment obtain target timing task, generate goal task queue collection corresponding with target timing task,
Goal task queue, which is concentrated, includes at least pending queue and execution queue.
In the present embodiment, for the operation maintenance personnel of service application when configuring target timing task, data processing system can root
The task ID of target timing task is generated according to target timing task.Wherein, the task ID of target timing task is for marking mesh
Mark the globally unique identification code (for example, from ID is increased) of timed task, it will be understood that in this data processing system, different timing
The task ID of task is different.Controlling equipment in data processing system can obtain mesh when obtaining target timing task simultaneously
Mark the task ID of timed task.After controlling equipment gets the task ID of target timing task and target timing task, Ke Yisheng
At goal task queue collection corresponding with target timing task, and goal task is marked by the task ID of target timing task
Queue collection, to establish the corresponding relationship of target timing missions and goals task queue collection.It should be noted that goal task team
Column collection for storing data concentrate including at least pending queue and execution by the corresponding semaphore of processing task, goal task queue
Queue, wherein semaphore is stored in pending queue, illustrate the corresponding data processing task of the semaphore be in etc. it is pending
State；Semaphore, which is stored in, to be executed in queue, illustrates that the corresponding data processing task of the semaphore is in the shape being carrying out
State.
Optionally, data processing system can also include management equipment, and operation maintenance personnel can be configured by management equipment
Timed task.Correspondingly, the step of before controlling equipment acquisition target timing task may include: that management equipment is based on CRON table
Target timing task is configured up to formula, and CRON expression formula is added to task allocation list, so that controlling equipment is from task allocation list
Read target timing task.
In one embodiment, management equipment can have UI (User Interface, user interface), answer convenient for business
Operation maintenance personnel operation.The operation maintenance personnel of service application can in management equipment by CRON expression formula come human configuration
Target timing task, the configuration data recorded in the CRON expression formula may include the data processing instructions of data processing task with
And the formation condition of data processing task.Wherein, the formation condition of data processing task may include the head of data processing task
The secondary parameters such as generation frequency for generating time, data processing task.After the configuration of target timing task, management equipment can be with
Target timing task is stored in task allocation list, so that controlling equipment reads target timing task from task allocation list.It can be with
Understand, controlling equipment can continue to monitor or periodically read timed task from task allocation list.In addition, the fortune of service application
Dimension personnel can modify already existing target timing task in task allocation list by management equipment, and controlling equipment can be
After reading modified target timing task, data processing task is generated according to modified target timing task.
S102, controlling equipment are based on the timing of target timing task and generate data processing task, and will be with data processing task
Corresponding semaphore is added to pending queue.
In the present embodiment, the data recorded in semaphore may include that task ID and the remaining of data processing task execute
Number, it will be understood that data processing task is every to be executed once, and the residue of the data processing task recorded in corresponding semaphore is held
Row number subtracts one.It, can be according to target timing task after controlling equipment in data processing system gets target timing task
Configuration data, from specific time every certain time length automatically generated data handle task, be not necessarily to manual intervention.Also, it is
The degree of coupling for reducing the controlling equipment and data processing platform (DPP) in data processing system is generating each data processing task
Meanwhile semaphore corresponding with data processing task can be generated in controlling equipment, and is added semaphore by scheduling process
The pending queue concentrated to goal task queue corresponding with target timing task, so that at data in data processing system
Platform obtains semaphore, and then executes the corresponding data processing task of semaphore.
It should be noted that the memory space for each queue that goal task queue is concentrated can be pre-set size
Fixed, being also possible to can be according to adjusted size of, the invention is not limited in this regard of actual demand dynamic.
S103, when detecting echo signal amount in pending queue, data processing platform (DPP) shifts echo signal amount
To executing queue, and execute corresponding with echo signal amount target data processing task.
In the present embodiment, since data processing platform (DPP) only just executes at the corresponding data of semaphore after getting semaphore
Reason task, therefore, data processing platform (DPP), which can be detected at regular intervals in the pending queue that goal task queue is concentrated, is
No there are semaphores, and when detecting echo signal amount in pending queue, data processing platform (DPP) determination can execute and mesh
The corresponding data processing task of semaphore is marked, at this point, echo signal amount is transferred to execution queue by data processing platform (DPP), then,
Data processing platform (DPP) executes target data corresponding with echo signal amount and handles task.
Optionally, the operation maintenance personnel of service application can carry out manual intervention to the execution priority of data processing task,
Execute sequence modify data processing task, correspondingly, method provided by the invention can also include: data processing platform (DPP) by
Echo signal amount is obtained from pending queue according to echo signal amount putting in order in pending queue；If detecting target
The modification of the execution priority of data processing task is requested, and controlling equipment modifies arrangement of the echo signal amount in pending queue
Sequentially, so that data processing platform (DPP) obtains echo signal amount according to modified put in order from pending queue.
In one embodiment, the higher semaphore that shows as of the execution priority of data processing task is in pending queue
Storage location it is more forward, data processing platform (DPP) according to each semaphore in pending queue putting in order successively obtain believe
Number amount, and then successively executes the corresponding data processing task of each semaphore.Therefore, when need to modify target data processing task
Execution priority when, the operation maintenance personnel of service application can by controlling equipment to echo signal amount in pending queue
Storage location is modified, and later, data processing platform (DPP) can be according to the storage location of echo signal amount from pending queue
Obtain echo signal amount.For example, controlling equipment detects the modification request of the execution priority of target data processing task, this is repaired
Change request and be changed to highest for the execution priority that target data is handled task, at this point, controlling equipment can be by echo signal amount
It is stored in the head of the queue of pending queue.It is appreciated that controlling equipment can if echo signal amount is stored in the tail of the queue of pending queue
The echo signal amount for being located at the tail of the queue of pending queue to be moved to the head of the queue of pending queue；If echo signal amount is stored in
The echo signal amount for being located at failure queue can be moved to the head of the queue of pending queue, later, number by failure queue, controlling equipment
Echo signal amount can be got from pending queue first according to processing platform.
S104, if target data processing task execution is abnormal, echo signal amount is added to again wait hold by controlling equipment
Row queue.
In the present embodiment, the implementing result of target data processing task, which can be, to be executed exception or runs succeeded, and works as target
When data processing task executes abnormal, in order to re-execute target data processing task, controlling equipment can be by echo signal amount
Again it is added to pending queue, so that data processing platform (DPP) can reacquire echo signal amount, and then re-executes target
The corresponding target data of semaphore handles task.
Optionally, the data processing task for occurring to execute failure to re-execute data processing platform (DPP) automatically, and
The data processing task for executing failure is differentiated with other data processing tasks, can be lost in goal task queue concentrated setting
Queue is lost, correspondingly, step S104 can be specifically included: if target data processing task execution failure, data processing platform (DPP) will
Echo signal amount is transferred to unsuccessfully queue；When detecting echo signal amount in failure queue, controlling equipment is by echo signal
Amount is transferred to pending queue.
In one embodiment, data processing platform (DPP) is during performance objective data processing task, if target data
Task execution failure is handled, echo signal amount first can be transferred to unsuccessfully queue from queue is executed by data processing platform (DPP), it
Afterwards, controlling equipment detects in failure queue is transferred to pending queue for echo signal amount after echo signal amount.It needs
Bright, controlling equipment can periodically detect in the failure queue of target timing task with the presence or absence of semaphore, when losing
It loses when detecting one or more semaphores in queue, controlling equipment is by the semaphore in failure queue according to putting in order one by one
It is transferred to pending queue.Wherein, the reason of causing data processing task to execute failure, can be external factor, for example, executing
Target data processing task needs data processing platform (DPP) to obtain pending data from exterior database, and target data processing is appointed
In the process of implementation due to external data bank interface temporary derangement, data processing platform (DPP) accesses external data base failure for business, causes
Target data handles task execution failure.
Optionally, the data processing task for occurring to execute time-out to re-execute data processing platform (DPP) automatically, and
The data processing task for executing time-out is differentiated with other data processing tasks, correspondingly, step S104 can be specifically included:
If echo signal amount is more than default storage time executing the storage time in queue, echo signal amount is transferred to by controlling equipment
Overtime queue；When detecting the corresponding semaphore of data processing task in overtime queue, controlling equipment shifts semaphore
To pending queue.
In one embodiment, the default storage time of semaphore is that the semaphore can store most in executing queue
For a long time, it represents and executes the duration that the corresponding data processing task of the semaphore allows to consume.If controlling equipment monitors mesh
It is more than default storage time that semaphore, which is marked, executing the storage time in queue, indicates data processing platform (DPP) in performance objective data
The duration consumed when processing task has been more than the duration for allowing to consume, at this point, the implementing result of target data processing task is to hold
Row time-out, data processing platform (DPP) terminate performance objective data processing task.Then, echo signal amount is transferred to super by controlling equipment
When queue, later, controlling equipment detects in overtime queue is transferred to pending team for echo signal amount after echo signal amount
Column.It should be noted that controlling equipment can periodically detect in the overtime queue of target timing task with the presence or absence of signal
Amount, when detecting one or more semaphores in overtime queue, controlling equipment is by the semaphore in overtime queue according to row
Column sequence is transferred to pending queue one by one.
It is noted that controlling equipment can judge echo signal amount in executing queue by time-out check device
Whether storage time is more than default storage time.For example, when echo signal amount starts to store in executing queue, time-out check
The timer for the default storage time that a time-out time is echo signal amount can be set in process, when the timer timing reaches
Time-out time, echo signal amount are not removed from execution queue also, illustrate that echo signal amount is executing the storage time in queue
More than default storage time.
In one embodiment, the residual memory space deficiency in order to prevent in pending queue causes semaphore storage to be lost
It loses, unallocated queue can be set before pending queue as buffer area.Wherein, data processing platform (DPP) is by semaphore by executing
Queue is transferred to unsuccessfully the case where semaphore is transferred to overtime queue by execution queue by queue and controlling equipment and is suitable for
The present embodiment.Before semaphore in failure queue and overtime queue is added to pending queue by controlling equipment, can first by
Semaphore is added to the unallocated queue of goal task queue concentration.In this way, when semaphore increases suddenly within a certain period of time,
Such as the corresponding data processing task of target timing task frequently executes exception within certain time, unallocated queue not only needs
The corresponding semaphore of data processing task for executing exception will be stored by storing newly-generated semaphore also, at this time, it may be necessary to storage
Semaphore quantity is more than the open ended quantity of pending queue, and the semaphore that pending queue powerlessly stores is temporarily stored into unallocated team
Column, scheduling system can be periodically detected with the presence or absence of semaphore in unallocated queue, and detect the residue of pending queue
Whether memory space stores one or more semaphores enough, when in unallocated queue there are semaphore, and pending queue
Residual memory space when storing one or more semaphores enough, controlling equipment can press the semaphore in unallocated queue
It is transferred to pending queue one by one according to putting in order.
Optionally, goal task queue collection can also include filing queue, for storing holding for Data Data processing task
Row is as a result, correspondingly, method provided by the invention can also include: that data processing platform (DPP) records data processing times in semaphore
Business runs succeeded or executes the implementing result of failure, and the semaphore for having recorded implementing result is added to filing queue；Scheduling
Equipment records data processing task in semaphore and executes the implementing result of time-out, and the semaphore for having recorded implementing result is added
Add to filing queue.
It in one embodiment, can be by data processing platform (DPP) due to executing failure and both implementing results that run succeeded
It is directly determined at the end of data processing task executes, and executing this implementing result of time-out is determined by controlling equipment, tool
Body, the time that controlling equipment can be stored in executing queue by time-out check process detection semaphore is corresponding to determine
Whether data processing task executes time-out.It herein, can be unified by three kinds of implementing results for the ease of subsequent acquisition implementing result
It is stored in filing queue.Therefore, data processing platform (DPP) can record at data in the corresponding semaphore of data processing task
The implementing result that reason task execution fails or runs succeeded, and the semaphore for having recorded implementing result is added to filing queue.
Controlling equipment can record the implementing result that data processing task executes time-out in the corresponding semaphore of data processing task, and
The semaphore for having recorded implementing result is added to filing queue.
Wherein, the situation of time-out, data processing task are run succeeded, execute failure and executed for data processing task
The transfer flow of corresponding semaphore can distinguish-Fig. 4 referring to fig. 2, wherein the serial number in Fig. 2-Fig. 4 indicates the shifting of semaphore
Dynamic sequence.
In one embodiment, the residual memory space deficiency in order to prevent in pending queue causes semaphore storage to be lost
It loses, unallocated queue can be set before pending queue as buffer area.Wherein, data processing platform (DPP) is by semaphore by executing
Queue is transferred to unsuccessfully the case where semaphore is transferred to overtime queue by execution queue by queue and controlling equipment and is suitable for
The present embodiment.Before semaphore in failure queue and overtime queue is added to pending queue by controlling equipment, can first by
Semaphore is added to the unallocated queue of goal task queue concentration.In this way, when semaphore increases suddenly within a certain period of time,
Such as the corresponding data processing task of target timing task frequently executes exception within certain time, unallocated queue not only needs
The corresponding semaphore of data processing task for executing exception will be stored by storing newly-generated semaphore also, at this time, it may be necessary to storage
Semaphore quantity is more than the open ended quantity of pending queue, and the semaphore that pending queue powerlessly stores is temporarily stored into unallocated team
Column, scheduling system can be periodically detected with the presence or absence of semaphore in unallocated queue, and detect the residue of pending queue
Whether memory space stores one or more semaphores enough, when in unallocated queue there are semaphore, and pending queue
Residual memory space when storing one or more semaphores enough, controlling equipment can press the semaphore in unallocated queue
It is transferred to pending queue one by one according to putting in order.
Optionally, in order to which the implementing result of all data processing tasks is carried out persistent storage, data processing system is also
It may include archival device, correspondingly, the semaphore for carrying implementing result is added to and returns by controlling equipment or data processing platform (DPP)
The step of after shelves queue may include: the filing when detecting the corresponding semaphore of data processing task from filing queue
Equipment obtains semaphore, reads implementing result, and implementing result is stored in the task action result collection in external data base.
In one embodiment, archival device can periodically detect in filing queue with the presence or absence of semaphore, when from
When detecting the corresponding semaphore of data processing task in filing queue, the institute that is stored in the available filing queue of archival device
There is semaphore, reads the implementing result of the corresponding data processing task of each semaphore, and implementing result is stored in external number
According to the task action result collection in library, task action result collection can be used as implementing result page presentation and follow-up data processing
The data source of task execution abnormal alarm.Wherein, external data base can be distributed storage device.It should be noted that returning
Shelves equipment can be distributed apparatus, may include multiple filing hosts, and multiple filing hosts can enable distributed lock service,
It is locked according to predetermined period request, any time can only have a filing host to get lock, by the filing master for getting lock
Machine realizes the function of archival device, other filing hosts for realizing archival device failover backup.
Optionally, in order to can also be wrapped with automatic alarm, data processing system when data processing task executes abnormal
Include warning device.Correspondingly, implementing result is stored in after the task action result collection in external data base by archival device
Step may include: when from task action result centralized detecting to target data handle task implementing result for execute failure and
When executing the number of time-out greater than default frequency of abnormity, warning device alarm.
In one embodiment, the general frame schematic diagram of data processing system can be found in Fig. 5.In order in time to again
It executes repeatedly but still executes abnormal data processing task and alarm, it can be on warning device in a data processing system
Default frequency of abnormity is set, when the abnormal number of execution, which occurs, for each data processing task is more than default frequency of abnormity, alarm
Equipment automatic alarm notifies operation maintenance personnel.For example, warning device can be according to certain measurement period regular visit external number
According to the task action result collection in library, the implementing result for calculating the target data processing task within each period is to execute failure and hold
The number of row time-out, when calculated result is greater than default frequency of abnormity, warning device alarm.It should be noted that warning device
Different alarm trigger conditions can also be set for different timing task, determine for example, being arranged according to the importance of timed task
When the corresponding each data processing task of task default frequency of abnormity preset frequency of abnormity for more important timed task
Value can be smaller.
Optionally, it can have been executed in time to guarantee to handle the data processing task using data in short time window
Finish, in any one of the above-described embodiment, data processing platform (DPP) can execute data processing task using streaming computing.
Correspondingly, method provided by the invention can also include: data processing platform (DPP) by it is corresponding with target timing task reside into
Journey is detected according to predetermined period whether there is semaphore in pending queue.
In one embodiment, the operation maintenance personnel of service application, can be in data processing when configuring target timing task
Platform creates and opens Resident Process corresponding with target timing task, and establishes target by the task ID of target timing task
The corresponding relationship of timed task and Resident Process.Then, data processing platform (DPP) can by it is corresponding with the task ID reside into
Journey, detecting according to predetermined period whether there is semaphore in the pending queue that goal task queue is concentrated.
It should be noted that the Computational frame that arbitrarily can be realized streaming computing, this hair can be used in data processing platform (DPP)
It is bright to this with no restriction.For example, storm Computational frame can be used in data processing platform (DPP), pass through the spout line in Resident Process
Journey come periodic detection goal task queue concentration pending queue in the presence or absence of semaphore and obtain echo signal amount and
Corresponding pending data, and handled by the bolt thread in Resident Process come the corresponding target data of performance objective semaphore
Task, therefore not to repeat here by the specific data handling procedure present invention.It is noted that each Resident Process may include more
A spout thread and multiple bolt threads, the corresponding Resident Process of target timing task may be performed simultaneously target timing task
Corresponding multiple data processing tasks.
Therefore the present invention is compared to the prior art, and first, the present invention is by scheduling signals amount come scheduling signals amount pair
The data processing task answered stores semaphore by task queue collection, reduces the coupling of controlling equipment and data processing platform (DPP)
It is right, enhance controlling equipment and the independently operated ability of data processing platform (DPP).Second, a variety of teams concentrated by task queue
Column classify to the execution state of data processing task, and reschedule the data processing task for executing time-out or executing failure
Corresponding semaphore, and then the data processing task is re-executed automatically, avoid manual intervention.Third, will by archival device
The implementing result of data processing task unifies persistent storage in external data base, the operation maintenance personnel convenient for service application consult and
Analysis of history implementing result.4th, the implementing result stored in external data base is periodically obtained by warning device, it can be to holding
The abnormal data processing task of row is alarmed.5th, data processing platform (DPP) can be passed through at Resident Process using streaming computing
Data are managed, without frequently creating and deleting a large amount of operation processes for executing data processing task, data processing system is improved and exists
Stability in operation and maintenance.
Based on the same technical idea, the embodiment of the invention also provides a kind of data processing system, the data processing systems
System includes controlling equipment and data processing platform (DPP), wherein controlling equipment is for obtaining target timing task, generation and target timing
The corresponding goal task queue collection of task, the goal task queue, which is concentrated, to be included at least pending queue, executes queue and mistake
Lose queue；
The controlling equipment is also used to generate data processing task based on target timing task timing, and will with it is described
The corresponding semaphore of data processing task is added to the pending queue；
When detecting echo signal amount in the pending queue, the data processing platform (DPP) is used for the target
Semaphore is transferred to the execution queue, and executes target data corresponding with the echo signal amount and handle task；
If the target data processing task execution is abnormal, the controlling equipment is also used to the echo signal amount again
It is added to the pending queue.Optionally, data processing system can also include management equipment；It is fixed that controlling equipment obtains target
When task before, management equipment be used for based on CRON expression formula configure target timing task, and by CRON expression formula be added to appoint
Business allocation list, so that controlling equipment reads target timing task from task allocation list.
Optionally, data processing platform (DPP) is also used to suitable according to arrangement of the echo signal amount in the pending queue
Sequence obtains the echo signal amount from the pending queue；If detecting the modification of the echo signal amount to put in order
Request, the controlling equipment is also used to modify putting in order for the echo signal amount according to the modification request, so that described
Data processing platform (DPP) obtains the echo signal amount according to modified put in order from the pending queue.
Optionally, it can also include failure queue that the goal task queue, which is concentrated,；If the target data handles task
Failure is executed, the data processing platform (DPP) is specifically used for for the echo signal amount being transferred to the failure queue；When described
In failure queue when detecting the echo signal amount, the controlling equipment be used to for the echo signal amount being transferred to it is described to
Execute queue.
Optionally, it can also include overtime queue that the goal task queue, which is concentrated,；If the echo signal amount is described
Executing the storage time in queue is more than default storage time, and the controlling equipment is specifically used for shifting the echo signal amount
To the overtime queue；When detecting the echo signal amount in the overtime queue, the controlling equipment is used for institute
It states echo signal amount and is transferred to the pending queue.
Optionally, the goal task queue concentration can also include filing queue；Data processing platform (DPP) is also used to believing
Record data processing task runs succeeded or executes the implementing result to fail, and the semaphore that will have recorded implementing result in number amount
It is added to filing queue；Controlling equipment is also used to record the implementing result that data processing task executes time-out in semaphore, and
The semaphore for having recorded implementing result is added to filing queue.
Further, the system can also include archival device；When detecting the data from the filing queue
When the corresponding semaphore of processing task, the archival device reads the implementing result for obtaining the semaphore, and by institute
State the task action result collection that implementing result is stored in external data base.
Further, the system can also include warning device；When from the task action result centralized detecting to mesh
When the implementing result of mark data processing task is that execution fails and the number of execution time-out is greater than preset frequency of abnormity, the report
Alert equipment alarm.
Optionally, the data processing platform (DPP) executes the data processing task using streaming computing；At the data
Platform can be used for detecting by Resident Process corresponding with the target timing task according to predetermined period described pending
It whether there is the semaphore in queue.
Controlling equipment provided in this embodiment and data processing platform (DPP) realize above-mentioned execution data processing task jointly
Method.Its realization principle and technical effect to be achieved above have discussion, and details are not described herein.
Through the above description of the embodiments, those skilled in the art can be understood that each embodiment can
It realizes by means of software and necessary general hardware platform, naturally it is also possible to be realized by hardware.Based on such
Understand, substantially the part that contributes to existing technology can embody above-mentioned technical proposal in the form of software products in other words
Out, which may be stored in a computer readable storage medium, as ROM/RAM, magnetic disk,
CD etc., including storage some instructions with so that a computer executes certain part institutes of each embodiment or embodiment
The method stated.
The foregoing is merely presently preferred embodiments of the present invention, is not intended to limit the invention, it is all in spirit of the invention and
Within principle, any modification, equivalent replacement, improvement and so on be should all be included in the protection scope of the present invention.
Claims (11)
1. a kind of method for executing data processing task, which is characterized in that the method is applied to data processing system, the system
System includes controlling equipment and data processing platform (DPP), which comprises
The controlling equipment obtains target timing task, generates goal task queue collection corresponding with the target timing task,
The goal task queue, which is concentrated, includes at least pending queue and execution queue；
The controlling equipment is based on the target timing task timing and generates data processing task, and will appoint with the data processing
Corresponding semaphore of being engaged in is added to the pending queue；
When detecting echo signal amount in the pending queue, the data processing platform (DPP) turns the echo signal amount
The execution queue is moved to, and executes target data corresponding with the echo signal amount and handles task；
If the target data processing task execution is abnormal, the echo signal amount is added to institute by the controlling equipment again
State pending queue.
2. the method as described in claim 1, which is characterized in that the system also includes management equipments；
The controlling equipment obtain target timing task before the step of include:
The management equipment is based on CRON expression formula and configures the target timing task, and the CRON expression formula is added to and is appointed
Business allocation list, so that the controlling equipment reads the target timing task from the task allocation list.
3. the method as described in claim 1, which is characterized in that the method also includes:
The data processing platform (DPP) putting in order from described pending in the pending queue according to the echo signal amount
The echo signal amount is obtained in queue；
If detecting the modification request of the execution priority of the target data processing task, the controlling equipment modifies the mesh
Semaphore putting in order in the pending queue is marked, so that the data processing platform (DPP) puts in order according to modified
The echo signal amount is obtained from the pending queue.
4. the method as described in claim 1, which is characterized in that the goal task queue concentration further includes unsuccessfully queue；
If the target data processing task execution is abnormal, the controlling equipment adds the echo signal amount again
Include: to the step of pending queue
If the target data processing task execution failure, the echo signal amount is transferred to described by the data processing platform (DPP)
Failure queue；
When detecting the echo signal amount in the failure queue, the controlling equipment shifts the echo signal amount
To the pending queue.
5. the method as described in claim 1, which is characterized in that the goal task queue concentration further includes overtime queue；
If the target data processing task execution is abnormal, the controlling equipment adds the echo signal amount again
Include: to the step of pending queue
If storage time of the echo signal amount in the execution queue is more than default storage time, the controlling equipment will
The echo signal amount is transferred to the overtime queue；
When detecting the echo signal amount in the overtime queue, the controlling equipment shifts the echo signal amount
To the pending queue.
6. the method as described in claim 1, which is characterized in that the goal task queue concentration further includes filing queue；
The method also includes:
The data processing platform (DPP) records the data processing task in the semaphore and runs succeeded or execute holding for failure
Row is as a result, and be added to the filing queue for the semaphore for having recorded implementing result；
The controlling equipment records the implementing result that the data processing task executes time-out in the semaphore, and will record
The semaphore of implementing result is added to the filing queue.
7. method as claimed in claim 6, which is characterized in that the system also includes archival devices；
The semaphore for having recorded implementing result is added to the filing by the controlling equipment or the data processing platform (DPP)
The step of after queue includes:
When detecting the corresponding semaphore of the data processing task from the filing queue, the archival device obtains institute
Semaphore is stated, reads the implementing result recorded in the semaphore, and the implementing result is stored in external data base
Task action result collection.
8. the method for claim 7, which is characterized in that the system also includes warning devices；
The step of implementing result is stored in after the task action result collection in external data base by archival device packet
It includes:
When the implementing result for handling task from the task action result centralized detecting to target data is to execute failure and execute
When the number of time-out is greater than preset frequency of abnormity, the warning device alarm.
9. such as method of any of claims 1-8, which is characterized in that the data processing platform (DPP) uses streaming computing
To execute the data processing task；
The method also includes:
Described in the data processing platform (DPP) is detected by Resident Process corresponding with the target timing task according to predetermined period
It whether there is the semaphore in pending queue.
10. a kind of data processing system, which is characterized in that the data processing system includes that controlling equipment and data processing are flat
Platform, wherein
The controlling equipment generates goal task queue corresponding with the target timing task for obtaining target timing task
Collection, the goal task queue concentrate and include at least pending queue, execute queue and failure queue；
The controlling equipment is also used to generate data processing task based on the target timing task timing, and will be with the data
The corresponding semaphore of processing task is added to the pending queue；
When detecting echo signal amount in the pending queue, the data processing platform (DPP) is used for the echo signal
Amount is transferred to the execution queue, and executes target data corresponding with the echo signal amount and handle task；
If the target data processing task execution is abnormal, the controlling equipment is also used to again add the echo signal amount
To the pending queue.
11. system as claimed in claim 10, which is characterized in that the system is for realizing any one of such as claim 2 to 9
The method of the execution data processing task.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910327896.7A CN110309024B (en) | 2019-04-23 | 2019-04-23 | Data processing system and method for executing data processing task |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910327896.7A CN110309024B (en) | 2019-04-23 | 2019-04-23 | Data processing system and method for executing data processing task |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110309024A true CN110309024A (en) | 2019-10-08 |
| CN110309024B CN110309024B (en) | 2023-07-18 |
Family
ID=68074529
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910327896.7A Active CN110309024B (en) | 2019-04-23 | 2019-04-23 | Data processing system and method for executing data processing task |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110309024B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110888760A (en) * | 2019-11-26 | 2020-03-17 | 中国工商银行股份有限公司 | Data recovery method and device, and data processing method and device |
| CN111209112A (en) * | 2019-12-31 | 2020-05-29 | 杭州迪普科技股份有限公司 | Exception handling method and device |
| CN111562969A (en) * | 2020-07-15 | 2020-08-21 | 百度在线网络技术（北京）有限公司 | Intelligent contract implementation method, device, equipment and medium for block chain |
| CN111625391A (en) * | 2020-05-29 | 2020-09-04 | 北京思特奇信息技术股份有限公司 | Task processing method and system and electronic equipment |
| CN111930490A (en) * | 2020-09-25 | 2020-11-13 | 武汉中科通达高新技术股份有限公司 | Streaming media task management method and device |
| CN112416589A (en) * | 2020-11-21 | 2021-02-26 | 广州西麦科技股份有限公司 | Method for timing operation peak-shifting execution of operation and maintenance platform |
| CN112965799A (en) * | 2021-03-05 | 2021-06-15 | 北京百度网讯科技有限公司 | Task state prompting method and device, electronic equipment and medium |
| WO2021147289A1 (en) * | 2020-01-22 | 2021-07-29 | 北京百度网讯科技有限公司 | Stream computing job processing method, stream computing system, and electronic device |
| CN113672358A (en) * | 2020-05-15 | 2021-11-19 | 北京沃东天骏信息技术有限公司 | Timing task processing method, device and system, electronic equipment and storage medium |
| CN113873033A (en) * | 2021-09-27 | 2021-12-31 | 江苏方天电力技术有限公司 | Intelligent edge computing gateway platform with fault-tolerant function |
| CN113975815A (en) * | 2021-11-04 | 2022-01-28 | 上海鱼尔网络科技有限公司 | Task transfer method, system, device, equipment, storage medium and program product |
| CN116501477A (en) * | 2023-06-28 | 2023-07-28 | 中国电子科技集团公司第十五研究所 | Automatic data processing method, device and equipment |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH10301805A (en) * | 1997-04-24 | 1998-11-13 | Mitsubishi Electric Corp | Program trace device |
| WO2013110816A2 (en) * | 2012-01-27 | 2013-08-01 | Tymis | Method of using a shared memory |
| US20140310438A1 (en) * | 2013-04-10 | 2014-10-16 | Wind River Systems, Inc. | Semaphore with Timeout and Lock-Free Fast Path for Message Passing Architectures |
| WO2014173339A1 (en) * | 2013-08-07 | 2014-10-30 | 中兴通讯股份有限公司 | Task scheduling service system and method |
| WO2016112701A1 (en) * | 2015-01-16 | 2016-07-21 | 华为技术有限公司 | Method and device for task scheduling on heterogeneous multi-core reconfigurable computing platform |
| JP2017033208A (en) * | 2015-07-31 | 2017-02-09 | 株式会社日立超エル・エス・アイ・システムズ | Api for preventing propagation of failure on normal operation unit to safety unit and processing unit thereof |
| CN106471486A (en) * | 2014-04-30 | 2017-03-01 | 甲骨文国际公司 | System and method for supporting self adaptation self-adjusting locking mechanism in transaction middleware machine environment |
| CN106681836A (en) * | 2016-12-28 | 2017-05-17 | 华为技术有限公司 | Creating method and device of signal quantity |
| CN106844151A (en) * | 2017-01-04 | 2017-06-13 | 南京国电南自电网自动化有限公司 | A kind of network task method for detecting abnormality of VxWorks system |
| CN106933681A (en) * | 2017-02-05 | 2017-07-07 | 深圳怡化电脑股份有限公司 | A kind of many object blocking methods and its system |
| CN108536532A (en) * | 2018-04-23 | 2018-09-14 | 中国农业银行股份有限公司 | A kind of batch tasks processing method and system |
| CN109634733A (en) * | 2018-12-13 | 2019-04-16 | 成都四方伟业软件股份有限公司 | Task scheduling and managing method, device and operation management server |
-
2019
- 2019-04-23 CN CN201910327896.7A patent/CN110309024B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH10301805A (en) * | 1997-04-24 | 1998-11-13 | Mitsubishi Electric Corp | Program trace device |
| WO2013110816A2 (en) * | 2012-01-27 | 2013-08-01 | Tymis | Method of using a shared memory |
| US20140310438A1 (en) * | 2013-04-10 | 2014-10-16 | Wind River Systems, Inc. | Semaphore with Timeout and Lock-Free Fast Path for Message Passing Architectures |
| WO2014173339A1 (en) * | 2013-08-07 | 2014-10-30 | 中兴通讯股份有限公司 | Task scheduling service system and method |
| CN106471486A (en) * | 2014-04-30 | 2017-03-01 | 甲骨文国际公司 | System and method for supporting self adaptation self-adjusting locking mechanism in transaction middleware machine environment |
| WO2016112701A1 (en) * | 2015-01-16 | 2016-07-21 | 华为技术有限公司 | Method and device for task scheduling on heterogeneous multi-core reconfigurable computing platform |
| JP2017033208A (en) * | 2015-07-31 | 2017-02-09 | 株式会社日立超エル・エス・アイ・システムズ | Api for preventing propagation of failure on normal operation unit to safety unit and processing unit thereof |
| CN106681836A (en) * | 2016-12-28 | 2017-05-17 | 华为技术有限公司 | Creating method and device of signal quantity |
| CN106844151A (en) * | 2017-01-04 | 2017-06-13 | 南京国电南自电网自动化有限公司 | A kind of network task method for detecting abnormality of VxWorks system |
| CN106933681A (en) * | 2017-02-05 | 2017-07-07 | 深圳怡化电脑股份有限公司 | A kind of many object blocking methods and its system |
| CN108536532A (en) * | 2018-04-23 | 2018-09-14 | 中国农业银行股份有限公司 | A kind of batch tasks processing method and system |
| CN109634733A (en) * | 2018-12-13 | 2019-04-16 | 成都四方伟业软件股份有限公司 | Task scheduling and managing method, device and operation management server |
Non-Patent Citations (1)
| Title |
|---|
| 文豪等: "无源节点上一种多任务内核的设计与实现", 《智能计算机与应用》 * |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110888760A (en) * | 2019-11-26 | 2020-03-17 | 中国工商银行股份有限公司 | Data recovery method and device, and data processing method and device |
| CN111209112A (en) * | 2019-12-31 | 2020-05-29 | 杭州迪普科技股份有限公司 | Exception handling method and device |
| WO2021147289A1 (en) * | 2020-01-22 | 2021-07-29 | 北京百度网讯科技有限公司 | Stream computing job processing method, stream computing system, and electronic device |
| CN113672358A (en) * | 2020-05-15 | 2021-11-19 | 北京沃东天骏信息技术有限公司 | Timing task processing method, device and system, electronic equipment and storage medium |
| CN111625391A (en) * | 2020-05-29 | 2020-09-04 | 北京思特奇信息技术股份有限公司 | Task processing method and system and electronic equipment |
| CN111562969A (en) * | 2020-07-15 | 2020-08-21 | 百度在线网络技术（北京）有限公司 | Intelligent contract implementation method, device, equipment and medium for block chain |
| CN111562969B (en) * | 2020-07-15 | 2020-10-20 | 百度在线网络技术（北京）有限公司 | Intelligent contract implementation method, device, equipment and medium for block chain |
| CN111930490A (en) * | 2020-09-25 | 2020-11-13 | 武汉中科通达高新技术股份有限公司 | Streaming media task management method and device |
| CN111930490B (en) * | 2020-09-25 | 2021-06-15 | 武汉中科通达高新技术股份有限公司 | Streaming media task management method and device |
| CN112416589A (en) * | 2020-11-21 | 2021-02-26 | 广州西麦科技股份有限公司 | Method for timing operation peak-shifting execution of operation and maintenance platform |
| CN112965799A (en) * | 2021-03-05 | 2021-06-15 | 北京百度网讯科技有限公司 | Task state prompting method and device, electronic equipment and medium |
| CN112965799B (en) * | 2021-03-05 | 2023-08-18 | 北京百度网讯科技有限公司 | Task state prompting method and device, electronic equipment and medium |
| CN113873033A (en) * | 2021-09-27 | 2021-12-31 | 江苏方天电力技术有限公司 | Intelligent edge computing gateway platform with fault-tolerant function |
| CN113873033B (en) * | 2021-09-27 | 2022-12-13 | 江苏方天电力技术有限公司 | Intelligent edge computing gateway platform with fault-tolerant function |
| CN113975815A (en) * | 2021-11-04 | 2022-01-28 | 上海鱼尔网络科技有限公司 | Task transfer method, system, device, equipment, storage medium and program product |
| CN116501477A (en) * | 2023-06-28 | 2023-07-28 | 中国电子科技集团公司第十五研究所 | Automatic data processing method, device and equipment |
| CN116501477B (en) * | 2023-06-28 | 2023-09-15 | 中国电子科技集团公司第十五研究所 | Automatic data processing method, device and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110309024B (en) | 2023-07-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110309024A (en) | Data processing system and its method for executing data processing task | |
| CN105357038B (en) | Monitor the method and system of cluster virtual machine | |
| US8949671B2 (en) | Fault detection, diagnosis, and prevention for complex computing systems | |
| US8930757B2 (en) | Operations management apparatus, operations management method and program | |
| US20130246001A1 (en) | Device monitoring system and method | |
| US20140068613A1 (en) | Non-transitory computer-readable storage medium, information processing apparatus and scheduling method | |
| US20150012635A1 (en) | Systems and methods for organic knowledge base runbook automation | |
| CN102947804B (en) | Key configuration when trace data disposal system | |
| CN104601668B (en) | Data push method, device and system based on condition managing | |
| WO2007139612A2 (en) | Evaluating performance of software application | |
| US9092278B2 (en) | Determining the processing order of a plurality of events | |
| CN108508874A (en) | A kind of method and apparatus of monitoring equipment fault | |
| CN105930226A (en) | Data processing method and apparatus | |
| US20090307526A1 (en) | Multi-cpu failure detection/recovery system and method for the same | |
| US20150026694A1 (en) | Method of processing information, storage medium, and information processing apparatus | |
| KR101212496B1 (en) | Method of representing usage of monitoring resource, computing apparatus for performing the same and record medium recording program for implementing the method | |
| US20150281037A1 (en) | Monitoring omission specifying program, monitoring omission specifying method, and monitoring omission specifying device | |
| US20160266808A1 (en) | Information processing device, information processing method, and recording medium | |
| US20170068603A1 (en) | Information processing method and information processing apparatus | |
| US20140040191A1 (en) | Inventorying and copying file system folders and files | |
| CN110162443A (en) | More lot flow monitoring methods, device, computer equipment and storage medium | |
| CN109710679A (en) | Data pick-up method and device | |
| US9306894B1 (en) | Automatic detection and notification of backlogs and delays | |
| WO2016153377A1 (en) | Smart logging of trace data for storage systems | |
| EP3396553B1 (en) | Method and device for processing data after restart of node |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |