CN108536532B - Batch task processing method and system - Google Patents
Batch task processing method and system Download PDFInfo
- Publication number
- CN108536532B CN108536532B CN201810367717.8A CN201810367717A CN108536532B CN 108536532 B CN108536532 B CN 108536532B CN 201810367717 A CN201810367717 A CN 201810367717A CN 108536532 B CN108536532 B CN 108536532B
- Authority
- CN
- China
- Prior art keywords
- task
- batch
- queue
- processing
- processed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/48—Program initiating; Program switching, e.g. by interrupt
- G06F9/4806—Task transfer initiation or dispatching
- G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
- G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/54—Indexing scheme relating to G06F9/54
- G06F2209/548—Queue
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a batch task processing method and a system, wherein the method comprises the following steps: according to the obtained batch task information sent by the application system, a task submission request is created; generating a task message queue, and storing the task submission request to a to-be-processed queue of the task message queue; and acquiring batch task information corresponding to the task submitting request from the queue to be processed, acquiring corresponding batch data according to the batch task information, and processing the batch data to obtain a batch processing result. The embodiment of the invention realizes the universality of batch task processing and the aim of improving the processing efficiency.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a batch task processing method and system.
Background
In an enterprise, there are generally a plurality of application systems, which respectively correspond to different processing services and belong to different parts. With the advent of the big data era, the scale of application system service data is continuously enlarged, and in order to better utilize the service data, each application system tends to upload mass data to a platform system with a specific function for storage and analysis, so that a strategy for efficiently processing mass data of a large number of application systems in batch, namely a batch task processing strategy, is needed.
At present, a processing strategy for complex batch tasks is generally implemented based on an existing task scheduling framework, and processing of a large number of batch tasks is shared by nodes of a cluster to improve processing efficiency of the batch tasks. The existing task scheduling framework mainly comprises spring batch and TBschedule, but the existing batch task processing strategy also has certain defects, such as lack of flexibility, poor reusability, poor excellence in processing of high-level or complex queues and lack of a perfect exception handling mechanism, so that the existing batch task processing strategy has low processing efficiency and poor universality.
Disclosure of Invention
In view of the above problems, the present invention provides a batch task processing method and system, which achieve the purposes of processing batch tasks in a versatile manner and improving processing efficiency.
In order to achieve the purpose, the invention provides the following technical scheme:
a method of batch task processing, the method comprising:
according to the obtained batch task information sent by the application system, a task submission request is created;
generating a task message queue, and storing the task submission request to a to-be-processed queue of the task message queue;
and acquiring batch task information corresponding to the task submitting request from the queue to be processed, acquiring corresponding batch data according to the batch task information, and processing the batch data to obtain a batch processing result.
Preferably, the method further comprises the following steps:
if the task submitting request created according to the batch task information fails, storing the batch task information into task cache information to be processed;
and re-issuing the batch task information in the task cache queue to be processed to the queue to be processed of the task message queue within a preset time range.
Preferably, the obtaining batch task information corresponding to the task submission request in the queue to be processed, obtaining corresponding batch data according to the batch task information, and processing the batch data to obtain a batch processing result includes:
acquiring batch task information corresponding to the task submission request in the queue to be processed;
performing task identification on the batch task information to obtain a storage address of batch data corresponding to the task submission request;
obtaining the batch data according to the storage address of the batch data, and verifying the batch data;
and processing the successfully verified batch data to obtain a batch processing result.
Preferably, the method further comprises the following steps:
saving the task state of the batch task information in the task message queue;
and updating the task state in real time according to the verification result and the processing result of the batch of task information, and storing the updated task state.
Preferably, the method further comprises the following steps:
judging whether the time for processing the batch data exceeds a threshold value, if so, stopping processing the batch data;
and clearing the batch data which completes the batch data processing.
A batch task processing system, comprising:
the task monitoring module is used for creating and obtaining a task submission request according to the obtained batch task information sent by the application system;
the task queue module is used for generating a task message queue and storing the task submitting request to a to-be-processed queue of the task message queue;
and the task processing module is used for acquiring batch task information corresponding to the task submitting request from the queue to be processed, acquiring corresponding batch data according to the batch task information, and processing the batch data to obtain a batch processing result.
Preferably, the method further comprises the following steps:
the storage module is used for storing the batch task information into task cache information to be processed if the task submission request created according to the batch task information fails;
and the timing task module is used for re-issuing the batch task information in the task cache queue to be processed to the queue to be processed of the task message queue within a preset time range.
Preferably, the task processing module includes:
the task obtaining unit is used for obtaining batch task information corresponding to the task submitting request in the queue to be processed;
the task identification unit is used for carrying out task identification on the batch task information to obtain a storage address of batch data corresponding to the task submission request;
the data acquisition unit is used for acquiring the batch data according to the storage address of the batch data and verifying the batch data;
and the processing unit is used for processing the batch data successfully verified to obtain a batch processing result.
Preferably, the method further comprises the following steps:
the task state monitoring unit is used for storing the task states of the batch task information in the task information queue;
and the system is also used for updating the task state in real time according to the verification result and the processing result of the batch of task information and storing the updated task state.
Preferably, the method further comprises the following steps:
the timing scheduling unit is used for judging whether the time for processing the batch data exceeds a threshold value or not, and if so, stopping the processing of the batch data;
and the data cleaning unit is used for cleaning the batch data which is processed by the batch data.
Compared with the prior art, the invention provides a batch task processing method and a system, the task submission requests corresponding to batch task information are stored in the task message queue, the task information is maintained by using the task message queue, the task processing adopts a parallelization mode to execute the processes of obtaining, identifying, processing and the like, and finally the batch processing results corresponding to the batch tasks are obtained. Because the message queue is adopted for batch task processing, various message queue products can be supported, and the universality and high processing efficiency of batch task processing are ensured.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flowchart illustrating a batch task processing method according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating a platform system process according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a batch task processing system according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first" and "second," and the like in the description and claims of the present invention and the above-described drawings are used for distinguishing between different objects and not for describing a particular order. Furthermore, the terms "comprising" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not set forth for a listed step or element but may include steps or elements not listed.
An embodiment of the present invention provides a batch task processing method, referring to fig. 1, including:
and S11, creating and obtaining a task submission request according to the obtained batch task information sent by the application system.
The user accessing the platform system generally refers to other systems in the enterprise, such as archives, finance, business management and the like, and is generally referred to as an application system. The platform system refers to a system for providing certain service for other application systems in an enterprise, such as file management, single sign-on and the like. A batch refers to a collection of data, usually including relevant description information of the batch, and is the minimum unit of data transmission between a sender and a receiver, i.e., the batch of data is either processed successfully or failed.
The application system transmits the batch (containing batch task information and batch data) to the platform system, wherein the transmission mode can adopt some batch data transmission mode, such as GTP. And batch task information is transmitted to a task monitoring module of the platform system, and batch data is stored in a storage module for shared storage. The batch task information is description information of batch data, and usually one batch of batch data corresponds to one batch task information and describes a plurality of pieces of batch data, and batch data storage addresses, application system numbers and other common information.
The task listening module executes step S11 and submits a task submission request to the task queue module.
And S12, generating a task message queue, and storing the task submission request to a to-be-processed queue of the task message queue.
The task queue module executes step S12 and includes the pending queue and the exception retry queue in the task message queue. The message queue can adopt a distributed message queue Kafka, and can also adopt other message queue products such as RabbitMQ, Redis, zeroMQ and ActiveMQ. A typical Kafka cluster includes several producers (which may be Page views generated by a web front end, or server logs, system CPUs, memories, etc.), several brokers (Kafka supports horizontal extension, generally the larger the number of brokers, the higher the cluster throughput), several Consumer groups, and a Zookeper cluster. Kafka manages cluster configuration through Zookeeper, elects leader, and rebalance when Consumer Group changes. The Producer publishes the message to the broker using push mode, and the Consumer subscribes and consumes the message from the broker using pull mode.
S13, batch task information corresponding to the task submitting request is obtained in the queue to be processed, corresponding batch data are obtained according to the batch task information, and the batch data are processed to obtain a batch processing result.
The task processing module executes step S13 to perform the acquisition task and process it. The task processing module adopts a plurality of mutually independent nodes to read related task information from a to-be-processed queue of the task message queue in parallel or an abnormal retry queue, performs task identification, extracts batch data required to be processed by the task, namely an application data storage address, acquires corresponding data according to the address, performs a series of subsequent data verification, service logic processing and the like, and generates a batch processing result.
The invention provides a batch task processing method, which is characterized in that task submission requests corresponding to batch task information are stored in a task information queue, the task information is maintained by using the task information queue, and the task processing adopts a parallelization mode to execute the processes of acquisition, identification, processing and the like, so that a batch processing result corresponding to the batch tasks is finally obtained. Because the message queue is adopted for batch task processing, various message queue products can be supported, and the universality and high processing efficiency of batch task processing are ensured.
When the task monitoring module creates a task submitting request according to the batch task information, the task is submitted to a to-be-processed queue of the task information queue, if the task is failed to be created, the task information is stored in the to-be-processed task cache information, and the information is re-issued to the to-be-processed queue of the task information queue by the timing task module after the to-be-processed task is cached.
Therefore, the timing task module re-issues the batch task information in the task buffer queue to be processed to the queue to be processed of the task message queue within a preset time range.
In another embodiment of the present invention, a method for batch data processing is further provided, including:
acquiring batch task information corresponding to the task submission request in the queue to be processed;
performing task identification on the batch task information to obtain a storage address of batch data corresponding to the task submission request;
obtaining the batch data according to the storage address of the batch data, and verifying the batch data;
and processing the successfully verified batch data to obtain a batch processing result.
An execution node of the task processing module acquires information of a task to be processed from the task message queue; and extracting an application data storage address required to be processed by the object, if the extraction is successful, acquiring corresponding data from the storage module according to the address, and if the extraction is successful, performing a series of work such as data verification, business logic processing and the like to generate a batch processing result so as to transmit the batch processing result to other subsystems of the platform for other functional processing.
If the data storage address is not successfully extracted, or the data is not successfully verified, or the data is unsuccessfully processed, judging whether the retry times exceed a preset threshold value according to the retry mark of the task, and if so, updating the failure state of the task to a task queue module for task state management. If not, the generate retry is to be committed to the exception retry queue.
In another embodiment of the present invention, the method further comprises:
judging whether the time for processing the batch data exceeds a threshold value, if so, stopping processing the batch data;
specifically, the step is executed by a timing scheduling unit, and the step is used for initiating a timing task for monitoring and managing the task state at a fixed time; acquiring a task state list from a task queue module; judging whether a next unprocessed task state exists, if so, reading the task state, judging whether the state is successful, if so, cleaning application data in the task successful state, acquiring an application data storage address processed by the person according to task information, cleaning, tracking the next task after the completion, if calculating that the execution time of the task state exceeds a threshold value, judging as overtime, terminating the overtime task, and setting the task state as failure.
Specifically, referring to fig. 2, which is a schematic diagram of a processing process of a platform system according to an embodiment of the present invention, after a platform system receives data uploaded by an application system, a task monitoring module analyzes task information according to application system access information and adds the task information to a to-be-processed task queue in a distributed message queue; the task processing module adopts a plurality of mutually independent nodes to acquire messages to be processed from the message queue in parallel, performs task identification, analyzes a data storage address of an application system, acquires data, performs batch data service logic processing, and records task state result information, provides a perfect exception handling mechanism in the process, and adopts a retry mechanism to ensure the correct execution of batch tasks and a comprehensive tracking and cleaning mechanism to improve the accuracy of exception positioning; by monitoring the task state information at regular time, the execution state of the task is tracked, the completed and overtime task is processed, and corresponding resources are released
In fig. 2, the application system sends the batch information to the platform system in units of batches, and the platform system includes a batch task processing subsystem, other platform subsystems, and a timing scheduler.
And receiving batch task information by a task monitoring module in the batch task processing subsystem, creating a corresponding task request, issuing a message to a to-be-processed queue in the task queue module if the creation is successful, storing the message to a to-be-processed task cache in shared storage if the creation is failed, and simultaneously, storing application data in the shared storage as a storage form of batch data.
And then each node in the task processing module pulls a message in the task message queue for identification, data acquisition, data checksum processing and the like, if a certain step fails, the processing abnormal message of the task is stored in an abnormal retry queue in the task queue module, and the task state management in the module is updated. And if the task processing module obtains the batch processing result, sending the batch processing result to other subsystems of the platform for other processing.
Meanwhile, the system also comprises a timing task module which is scheduled by a timing scheduler at regular time and mainly comprises task state monitoring, namely acquiring a task state from task state management of the task queue module; processing the tasks to be processed, namely recreating the tasks from the shared and stored cache of the tasks to be processed; application data cleaning, namely cleaning the processed data from the application data in the shared cache,
the embodiment of the invention provides and realizes a general solution for processing the batch tasks by using the message queue products, can support various message queue products (such as RabbitMQ, Redis, ZeroMQ, ActiveMQ, Kafka and the like), can flexibly use the solution by systems with batch processing requirements, and has higher practical value.
The task processing module also adopts mutually independent nodes to carry out service logic processing in parallel, the nodes do not influence each other, and the fault of any node does not influence other nodes. Meanwhile, a task state monitoring function is provided, successful tasks are cleared in time, overtime tasks are suspended, and accordingly batch resource waste is avoided.
In the embodiment of the invention, the distributed message queue product is fully utilized to release and obtain the batch task information, and the task processing module adopts a parallelization mode to obtain and execute the batch task information from the message queue, so that the method has good processing performance and can fully ensure the system performance under high load. The method can automatically perform fault-tolerant processing according to the calling return result and perform task retry, effectively makes up the defects of the message queue product in the aspects of fault detection and abnormal recovery, and avoids batch task failure caused by accidental factors.
Correspondingly, in another embodiment of the present invention, a batch task processing system is further provided, referring to fig. 3, including:
the task monitoring module 10 creates and obtains a task submission request according to the obtained batch task information sent by the application system;
the task queue module 20 generates a task message queue and stores the task submission request to a to-be-processed queue of the task message queue;
the task processing module 30 obtains batch task information corresponding to the task submission request from the queue to be processed, obtains corresponding batch data according to the batch task information, and processes the batch data to obtain a batch processing result.
Optionally, the method further comprises:
the storage module is used for storing the batch task information into task cache information to be processed if the task submission request created according to the batch task information fails;
and the timing task module is used for re-issuing the batch task information in the task cache queue to be processed to the queue to be processed of the task message queue within a preset time range.
Optionally, the task processing module includes:
the task obtaining unit is used for obtaining batch task information corresponding to the task submitting request in the queue to be processed;
the task identification unit is used for carrying out task identification on the batch task information to obtain a storage address of batch data corresponding to the task submission request;
the data acquisition unit is used for acquiring the batch data according to the storage address of the batch data and verifying the batch data;
and the processing unit is used for processing the batch data successfully verified to obtain a batch processing result.
Optionally, the method further comprises:
the task state monitoring unit is used for storing the task states of the batch task information in the task information queue;
and the system is also used for updating the task state in real time according to the verification result and the processing result of the batch of task information and storing the updated task state.
Optionally, the method further comprises:
the timing scheduling unit is used for judging whether the time for processing the batch data exceeds a threshold value or not, and if so, stopping the processing of the batch data;
and the data cleaning unit is used for cleaning the batch data which is processed by the batch data.
The invention provides a batch task processing system, which stores task submission requests corresponding to batch task information into a task information queue, maintains the task information by using the task information queue, and executes processes of acquisition, identification, processing and the like by adopting a parallelization mode in task processing to finally obtain batch processing results corresponding to the batch tasks. Because the message queue is adopted for batch task processing, various message queue products can be supported, and the universality and high processing efficiency of batch task processing are ensured.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (6)
1. A method for batch task processing, the method comprising:
according to the obtained batch task information sent by the application system, a task submission request is created;
generating a task message queue, and storing the task submission request to a to-be-processed queue of the task message queue, wherein the task message queue comprises the to-be-processed queue and an abnormal retry queue;
acquiring batch task information corresponding to the task submission request in the queue to be processed, acquiring corresponding batch data according to the batch task information, and processing the batch data to obtain a batch processing result, wherein the batch task information corresponding to the task submission request is acquired in the queue to be processed, the corresponding batch data is acquired according to the batch task information, and the batch data is processed to obtain a batch processing result, and the method comprises the following steps: acquiring batch task information corresponding to the task submission request in the queue to be processed; performing task identification on the batch task information to obtain a storage address of batch data corresponding to the task submission request; obtaining the batch data according to the storage address of the batch data, and verifying the batch data; processing the successfully verified batch data to obtain a batch processing result;
saving the task state of the batch task information in the task message queue;
and updating the task state in real time according to the verification result and the processing result of the batch of task information, and storing the updated task state.
2. The method of claim 1, further comprising:
if the task submitting request created according to the batch task information fails, storing the batch task information into task cache information to be processed;
and re-issuing the batch task information in the task cache queue to be processed to the queue to be processed of the task message queue within a preset time range.
3. The method of claim 1, further comprising:
judging whether the time for processing the batch data exceeds a threshold value, if so, stopping processing the batch data;
and clearing the batch data which completes the batch data processing.
4. A batch task processing system, comprising:
the task monitoring module is used for creating and obtaining a task submission request according to the obtained batch task information sent by the application system;
the task queue module is used for generating a task message queue and storing the task submitting request to a to-be-processed queue of the task message queue, wherein the task message queue comprises the to-be-processed queue and an abnormal retry queue;
the task processing module is configured to obtain batch task information corresponding to the task submission request in the queue to be processed, obtain corresponding batch data according to the batch task information, and process the batch data to obtain a batch processing result, where the task processing module includes: the task obtaining unit is used for obtaining batch task information corresponding to the task submitting request in the queue to be processed; the task identification unit is used for carrying out task identification on the batch task information to obtain a storage address of batch data corresponding to the task submission request; the data acquisition unit is used for acquiring the batch data according to the storage address of the batch data and verifying the batch data; the processing unit is used for processing the batch data successfully verified to obtain a batch processing result;
the task state monitoring unit is used for storing the task states of the batch task information in the task information queue;
and the system is also used for updating the task state in real time according to the verification result and the processing result of the batch of task information and storing the updated task state.
5. The system of claim 4, further comprising:
the storage module is used for storing the batch task information into task cache information to be processed if the task submission request created according to the batch task information fails;
and the timing task module is used for re-issuing the batch task information in the task cache queue to be processed to the queue to be processed of the task message queue within a preset time range.
6. The system of claim 4, further comprising:
the timing scheduling unit is used for judging whether the time for processing the batch data exceeds a threshold value or not, and if so, stopping the processing of the batch data;
and the data cleaning unit is used for cleaning the batch data which is processed by the batch data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810367717.8A CN108536532B (en) | 2018-04-23 | 2018-04-23 | Batch task processing method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810367717.8A CN108536532B (en) | 2018-04-23 | 2018-04-23 | Batch task processing method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108536532A CN108536532A (en) | 2018-09-14 |
| CN108536532B true CN108536532B (en) | 2021-06-22 |
Family
ID=63477473
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810367717.8A Active CN108536532B (en) | 2018-04-23 | 2018-04-23 | Batch task processing method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108536532B (en) |
Families Citing this family (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109558789A (en) * | 2018-10-09 | 2019-04-02 | 珠海亿联德源信息技术有限公司 | A kind of biological characteristic system for rapidly identifying based on distributed computing |
| CN109740934A (en) * | 2018-12-29 | 2019-05-10 | 深圳前海微众银行股份有限公司 | O&M job control method, device, equipment and computer readable storage medium |
| CN110109749A (en) * | 2019-03-29 | 2019-08-09 | 北京奇安信科技有限公司 | A kind of the computing resource method for cleaning and device of distributed system |
| CN110309024B (en) * | 2019-04-23 | 2023-07-18 | 网宿科技股份有限公司 | Data processing system and method for executing data processing task |
| CN110297711B (en) * | 2019-05-16 | 2024-01-19 | 平安科技（深圳）有限公司 | Batch data processing method, device, computer equipment and storage medium |
| CN110381150B (en) * | 2019-07-24 | 2021-12-17 | 中国工商银行股份有限公司 | Data processing method and device on block chain, electronic equipment and storage medium |
| CN110569120B (en) * | 2019-09-11 | 2022-03-04 | 华云数据控股集团有限公司 | Processing method and device for timing task |
| CN110768914B (en) * | 2019-10-23 | 2022-03-29 | 上海交通大学 | Decentralized Internet of things gateway system based on semantic scene instance migration |
| CN111367688A (en) * | 2020-02-28 | 2020-07-03 | 京东数字科技控股有限公司 | Service data processing method and device |
| CN111400390B (en) * | 2020-04-08 | 2023-11-17 | 上海东普信息科技有限公司 | Data processing method and device |
| CN113568761B (en) * | 2020-04-28 | 2023-06-27 | 中国联合网络通信集团有限公司 | Data processing method, device, equipment and storage medium |
| CN112631943B (en) * | 2020-12-31 | 2024-01-16 | 中国农业银行股份有限公司 | Recognition method and device for computing software |
| CN113609031B (en) * | 2021-07-08 | 2024-03-29 | 深圳市晨北科技有限公司 | Data cleaning model construction method, data cleaning method, related equipment and medium |
| CN113778650A (en) * | 2021-09-14 | 2021-12-10 | 北京沃东天骏信息技术有限公司 | Task scheduling method and device, electronic equipment and storage medium |
| CN114153573A (en) * | 2021-10-29 | 2022-03-08 | 深圳精智达技术股份有限公司 | Multitasking method based on single thread pool and related device |
| CN114257582A (en) * | 2021-12-17 | 2022-03-29 | 中国农业银行股份有限公司 | Batch job processing method, distributed system and batch job processing architecture |
| CN116132309B (en) * | 2023-02-08 | 2024-04-26 | 浪潮通信信息系统有限公司 | Batch business processing method and system for network management resources |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7681087B2 (en) * | 2005-12-14 | 2010-03-16 | Business Objects Software Ltd. | Apparatus and method for persistent report serving |
-
2018
- 2018-04-23 CN CN201810367717.8A patent/CN108536532B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7681087B2 (en) * | 2005-12-14 | 2010-03-16 | Business Objects Software Ltd. | Apparatus and method for persistent report serving |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108536532A (en) | 2018-09-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108536532B (en) | Batch task processing method and system | |
| CN108874558B (en) | Message subscription method of distributed transaction, electronic device and readable storage medium | |
| WO2021190087A1 (en) | Task execution method, device and system, and server | |
| CN110968478B (en) | Log acquisition method, server and computer storage medium | |
| CN111400011B (en) | Real-time task scheduling method, system, equipment and readable storage medium | |
| CN111210340B (en) | Automatic task processing method, device, server and storage medium | |
| US11321115B2 (en) | Scalable and dynamic data collection and processing | |
| CN111835467B (en) | Message sending method, device, computer equipment and storage medium | |
| CN112099935A (en) | Task processing method and device | |
| CN113672500B (en) | Deep learning algorithm testing method and device, electronic device and storage medium | |
| CN109426550A (en) | The dispatching method and equipment of resource | |
| CN112306719A (en) | Task scheduling method and device | |
| US20170078049A1 (en) | Freshness-sensitive message delivery | |
| CN111367792A (en) | Test method, test device, storage medium and electronic equipment | |
| CN111831408A (en) | Asynchronous task processing method and device, electronic equipment and medium | |
| CN114817050A (en) | Task execution method and device, electronic equipment and computer readable storage medium | |
| CN107819855B (en) | Message distribution method and device | |
| CN113553067A (en) | REST-based multi-frame machine learning model deployment prediction method | |
| CN111597056B (en) | Distributed scheduling method, system, storage medium and equipment | |
| CN112948096A (en) | Batch scheduling method, device and equipment | |
| US11379694B2 (en) | Scalable and dynamic data collection and processing | |
| CN110019045B (en) | Log floor method and device | |
| CN112825525A (en) | Method and apparatus for processing transactions | |
| CN114390104A (en) | Process forensics system, method, apparatus, computer device and medium | |
| CN111401819B (en) | Intersystem data pushing method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20221104 Address after: 100005 No. 69, inner main street, Dongcheng District, Beijing, Jianguomen Patentee after: AGRICULTURAL BANK OF CHINA Patentee after: Agricultural Bank of China Financial Technology Co.,Ltd. Address before: 100005 No. 69, inner main street, Dongcheng District, Beijing, Jianguomen Patentee before: AGRICULTURAL BANK OF CHINA |