CN115328624A - Configuration-based triggered task scheduling device - Google Patents

Configuration-based triggered task scheduling device Download PDF

Info

Publication number
CN115328624A
CN115328624A CN202210735209.7A CN202210735209A CN115328624A CN 115328624 A CN115328624 A CN 115328624A CN 202210735209 A CN202210735209 A CN 202210735209A CN 115328624 A CN115328624 A CN 115328624A
Authority
CN
China
Prior art keywords
task
processing
table processing
text
module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210735209.7A
Other languages
Chinese (zh)
Inventor
苗青
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Bank of China Financial Technology Co Ltd
Original Assignee
Bank of China Financial Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Bank of China Financial Technology Co Ltd filed Critical Bank of China Financial Technology Co Ltd
Priority to CN202210735209.7A priority Critical patent/CN115328624A/en
Publication of CN115328624A publication Critical patent/CN115328624A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
    • G06F11/302—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a software system
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3055—Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Quality & Reliability (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention provides a configuration-based trigger type task scheduling device, which comprises a driving module, an ETL (extract transform load) task editing module, a task processing module and a flow control polling module, wherein the ETL task editing module is used for editing tasks; the driving module is used for recording the text state parameters in the process of uploading the upstream text to the HDFS distributed file system; the ETL task editing module is used for generating a front-end interface so as to edit or query the table processing configuration information through the front-end interface; the task processing module is used for finishing the processing of at least one table processing task according to the table processing configuration information and recording the table processing state information corresponding to each table processing task; the flow control polling module is used for driving the task processing module to complete the processing of at least one table processing task under the condition of monitoring the entry of the table processing task based on the text state parameter, and monitoring the state parameter of each table processing task so as to schedule the table processing task according to the state parameter, thereby realizing the effects of simplifying operation, improving efficiency and reducing development cost and time.

Description

Configuration-based triggered task scheduling device
Technical Field
The invention relates to the technical field of data processing, in particular to a configuration-based trigger type task scheduling device.
Background
ETL (extract-transform-load) is used to describe the process of extracting, transforming, and loading data from a source to a destination. The back-line warehouse products of large enterprises usually collect data of each upstream business system as a data source of the system. Common traditional data transmission methods are text or other heterogeneous data source import and export tools are used for storing different kinds of data in new storage media. For this purpose, it is usually necessary to perform a related ETL process to convert the upstream data into corresponding data that can be used in warehouse-type products.
In the existing scheme, big data task scheduling middleware such as an Apache Oozie workflow scheduling system, an airflow workflow platform, a dolpthin schedule visual distributed big data workflow task scheduling system and the like is mostly used, development of related ETL tasks is completed by combining a big data development engine and an ETL processing tool, for example, a heterogeneous data source importing tool is used for completing a data conversion data table process, however, in the process, on one hand, manual coding is usually required to be performed by a task creation object to automatically complete development of the related ETL tasks, different codes are required to be written for different application scenes, when the ETL task requirements change, operations such as updating of the codes are also required, on the other hand, such as a datax offline data synchronization tool provides rich heterogeneous data source data extraction functions, but for a database without a main key, functions such as data updating and data quality check cannot be automatically completed, and therefore, a complete task scheduling process needs to be additionally designed by using datax, thereby resulting in cumbersome operation, low efficiency, and influence on business processing, and development cost and time and development cost are increased.
Disclosure of Invention
The invention provides a configuration-based trigger type task scheduling device, which is used for overcoming the defects that in the prior art, the operation is complicated, the efficiency is low, the service processing is influenced and the development cost and time are increased due to the fact that code development of related ETL tasks needs to be carried out manually, and the configuration-based trigger type task scheduling device can be adapted to various service scenes and requirements, enables services to define processes by themselves and further reduces the development cost and time.
The invention provides a trigger type task scheduling device based on configuration, which comprises: the system comprises a driving module, an ETL task editing module, a task processing module and a flow control polling module;
the driving module is used for recording the text state parameters in the process of uploading the upstream text to the HDFS distributed file system;
the ETL task editing module is used for generating a front-end interface so as to edit or query the table processing configuration information through the front-end interface;
the task processing module is used for finishing the processing of at least one table processing task according to the table processing configuration information and recording the table processing state information corresponding to each table processing task;
and the flow control polling module is used for driving the task processing module to complete the processing of at least one table processing task under the condition that the table processing task is monitored to enter based on the text state parameters, and monitoring the state parameters of each table processing task so as to schedule the table processing task according to the state parameters.
According to the configuration-based triggered task scheduling device provided by the invention, the flow control polling module comprises a text monitor, a table processing monitor and a task monitor, and the text monitor, the table processing monitor and the task monitor form a flow processing frame.
According to the configuration-based triggered task scheduling device provided by the invention, the text listener is used for monitoring the text state table in the text state parameters, and driving the task processing module to complete the processing of at least one table processing task and trigger the table listener and the table processing process listener to monitor under the condition that the table processing triggering condition is reached based on the monitoring of the text state table.
According to the configuration-based triggered task scheduling device provided by the invention, the text listener is further configured to update the text abnormal state when the text abnormal state in the text state table is monitored.
According to the configuration-based triggered task scheduling device provided by the invention, the table processing monitor is used for monitoring the table processing state table and triggering the processing of the next-level table processing task according to the monitoring result and the memory resource use condition.
According to the configuration-based triggered task scheduling device provided by the invention, the task monitor is used for monitoring the memory resource use condition of the node corresponding to each table processing task, performing cross-node table processing task scheduling under the condition that the memory resource use condition reaches the preset maximum load limit, and performing concurrency control according to the scheduled memory resource use condition.
According to the configuration-based triggered task scheduling device provided by the invention, the task listener is further configured to allow submission of a new form processing task when the scheduled memory resource is less than a preset limited memory resource.
According to the configuration-based triggered task scheduling device provided by the invention, the driving module is also used for connecting the HDFS distributed file system through a socket interface so as to realize the metadata information polling function of the HDFS file.
According to the configuration-based triggered task scheduling device provided by the invention, the table processing configuration information comprises at least one of a scheduling service and running server list, a processing relation parameter of a file and a table, and a mapping relation parameter of the file and the table.
According to the configuration-based trigger type task scheduling device provided by the invention, the table processing task comprises at least one of a data format conversion task, a column number verification task, a data type verification task, a main key verification task, a data line number passing rate verification task, a table analysis task, a data increase amount conversion task and a partition table analysis task.
The invention provides a configuration-based trigger type task scheduling device, which comprises a driving module, an ETL (extract transform load) task editing module, a task processing module and a flow control polling module; the driving module is used for recording a text state parameter in the process of uploading an upstream text to the HDFS distributed file system; the ETL task editing module is used for generating a front-end interface so as to edit or query the table processing configuration information through the front-end interface; the task processing module is used for finishing the processing of at least one table processing task according to the table processing configuration information and recording the table processing state information corresponding to each table processing task; the flow control polling module is used for driving the task processing module to complete the processing of at least one table processing task under the condition of monitoring the entry of the table processing task based on the text state parameter, monitoring the state parameter of each table processing task and scheduling the table processing task according to the state parameter, so that the driving module, the ETL task editing module and the flow control polling module support the data circulation among parent-child flows, the processing process, the processing method and the task parameter of the ETL task in the flow support the self-defined setting of the ETL task editing module, the circulation of the whole flow does not need to be processed according to business logic and is completely isolated from the requirement corresponding to the flow, and therefore, the flow control polling module can be adapted to various business scenes and requirements, does not need to manually program codes according to different business scenes and requirements, and achieves the effects of simplifying operation, improving efficiency and reducing development cost and time.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic structural diagram of a triggered task scheduler based on configuration provided in the present invention;
fig. 2 is a schematic structural diagram of the flow control polling module provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The configuration-based triggered task scheduler of the present invention is described below in conjunction with fig. 1-2.
Fig. 1 is a schematic structural diagram of a configuration-based triggered task scheduling device according to the present invention, and as shown in fig. 1, the configuration-based triggered task scheduling device includes a driver module 10, an ETL task editing module 20, a task processing module 30, and a flow control polling module 40.
The driving module 10 is configured to record a text status parameter in a process of uploading an upstream text to the HDFS distributed file system.
Specifically, the upstream text refers to data which is extracted from a service system and transmitted in a text format, and the service system needs to perform data extraction, data cleaning and conversion, and data loading.
The text state parameters refer to the time for starting uploading of the upstream text, the uploading progress, the uploading mode of the upstream text, abnormal condition parameters in the uploading process and the like.
It should be noted that the upstream text uploading method in the present invention is divided into two types, one is to embed the driving module 10 into the port left after the data uploading of the service system is finished, and obtain the relevant upstream text information pushed by the service system from the port left, and the other is to plug the driving module 10 in the data generating end of the service system, and directly extract the relevant upstream text information from the data generating end.
In addition, in practical application, in the process of uploading the upstream text, abnormal conditions that data are not satisfactory, such as data missing, data error, data duplication, and the like, may occur, so the driving module 10 in the present invention is further configured to record abnormal condition parameters in the process of uploading in real time, and visually output the abnormal condition parameters after the upstream text is finished, so as to notify the user to update the text data.
It should be further noted that the driver module 10 is further configured to connect to the HDFS distributed file system through a socket interface, so as to implement a polling function for metadata information of the HDFS file
Specifically, the driver module 10 is connected to the HDFS distributed file system through a socket interface, and performs HDFS file metadata information polling on an upstream text stored in the HDFS distributed file system, where it needs to be described that data in any file system is divided into data and metadata, the data refers to actual data stored in a common file, and the metadata refers to system data used for describing features of the common file, such as distribution information of the actual data stored in the common file, the distribution information includes a location of the file on a disk and a location of the disk in a cluster, a user needs to operate a file to obtain its metadata first to locate the location of the file and obtain content or related attributes of the file, so in the present invention, the driver module 10 is connected to the HDFS distributed file system through the socket interface, performs HDFS file metadata information polling on the upstream text stored in the HDFS distributed file system according to a polling cycle or an ETL task processing process set by the user, and performs table processing on corresponding upstream text data obtained at the location according to the distribution information in the HDFS file metadata information.
The ETL task editing module 20 is configured to generate a front-end interface, so as to edit or query the table processing configuration information through the front-end interface, in other words, the configuration-based trigger task scheduling apparatus provided in the present invention can select and run a configurable table processing task based on the table processing configuration information set by a user, the user can select a configuration according to a task requirement in a configuration information editing window in the front-end interface, and when the user does not select a configuration according to the task requirement in the configuration information editing window in the front-end interface, the user runs according to the table processing configuration information set in the process of executing the ETL task last time.
Specifically, the table processing configuration information that can be supported and set based on the front-end interface in the present invention includes at least one of a scheduling service and an operating server list, a processing relationship parameter between a file and a table, and a mapping relationship parameter between a file and a table.
The scheduling service and running server list includes server information for performing processing of the table processing task and server information for supporting scheduling of the table processing task whose load exceeds the set maximum load when the load of the server performing processing of the table processing task exceeds the set maximum load.
The processing relation parameters of the file and the table refer to processing procedures required by a process of loading an upstream text into the table, for example, for a table processing procedure of upstream text data which does not need to be fused, the processing relation parameters of the file and the table comprise establishing a data warehouse ODS layer partition, deleting blanks at the head and the tail of a character string by using a Trim function, then sequentially performing eventPath event creation, checking the current ODS layer main key and performing table analysis, and finally performing floor storage to the ODS layer.
The mapping relation parameter of the file and the table refers to the number of files required for processing the table, for example, for an ETL task, 50 upstream files need to be loaded into a data table a, and then the mapping relation parameter of the file and the table of the ETL task is data table a- { upstream text 1, upstream text 2, upstream text 3 … upstream text 50}.
The task processing module 30 is configured to complete processing of at least one table processing task according to the table processing configuration information, and record table processing status information corresponding to each table processing task.
Specifically, the table processing task includes at least one of a data format conversion task, a column number verification task, a data type verification task, a primary key verification task, a data line number passing rate verification task, a table analysis task, a data increase amount conversion task, and a partition table analysis task.
In addition, in practical applications, the table processing tasks supported by the configuration-based triggered task scheduling device provided by the present invention further include all common ETL-class task processing such as updating the table analysis source database and marking the table analysis source database as creating and driving the downstream state. In the invention, the user-defined workflow of completing the data from the text to the table can be generated according to the configuration file generated by the table processing configuration information edited by the front-end interface user, and the processing flow of the data from the text to the table can be completed according to the table processing task related in the workflow.
The task processing module 30 in the present invention is composed of a plurality of task processing submodules, such as a data format conversion submodule, a column number verification submodule, a data type verification submodule, a main key verification submodule, a data line number passing rate verification submodule, a table analysis submodule, a data increment conversion submodule, a partition table analysis submodule, and the like, and the configurable selective operation submodule is performed through a configuration file of a workflow.
The flow control polling module 40 is configured to, when it is monitored that a table processing task is entered based on the text state parameter, drive the task processing module to complete processing of at least one table processing task, and monitor the state parameter of each table processing task, so as to schedule the table processing task according to the state parameter.
Specifically, the flow control polling module 40 runs a text trigger detection thread, drives the task processing module to start the processing flow of the table processing tasks after the text trigger detection thread monitors that all the upstream texts are accurately and unmistakably uploaded according to the text state parameters, and simultaneously runs the table processing trigger detection thread to monitor the state parameters of the table processing tasks, wherein the state parameters refer to the use conditions of the memory resources of the nodes or servers running the table processing tasks.
In practical application, when big data and related distributed tasks run, corresponding distributed task calling needs to start a driver-type task to monitor the running condition of the tasks submitted to a distributed cluster, and when the concurrency degree of the tasks is high, bottlenecks easily occur in memories of corresponding running nodes or servers.
In another embodiment, referring to fig. 2, fig. 2 is a schematic structural diagram of a flow control polling module provided in the present invention.
The flow control polling module 40 includes a text monitor 401, a table processing monitor 402, and a task monitor 403, where the text monitor 401, the table processing monitor 402, and the task monitor 403 form a flow processing framework. In other words, the configuration-based triggered task scheduling device provided by the invention is suitable for various different service scenes and requirements through a processing framework formed by three listeners.
Specifically, the text listener 401 is configured to monitor a text state table in the text state parameter, and drive the task processing module to complete processing of at least one table processing task and trigger the table listener and the table processing process listener to perform monitoring when a table processing triggering condition is reached based on the monitoring of the text state table.
In the present invention, the text listener 401 determines that the table processing trigger condition is reached when the text state in the text state table is consistent with the text state set by the user in the table processing configuration information after comparing the table processing configuration information with the text state table.
In addition, the text listener 401 is further configured to perform text abnormal state update processing when it is monitored that a text abnormal state occurs in the text state table.
In practical applications, the text listener 401 is further configured to visually output the abnormal text state when it is monitored that the abnormal text state occurs in the text state table, so as to prompt the user to perform verification and correction of the abnormal text state, and update the abnormal text state in the text state table after it is monitored that the user performs correction.
The table processing listener 402 is configured to listen to the table processing state table, and trigger processing of a next-level table processing task according to a listening result and a memory resource usage.
Specifically, the monitoring result indicates whether the currently executed table processing task is executed completely, and the memory resource usage indicates the memory resource usage of the currently executed table processing task. In practical application, when the currently executed table processing task is completed, the table processing task meeting the processing task requirement is triggered to be started according to the priorities of a plurality of table processing tasks to be executed at the next level and the memory resource use condition of the currently executed table processing task, and after the table processing task is started, the related task processing starter tasks are distributed to different nodes or servers according to the resource management condition of the nodes or servers on the distributed cluster, so that the reasonable resource distribution and the convenient transverse expansion mechanism of the related distributed task starter are realized.
Specifically, the task monitor 403 is configured to monitor a memory resource usage of a node corresponding to execution of each table processing task, schedule the cross-node table processing tasks when it is monitored that the memory resource usage reaches a preset maximum load limit, and perform concurrency control according to the scheduled memory resource usage.
In addition, in another embodiment, the task listener 403 is further configured to allow the submission of a new table processing task when the scheduled memory resource is smaller than the preset limited memory resource, and to deny the submission of the new table processing task when the scheduled memory resource is larger than the preset limited memory resource.
In practical application, after each task is scheduled, the use condition of the memory resource of the related node or the server is recorded, and the submission of a new task is allowed or refused according to the use condition of the memory resource. Because the distributed tasks are mainly started, the use condition of the computing memory resources is mainly provided by the distributed system, but the distributed system usually has no good management and control on the condition that the starter tasks submitted to the distributed tasks are used for monitoring the running condition of the distributed tasks, in the invention, the functions of monitoring and controlling concurrency are realized by three monitors of the flow control polling module, the specific concurrent number of the tasks or the use condition of the memory is controlled, the use condition of the resources is reasonably configured according to the actual condition, and the full utilization of the resources is completed.
The configuration-based trigger type task scheduling device provided by the invention comprises a driving module, an ETL task editing module, a task processing module and a flow control polling module; the driving module is used for recording the text state parameters in the process of uploading the upstream text to the HDFS distributed file system; the ETL task editing module is used for generating a front-end interface so as to obtain table processing configuration information through the front-end interface; the task processing module is used for finishing the processing of at least one table processing task according to the table processing configuration information and recording the table processing state information corresponding to each table processing task; the flow control polling module is used for driving the task processing module to complete the processing of at least one table processing task under the condition of monitoring the entry of the table processing task based on the text state parameter, monitoring the state parameter of each table processing task and scheduling the table processing task according to the state parameter, so that the driving module, the ETL task editing module and the flow control polling module support the data circulation among parent-child flows, the processing process, the processing method and the task parameter of the ETL task in the flow support the self-defined setting of the ETL task editing module, the circulation of the whole flow does not need to be processed according to business logic and is completely isolated from the requirement corresponding to the flow, and therefore, the flow control polling module can be adapted to various business scenes and requirements, does not need to manually program codes according to different business scenes and requirements, and achieves the effects of simplifying operation, improving efficiency and reducing development cost and time.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1. A configuration-based triggered task scheduler, comprising: the system comprises a driving module, an ETL task editing module, a task processing module and a flow control polling module;
the driving module is used for recording a text state parameter in the process of uploading an upstream text to the HDFS distributed file system;
the ETL task editing module is used for generating a front-end interface so as to edit or query the table processing configuration information through the front-end interface;
the task processing module is used for finishing the processing of at least one table processing task according to the table processing configuration information and recording the table processing state information corresponding to each table processing task;
and the flow control polling module is used for driving the task processing module to finish the processing of at least one table processing task under the condition of monitoring the entry of the table processing task based on the text state parameters, and monitoring the state parameters of each table processing task so as to schedule the table processing task according to the state parameters.
2. The configuration-based triggered task scheduling device of claim 1, wherein the process control polling module comprises a text listener, a table processing listener, and a task listener, and the text listener, the table processing listener, and the task listener form a process processing framework.
3. The configuration-based triggered task scheduling device of claim 2, wherein the text listener is configured to listen to a text status table in the text status parameters, and drive the task processing module to complete processing of at least one table processing task and trigger the table listener and the table processing process listener to listen when a table processing triggering condition is reached based on the text status table.
4. The device according to claim 2, wherein the text listener is further configured to perform a text exception status update process when it is detected that a text exception status occurs in the text status table.
5. The configuration-based triggered task scheduler of claim 2, wherein the table processing snooper is configured to snoop the table processing status table, and trigger processing of a next-level table processing task according to a snoop result and a memory resource usage condition.
6. The configuration-based triggered task scheduling device of claim 2, wherein the task monitor is configured to monitor a memory resource usage of a node that executes each of the table processing tasks, perform cross-node table processing task scheduling when it is monitored that the memory resource usage reaches a preset maximum load limit, and perform concurrency control according to the scheduled memory resource usage.
7. The configuration-based triggered task scheduler of claim 6, wherein the task listener is further configured to allow submission of a new table processing task if the scheduled memory resource is less than a preset limit memory resource.
8. The configuration-based triggered task scheduling device according to claim 1, wherein the driver module is further configured to connect to an HDFS distributed file system through a socket interface to implement an HDFS file metadata information polling function.
9. The configuration-based triggered task scheduler of claim 1, wherein the table processing configuration information comprises at least one of a scheduling service and running server list, a file-to-table processing relation parameter, and a file-to-table mapping relation parameter.
10. The configuration-based triggered task scheduler of any of claims 1 to 9, wherein the table processing task comprises at least one of a data format conversion task, a column number check task, a data type check task, a primary key check task, a data row number pass rate check task, a table analysis task, a data enrichment conversion task, and a partitioned table analysis task.
CN202210735209.7A 2022-06-27 2022-06-27 Configuration-based triggered task scheduling device Pending CN115328624A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210735209.7A CN115328624A (en) 2022-06-27 2022-06-27 Configuration-based triggered task scheduling device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210735209.7A CN115328624A (en) 2022-06-27 2022-06-27 Configuration-based triggered task scheduling device

Publications (1)

Publication Number Publication Date
CN115328624A true CN115328624A (en) 2022-11-11

Family

ID=83916423

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210735209.7A Pending CN115328624A (en) 2022-06-27 2022-06-27 Configuration-based triggered task scheduling device

Country Status (1)

Country Link
CN (1) CN115328624A (en)

Similar Documents

Publication Publication Date Title
CN108462746B (en) Container deployment method and framework based on openstack
CN108647883B (en) Business approval method, device, equipment and medium
WO2020186809A1 (en) Big data platform-based hive task scheduling method and apparatus, device, and storage medium
CN106777101B (en) Data processing engine
US20180225346A1 (en) Data processing method, device and system
CN112597014B (en) Automatic test method and device based on data driving, medium and electronic equipment
WO2018001200A1 (en) Data processing method, cluster manager, resource manager and data processing system
US20230259509A1 (en) Data association query method and apparatus, device, and storage medium
CN114741375A (en) Rapid and automatic data migration system and method for multi-source heterogeneous database
CN111339118A (en) Kubernetes-based resource change history recording method and device
CN105512188A (en) Data connection method and data connection system
CN112328708A (en) Mixed data warehouse technology for real-time aggregation of multiple data sources
CN112311902A (en) File sending method and device based on micro-service
CN115599524A (en) Data lake system based on cooperative scheduling processing of streaming data and batch data
CN110162563B (en) Data warehousing method and system, electronic equipment and storage medium
CN114066394A (en) Implementation method of multi-tenant workflow engine and multi-tenant workflow engine system
CN115328624A (en) Configuration-based triggered task scheduling device
CN116974994A (en) High-efficiency file collaboration system based on clusters
CN105468445B (en) WEB-based Spark application program scheduling method and system
CN111930862A (en) SQL interactive analysis method and system based on big data platform
KR102518772B1 (en) Data loading profiling apparatus and method for automatic processing of collected data
CN110750362A (en) Method and apparatus for analyzing biological information, and storage medium
CN100576182C (en) The real-time monitoring system of computer documents and method
WO2022116544A1 (en) Method for user to export short video draft queue online
CN112527497B (en) Serialized multithreaded data processing system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination