CN101567013B - Method and apparatus for implementing ETL scheduling - Google Patents
Method and apparatus for implementing ETL scheduling Download PDFInfo
- Publication number
- CN101567013B CN101567013B CN2009102032769A CN200910203276A CN101567013B CN 101567013 B CN101567013 B CN 101567013B CN 2009102032769 A CN2009102032769 A CN 2009102032769A CN 200910203276 A CN200910203276 A CN 200910203276A CN 101567013 B CN101567013 B CN 101567013B
- Authority
- CN
- China
- Prior art keywords
- program
- flow
- subtask
- flows
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 46
- 230000001960 triggered effect Effects 0.000 claims abstract description 74
- 230000008569 process Effects 0.000 claims description 27
- 238000012545 processing Methods 0.000 claims description 25
- 238000007726 management method Methods 0.000 claims description 12
- 238000012544 monitoring process Methods 0.000 claims description 7
- 238000013075 data extraction Methods 0.000 claims description 4
- 238000011068 loading method Methods 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 238000013501 data transformation Methods 0.000 claims description 2
- 230000009466 transformation Effects 0.000 claims description 2
- 238000012423 maintenance Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000002159 abnormal effect Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 238000007418 data mining Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000004140 cleaning Methods 0.000 description 1
- 125000004122 cyclic group Chemical group 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
Images
Landscapes
- Stored Programmes (AREA)
Abstract
Description
Claims (12)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2009102032769A CN101567013B (en) | 2009-06-02 | 2009-06-02 | Method and apparatus for implementing ETL scheduling |
HK10104106.1A HK1137244A1 (en) | 2009-06-02 | 2010-04-27 | Method for implementing etl scheduling and apparatus thereof |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2009102032769A CN101567013B (en) | 2009-06-02 | 2009-06-02 | Method and apparatus for implementing ETL scheduling |
Publications (2)
Publication Number | Publication Date |
---|---|
CN101567013A CN101567013A (en) | 2009-10-28 |
CN101567013B true CN101567013B (en) | 2011-09-28 |
Family
ID=41283162
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN2009102032769A Active CN101567013B (en) | 2009-06-02 | 2009-06-02 | Method and apparatus for implementing ETL scheduling |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN101567013B (en) |
HK (1) | HK1137244A1 (en) |
Families Citing this family (50)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN102236580B (en) * | 2010-04-26 | 2013-03-20 | 阿里巴巴集团控股有限公司 | Method for distributing node to ETL (Extraction-Transformation-Loading) task and dispatching system |
CN102004666A (en) * | 2010-11-25 | 2011-04-06 | 中国工商银行股份有限公司 | Large-scale computer operation scheduling equipment and system |
WO2012075622A1 (en) * | 2010-12-07 | 2012-06-14 | Sap Ag | Implemantion of a process based on a user-defined sub-task sequence |
CN102073540B (en) * | 2010-12-15 | 2013-05-08 | 北京新媒传信科技有限公司 | Distributed affair submitting method and device thereof |
CN102541959B (en) * | 2010-12-31 | 2014-03-12 | 中国移动通信集团安徽有限公司 | Method, device and system for scheduling electron transport layer (ETL) |
CN102750179B (en) * | 2011-04-22 | 2014-10-01 | 中国移动通信集团河北有限公司 | Method and device for scheduling tasks between cloud computing platform and data warehouse |
CN102279888B (en) * | 2011-08-24 | 2014-04-30 | 北京新媒传信科技有限公司 | Method and system for scheduling tasks |
CN102375891A (en) * | 2011-11-15 | 2012-03-14 | 山东浪潮金融信息系统有限公司 | Implementation tool for unloading and loading incremental data |
CN103514028B (en) * | 2012-06-14 | 2016-12-21 | 北京新媒传信科技有限公司 | A kind of method and apparatus processing distributed transaction |
CN102999816B (en) * | 2012-12-05 | 2016-02-24 | 中邮科通信技术股份有限公司 | The workflow engine of personalized operation flow |
CN103902574A (en) * | 2012-12-27 | 2014-07-02 | 中国移动通信集团内蒙古有限公司 | Real-time data loading method and device based on data flow technology |
CN103034554B (en) * | 2012-12-30 | 2015-11-18 | 焦点科技股份有限公司 | The ETL dispatching system that a kind of error correction is restarted and automatic decision starts and method |
CN103164337B (en) * | 2013-02-28 | 2015-12-09 | 汉柏科技有限公司 | Based on the cloud computing method for testing software of finite state machine |
CN104679482A (en) * | 2013-11-27 | 2015-06-03 | 北京拓尔思信息技术股份有限公司 | OSGI (Open Service Gateway Initiative)-based ETL (Extraction-Transformation-Loading) processing device and method |
CN104778074B (en) * | 2014-01-14 | 2019-02-26 | 腾讯科技(深圳)有限公司 | A kind of calculating task processing method and processing device |
CN105095327A (en) * | 2014-05-23 | 2015-11-25 | 深圳市珍爱网信息技术有限公司 | Distributed ELT system and scheduling method |
CN104572257A (en) * | 2014-07-30 | 2015-04-29 | 南京坦道信息科技有限公司 | United coordination dispatching algorithm based on finite state automata for various high-concurrency jobs |
CN104462243B (en) * | 2014-11-19 | 2018-09-07 | 上海烟草集团有限责任公司 | A kind of ETL scheduling system and methods of combination data check |
CN104484167B (en) * | 2014-12-05 | 2018-03-09 | 广州华多网络科技有限公司 | Task processing method and device |
CN105808619B (en) * | 2014-12-31 | 2019-08-06 | 华为技术有限公司 | Method, impact analysis computing device and the key reset apparatus that task based on impact analysis is reformed |
CN104750522B (en) * | 2015-03-12 | 2018-01-05 | 用友网络科技股份有限公司 | The Dynamic Execution method and system of task or flow |
CN106528275A (en) * | 2015-09-10 | 2017-03-22 | 网易(杭州)网络有限公司 | Processing method of data tasks and task scheduler |
CN105321045A (en) * | 2015-11-04 | 2016-02-10 | 北京知聚科技有限公司 | Service process formal model construction method and system |
CN105446808B (en) * | 2015-11-12 | 2019-05-21 | 国云科技股份有限公司 | A kind of method that combined task completes complex task |
CN106708854B (en) * | 2015-11-13 | 2020-05-22 | 博雅网络游戏开发(深圳)有限公司 | Data export method and device |
CN106712924B (en) * | 2015-11-16 | 2021-03-19 | 方正国际软件(北京)有限公司 | Method and device for realizing universal time sequence communication |
CN105677462A (en) * | 2015-12-30 | 2016-06-15 | 生迪光电科技股份有限公司 | Distributed task system based on internet of things and business processing method |
CN107025224B (en) * | 2016-01-29 | 2020-10-16 | 阿里巴巴集团控股有限公司 | Method and equipment for monitoring task operation |
CN105976158A (en) * | 2016-04-26 | 2016-09-28 | 中国电子科技网络信息安全有限公司 | Visual ETL flow management and scheduling monitoring method |
CN107479962B (en) * | 2016-06-08 | 2021-05-07 | 阿里巴巴集团控股有限公司 | Method and equipment for issuing task |
CN106293920A (en) * | 2016-08-15 | 2017-01-04 | 北京票之家科技有限公司 | Method for scheduling task and device |
CN107145576B (en) * | 2017-05-08 | 2020-06-23 | 科技谷(厦门)信息技术有限公司 | Big data ETL scheduling system supporting visualization and process |
CN109408204A (en) * | 2017-08-15 | 2019-03-01 | 阿里巴巴集团控股有限公司 | A kind of method for scheduling task and device of distributed task scheduling system |
CN108564281B (en) * | 2018-04-13 | 2022-04-05 | 浙江传媒学院 | Method for realizing outsourcing work task scheduling system based on structuralization |
CN109240810B (en) * | 2018-08-03 | 2021-02-23 | 腾讯科技(深圳)有限公司 | Task processing method and device and storage medium |
CN109359949B (en) * | 2018-10-30 | 2022-05-27 | 中国建设银行股份有限公司 | Flow display method and device |
CN109445929A (en) * | 2018-11-16 | 2019-03-08 | 杭州数澜科技有限公司 | A kind of method and system of scheduler task |
CN109857794A (en) * | 2018-12-29 | 2019-06-07 | 南瑞集团有限公司 | A kind of implementation method and its system of the high concurrent lightweight data integration framework based on response |
CN111176802B (en) * | 2019-07-26 | 2023-03-14 | 腾讯科技(深圳)有限公司 | Task processing method and device, electronic equipment and storage medium |
CN111082976B (en) * | 2019-12-02 | 2022-07-29 | 东莞数汇大数据有限公司 | Method for supporting ETL task scheduling visualization |
CN113127522B (en) * | 2019-12-31 | 2024-05-10 | 阿里巴巴集团控股有限公司 | Data processing method, device, system and storage medium |
CN111427943A (en) * | 2020-03-27 | 2020-07-17 | 北京明略软件系统有限公司 | Task management method and device in ET L system |
CN111930814B (en) * | 2020-05-29 | 2024-02-27 | 武汉达梦数据库股份有限公司 | File event scheduling method based on ETL system and ETL system |
CN111857984A (en) * | 2020-06-01 | 2020-10-30 | 北京文思海辉金信软件有限公司 | Job calling processing method and device in bank system and computer equipment |
CN111914010B (en) * | 2020-08-04 | 2024-02-20 | 北京百度网讯科技有限公司 | Method, device, equipment and storage medium for processing business |
CN112486502A (en) * | 2020-11-30 | 2021-03-12 | 京东方科技集团股份有限公司 | Distributed task deployment method and device, computer equipment and storage medium |
CN112667383B (en) * | 2020-12-31 | 2024-02-09 | 北京高途云集教育科技有限公司 | Task execution and scheduling method, system, device, computing equipment and medium |
CN113111106A (en) * | 2021-04-06 | 2021-07-13 | 创意信息技术股份有限公司 | ETL design data access method and data access module based on Web |
CN113138807B (en) * | 2021-04-25 | 2022-09-09 | 上海淇玥信息技术有限公司 | Method and device for executing multi-node service task and electronic equipment |
CN117112668B (en) * | 2023-08-23 | 2024-02-20 | 广州嘉磊元新信息科技有限公司 | ETL-based RPA flow management method and system |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1897025A (en) * | 2006-04-27 | 2007-01-17 | 南京联创科技股份有限公司 | Parallel ETL technology of multi-thread working pack in mass data process |
CN1953490A (en) * | 2006-09-06 | 2007-04-25 | 南京中兴软创科技有限责任公司 | A method to extract and provide the charging data with the technology of ETL |
CN101216782A (en) * | 2007-12-29 | 2008-07-09 | 中国建设银行股份有限公司 | Method and system for financial data accomplishing ETL processing |
-
2009
- 2009-06-02 CN CN2009102032769A patent/CN101567013B/en active Active
-
2010
- 2010-04-27 HK HK10104106.1A patent/HK1137244A1/en unknown
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1897025A (en) * | 2006-04-27 | 2007-01-17 | 南京联创科技股份有限公司 | Parallel ETL technology of multi-thread working pack in mass data process |
CN1953490A (en) * | 2006-09-06 | 2007-04-25 | 南京中兴软创科技有限责任公司 | A method to extract and provide the charging data with the technology of ETL |
CN101216782A (en) * | 2007-12-29 | 2008-07-09 | 中国建设银行股份有限公司 | Method and system for financial data accomplishing ETL processing |
Also Published As
Publication number | Publication date |
---|---|
CN101567013A (en) | 2009-10-28 |
HK1137244A1 (en) | 2010-07-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN101567013B (en) | Method and apparatus for implementing ETL scheduling | |
US9588822B1 (en) | Scheduler for data pipeline | |
US10101991B2 (en) | Managing a software-patch submission queue | |
CN100487700C (en) | Data processing method and system of data library | |
CN111125444A (en) | Big data task scheduling management method, device, equipment and storage medium | |
US8538793B2 (en) | System and method for managing real-time batch workflows | |
US20100223446A1 (en) | Contextual tracing | |
CN110287052A (en) | A kind of root of abnormal task determines method and device because of task | |
CN107577586B (en) | Method and equipment for determining service execution link in distributed system | |
CN101477524A (en) | System performance optimization method and system based on materialized view | |
CN105630588A (en) | Distributed job scheduling method and system | |
CN110569090A (en) | data processing method and device, electronic equipment and storage medium | |
CN101751288A (en) | Method, device and system applying process scheduler | |
CN105719126A (en) | System and method for internet big data task scheduling based on life cycle model | |
CN104536819A (en) | Task scheduling method based on WEB service | |
CN101639803A (en) | Exception handling method and exception handling device for multithread application system | |
CN111930354B (en) | Framework component system for software development and construction method thereof | |
CN111984447B (en) | Registration compensation system and method in overtime or abnormal situation of bank transaction | |
CN103744730A (en) | Task scheduling method and device | |
US20130239123A1 (en) | Milestone manager | |
CN105446812A (en) | Multitask scheduling configuration method | |
CN112948096A (en) | Batch scheduling method, device and equipment | |
CN105450737B (en) | A kind of data processing method, device and system | |
US20070074225A1 (en) | Apparatus, method and computer program product providing integration environment having an integration control functionality coupled to an integration broker | |
CN105354083A (en) | Method and apparatus for checking precondition of scheduling task |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1137244 Country of ref document: HK |
|
C14 | Grant of patent or utility model | ||
GR01 | Patent grant | ||
REG | Reference to a national code |
Ref country code: HK Ref legal event code: GR Ref document number: 1137244 Country of ref document: HK |
|
TR01 | Transfer of patent right |
Effective date of registration: 20191209 Address after: P.O. Box 31119, grand exhibition hall, hibiscus street, 802 West Bay Road, Grand Cayman, Cayman Islands Patentee after: Innovative advanced technology Co., Ltd Address before: Greater Cayman Islands, British Cayman Islands Patentee before: Alibaba Group Holding Co., Ltd. |
|
TR01 | Transfer of patent right |