CN105359141B - 支持基于流的etl和基于实体关系的etl的组合 - Google Patents

支持基于流的etl和基于实体关系的etl的组合 Download PDF

Info

Publication number
CN105359141B
CN105359141B CN201480037071.7A CN201480037071A CN105359141B CN 105359141 B CN105359141 B CN 105359141B CN 201480037071 A CN201480037071 A CN 201480037071A CN 105359141 B CN105359141 B CN 105359141B
Authority
CN
China
Prior art keywords
data
entity
attributes
information
logic design
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201480037071.7A
Other languages
English (en)
Chinese (zh)
Other versions
CN105359141A (zh
Inventor
D·阿兰
刘国洪（托马斯）
龚宇（杰夫）
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Oracle International Corp
Original Assignee
Oracle International Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oracle International Corp filed Critical Oracle International Corp
Publication of CN105359141A publication Critical patent/CN105359141A/zh
Application granted granted Critical
Publication of CN105359141B publication Critical patent/CN105359141B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/25—Integrating or interfacing systems involving database management systems
    • G06F16/258—Data format conversion from or to a database
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/40—Transformation of program code
    • G06F8/41—Compilation
    • G06F8/43—Checking; Contextual analysis
    • G06F8/433—Dependency analysis; Data or control flow analysis

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Stored Programmes (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
CN201480037071.7A 2013-05-17 2014-03-26 支持基于流的etl和基于实体关系的etl的组合 Active CN105359141B (zh)

Applications Claiming Priority (7)

Application Number Priority Date Filing Date Title
US201361824544P 2013-05-17 2013-05-17
US61/824,544 2013-05-17
US201361843203P 2013-07-05 2013-07-05
US61/843,203 2013-07-05
US14/044,429 2013-10-02
US14/044,429 US10216814B2 (en) 2013-05-17 2013-10-02 Supporting combination of flow based ETL and entity relationship based ETL
PCT/US2014/031919 WO2014186057A1 (en) 2013-05-17 2014-03-26 Supporting combination of flow based etl and entity relationship based etl

Publications (2)

Publication Number Publication Date
CN105359141A CN105359141A (zh) 2016-02-24
CN105359141B true CN105359141B (zh) 2021-10-01

Family

ID=51896601

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201480037071.7A Active CN105359141B (zh) 2013-05-17 2014-03-26 支持基于流的etl和基于实体关系的etl的组合

Country Status (5)

Country Link
US (1) US10216814B2 (enExample)
EP (1) EP2997513A4 (enExample)
JP (1) JP6434960B2 (enExample)
CN (1) CN105359141B (enExample)
WO (1) WO2014186057A1 (enExample)

Families Citing this family (38)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10216814B2 (en) 2013-05-17 2019-02-26 Oracle International Corporation Supporting combination of flow based ETL and entity relationship based ETL
US10206770B2 (en) 2013-07-05 2019-02-19 Oracle International Corporation Load plan generation
US9460171B2 (en) * 2013-11-08 2016-10-04 International Business Machines Corporation Processing data in data migration
US9823842B2 (en) 2014-05-12 2017-11-21 The Research Foundation For The State University Of New York Gang migration of virtual machines using cluster-wide deduplication
US10055545B2 (en) * 2014-07-10 2018-08-21 Quintiles Ims Incorporated System and method for master data management
US9922104B1 (en) * 2014-08-13 2018-03-20 Numerify, Inc. Metadata driven code-generated external data feeds
US10216504B2 (en) * 2015-06-05 2019-02-26 Oracle International Corporation System and method for insulating a web user interface application from underlying technologies in an integration cloud service
US10776357B2 (en) * 2015-08-26 2020-09-15 Infosys Limited System and method of data join and metadata configuration
WO2018039266A1 (en) 2016-08-22 2018-03-01 Oracle International Corporation System and method for dynamic lineage tracking, reconstruction, and lifecycle management
US11487716B2 (en) 2016-09-17 2022-11-01 Oracle International Corporation Application materialization in hierarchical systems
US10614093B2 (en) * 2016-12-07 2020-04-07 Tata Consultancy Services Limited Method and system for creating an instance model
CN108632300B (zh) * 2017-03-15 2021-12-10 阿里巴巴集团控股有限公司 数据同步系统、方法、服务器、客户端及电子设备
US11494717B2 (en) 2017-06-08 2022-11-08 Elemica, Inc. System and method for supply chain management
US10740346B2 (en) * 2017-09-18 2020-08-11 Agile Handover And Automation Solutions, Llc System and method for automating information handover from facility project to operations/maintenance
JP7039232B2 (ja) * 2017-09-29 2022-03-22 株式会社日立製作所 技術情報共有システム及び技術情報共有方法
CN107909344B (zh) * 2017-11-21 2020-07-17 杭州电子科技大学 基于关系矩阵的工作流日志重复任务识别方法
US10783161B2 (en) * 2017-12-15 2020-09-22 International Business Machines Corporation Generating a recommended shaping function to integrate data within a data repository
CN108170832B (zh) * 2018-01-11 2021-09-14 哈尔滨工业大学 一种面向工业大数据的异构数据库的监控系统及监控方法
CN109376339B (zh) * 2018-08-02 2020-07-03 浙江大学 一种基于用户行为的文本转换候选规则信息提取方法
CN109299180B (zh) * 2018-10-31 2021-08-27 武汉光谷联众大数据技术有限责任公司 一种数据仓库etl操作系统
CN109684093B (zh) * 2018-12-24 2021-03-09 成都四方伟业软件股份有限公司 数据处理方法及系统
CN111435344B (zh) * 2019-01-15 2023-03-21 中国石油集团川庆钻探工程有限公司长庆钻井总公司 一种基于大数据的钻井提速影响因素分析模型
US11140036B2 (en) * 2019-01-16 2021-10-05 International Business Machines Corporation Identifying groups of related nodes in an integration flow
CN109918437A (zh) * 2019-03-08 2019-06-21 北京中油瑞飞信息技术有限责任公司 分布式数据处理方法、装置及数据资产管理系统
US11061856B2 (en) * 2019-07-03 2021-07-13 Bank Of America Corporation Data ingestion system
US12019601B2 (en) * 2019-12-26 2024-06-25 Yahoo Assets Llc Horizontal skimming of composite datasets
EP4238028A4 (en) 2020-11-03 2024-12-25 LiveRamp, Inc. Asserted relationships matching in an identity graph data structure
CN112163031B (zh) * 2020-11-11 2023-06-16 西安四叶草信息技术有限公司 一种基于思维导图的图数据抽取方法
US11704335B2 (en) * 2020-11-13 2023-07-18 International Business Machines Corporation Data synchronization in a data analysis system
CN114579648B (zh) * 2020-11-30 2025-05-13 中移(苏州)软件技术有限公司 数据核查方法、装置、设备及可读存储介质
US11734238B2 (en) 2021-05-07 2023-08-22 Bank Of America Corporation Correcting data errors for data processing fault recovery
US11789967B2 (en) 2021-05-07 2023-10-17 Bank Of America Corporation Recovering from data processing errors by data error detection and correction
US12189620B2 (en) * 2021-08-31 2025-01-07 International Business Machines Corporation Creating virtualized data assets using existing definitions of ETL/ELT jobs
US12197456B2 (en) * 2021-09-20 2025-01-14 Salesforce, Inc. Data virtualization adapter in an integration platform
CN113934786B (zh) * 2021-09-29 2023-09-08 浪潮卓数大数据产业发展有限公司 一种构建统一etl的实施方法
US11750488B1 (en) * 2022-04-12 2023-09-05 International Business Machines Corporation Minimizing latency of round trip network communication in a computing infrastructure
US12093249B2 (en) 2022-08-26 2024-09-17 Oracle International Corporation Dynamic inclusion of metadata configurations into a logical model
US20240176766A1 (en) * 2022-11-30 2024-05-30 Sap Se Dynamic modeling using profiles

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102349081A (zh) * 2009-02-10 2012-02-08 渣普控股有限公司 Etl构建器

Family Cites Families (42)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5193182A (en) * 1990-04-27 1993-03-09 Bachman Information Systems, Inc. Computer system for defining logical operations on design data including retrieve entity-set, send, receive, signal, when, reference to entity-set, reference to entity method, connect and disconnect
US5659723A (en) * 1991-12-20 1997-08-19 International Business Machines Corporation Entity/relationship to object oriented logical model conversion method
US5937410A (en) * 1997-10-16 1999-08-10 Johnson Controls Technology Company Method of transforming graphical object diagrams to product data manager schema
US7185016B1 (en) * 2000-09-01 2007-02-27 Cognos Incorporated Methods and transformations for transforming metadata model
US7343585B1 (en) * 2002-01-30 2008-03-11 Oracle International Corporation Operator approach for generic dataflow designs
US7350191B1 (en) 2003-04-22 2008-03-25 Noetix, Inc. Computer implemented system and method for the generation of data access applications
US20050228808A1 (en) 2003-08-27 2005-10-13 Ascential Software Corporation Real time data integration services for health care information data integration
US7739223B2 (en) 2003-08-29 2010-06-15 Microsoft Corporation Mapping architecture for arbitrary data models
US7930432B2 (en) * 2004-05-24 2011-04-19 Microsoft Corporation Systems and methods for distributing a workplan for data flow execution based on an arbitrary graph describing the desired data flow
US7849438B1 (en) 2004-05-27 2010-12-07 Sprint Communications Company L.P. Enterprise software development process for outsourced developers
US7386566B2 (en) 2004-07-15 2008-06-10 Microsoft Corporation External metadata processing
US7430498B2 (en) 2004-09-07 2008-09-30 The Boeing Company System, method and computer program product for developing a system-of-systems architecture model
JP4706001B2 (ja) * 2004-12-15 2011-06-22 信夫 濱田 設計コンピュータプログラム
TWI311705B (en) 2005-05-23 2009-07-01 Via Tech Inc Peripheral component interconnect express and changing method of link power states thereof
US7581189B2 (en) 2005-09-09 2009-08-25 Microsoft Corporation Dynamically generating a database report during a report building process
WO2007083371A1 (ja) 2006-01-18 2007-07-26 Fujitsu Limited データ統合装置、方法、プログラムを記録した記録媒体
US8307012B2 (en) * 2006-02-28 2012-11-06 Sap Ag Schema mapping and data transformation on the basis of a conceptual model
US7689582B2 (en) * 2006-03-10 2010-03-30 International Business Machines Corporation Data flow system and method for heterogeneous data integration environments
US7610299B2 (en) 2006-11-30 2009-10-27 International Business Machines Corporation Method of processing data
US7668860B2 (en) * 2007-04-02 2010-02-23 Business Objects Software Ltd. Apparatus and method for constructing and using a semantic abstraction for querying hierarchical data
US7860905B2 (en) * 2007-04-24 2010-12-28 Microsoft Corporation Systems and methods for modularizing data flows
US8245105B2 (en) 2008-07-01 2012-08-14 International Business Machines Corporation Cascade interconnect memory system with enhanced reliability
US10438166B2 (en) * 2008-12-05 2019-10-08 Stereologic Inc. Systems and methods for business process modelling
US9557889B2 (en) 2009-01-28 2017-01-31 Headwater Partners I Llc Service plan design, user interfaces, application programming interfaces, and device management
JP5375413B2 (ja) 2009-07-30 2013-12-25 富士通株式会社 データ変換装置、データ変換方法、およびデータ変換プログラム
US8719769B2 (en) * 2009-08-18 2014-05-06 Hewlett-Packard Development Company, L.P. Quality-driven ETL design optimization
US8214324B2 (en) * 2009-08-25 2012-07-03 International Business Machines Corporation Generating extract, transform, and load (ETL) jobs for loading data incrementally
US9218408B2 (en) 2010-05-27 2015-12-22 Oracle International Corporation Method for automatically creating a data mart by aggregated data extracted from a business intelligence server
JP5601066B2 (ja) 2010-07-23 2014-10-08 富士通株式会社 情報統合プログラム、装置及び方法
US20120054147A1 (en) * 2010-08-25 2012-03-01 Christophe Goetz System and method for extract, transform, and load workflow generation
WO2012050579A1 (en) * 2010-10-14 2012-04-19 Hewlett-Packard Development Company, L.P. Providing operational business intelligence
US8762934B2 (en) 2010-10-15 2014-06-24 Serghei Sarafudinov Method of extensible business object modeling and generation of system artifacts from the models
CN102081661A (zh) * 2011-01-19 2011-06-01 吉林大学 基于xml的异构关系型数据库的数据集成方法和系统
US8887076B2 (en) 2011-11-01 2014-11-11 Aver Informatics Inc. Software user interface allowing logical expression to be expressed as a flowchart
US8676772B2 (en) 2011-12-09 2014-03-18 Telduráðgevin Sp/f Systems and methods for improving database performance
US9396037B2 (en) * 2012-02-27 2016-07-19 Microsoft Technology Licensing, Llc Model-based data pipeline system optimization
CN102708161B (zh) * 2012-04-24 2014-01-15 清华大学 一种使用公共概念集的数据逻辑模型建模方法
US9256968B2 (en) * 2012-07-22 2016-02-09 International Business Machines Corporation Method for modeling using sketches
US9020945B1 (en) 2013-01-25 2015-04-28 Humana Inc. User categorization system and method
US10216814B2 (en) 2013-05-17 2019-02-26 Oracle International Corporation Supporting combination of flow based ETL and entity relationship based ETL
US10102039B2 (en) * 2013-05-17 2018-10-16 Entit Software Llc Converting a hybrid flow
US9507838B2 (en) 2013-05-17 2016-11-29 Oracle International Corporation Use of projector and selector component types for ETL map design

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102349081A (zh) * 2009-02-10 2012-02-08 渣普控股有限公司 Etl构建器

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
结构图ETL概念模型的设计方法;张忠平等;《计算机工程与应用》;20090630;第45卷(第6期);全文 *
面向商业智能的ETL模型研究综述;廖林伟等;《信息技术》;20120430(第4期);第25页第1-2段，第25-27页第1.1.1-2.1.3部分 *

Also Published As

Publication number Publication date
WO2014186057A1 (en) 2014-11-20
US20140344211A1 (en) 2014-11-20
CN105359141A (zh) 2016-02-24
EP2997513A1 (en) 2016-03-23
EP2997513A4 (en) 2017-01-25
JP6434960B2 (ja) 2018-12-05
JP2016529574A (ja) 2016-09-23
US10216814B2 (en) 2019-02-26

Similar Documents

Publication Publication Date Title
CN105359141B (zh) 支持基于流的etl和基于实体关系的etl的组合
US11847131B2 (en) Optimizing incremental loading of warehouse data
US10073867B2 (en) System and method for code generation from a directed acyclic graph using knowledge modules
US11789964B2 (en) Load plan generation
US9659012B2 (en) Debugging framework for distributed ETL process with multi-language support
US9507838B2 (en) Use of projector and selector component types for ETL map design
CN110023923B (zh) 生成数据变换工作流
Panda Mastering Microsoft Fabric: Unified Data Engineering, Governance, and Artificial Intelligence in the Cloud
US12039416B2 (en) Facilitating machine learning using remote data
US12405967B2 (en) System and method for generating a network graph from analytic artifacts in an analytics environment
CN114637767A (zh) 使用通用数据库搜索模型的系统和方法
US12169501B2 (en) Optimizing incremental loading of warehouse data
Mauri et al. Practical Azure SQL Database for Modern Developers
Schuchardt et al. Applying content management to automated provenance capture
Aytas Designing Big Data Platforms: How to Use, Deploy, and Maintain Big Data Systems
US20250278343A1 (en) System and method for providing a data analytics assistant and automatic generation of data analytics stories
Garcia Curating and Standardizing Data: The Case Study of Pharmaceutical Company Using a Common Data Layer
Virolainen Migrating Microservices to Graph Database
WO2025184392A1 (en) System and method for providing a data analytics assistant and automatic generation of data analytics stories
L’Esteve The Definitive Guide to Azure Data Engineering
Gupta et al. Databricks Data Intelligence Platform

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant