AU2017367772B2 - Generating, accessing, and displaying lineage metadata - Google Patents

Generating, accessing, and displaying lineage metadata Download PDF

Info

Publication number
AU2017367772B2
AU2017367772B2 AU2017367772A AU2017367772A AU2017367772B2 AU 2017367772 B2 AU2017367772 B2 AU 2017367772B2 AU 2017367772 A AU2017367772 A AU 2017367772A AU 2017367772 A AU2017367772 A AU 2017367772A AU 2017367772 B2 AU2017367772 B2 AU 2017367772B2
Authority
AU
Australia
Prior art keywords
data
metadata
lineage
data structure
node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
AU2017367772A
Other languages
English (en)
Other versions
AU2017367772A1 (en
Inventor
David Clemens
Neil Galarneau
Dusan Radivojevic
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ab Initio Technology LLC
Original Assignee
Ab Initio Technology LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ab Initio Technology LLC filed Critical Ab Initio Technology LLC
Publication of AU2017367772A1 publication Critical patent/AU2017367772A1/en
Priority to AU2020203027A priority Critical patent/AU2020203027B2/en
Application granted granted Critical
Publication of AU2017367772B2 publication Critical patent/AU2017367772B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/80—Information retrieval; Database structures therefor; File system structures therefor of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML
    • G06F16/83—Querying
    • G06F16/835—Query processing
    • G06F16/8365—Query optimisation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/248—Presentation of query results
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/80—Information retrieval; Database structures therefor; File system structures therefor of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML
    • G06F16/81—Indexing, e.g. XML tags; Data structures therefor; Storage structures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/80—Information retrieval; Database structures therefor; File system structures therefor of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML
    • G06F16/83—Querying
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/80—Information retrieval; Database structures therefor; File system structures therefor of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML
    • G06F16/83—Querying
    • G06F16/832—Query formulation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/10—Text processing
    • G06F40/103—Formatting, i.e. changing of presentation of documents
    • G06F40/117—Tagging; Marking up; Designating a block; Setting of attributes

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
AU2017367772A 2016-12-01 2017-12-01 Generating, accessing, and displaying lineage metadata Active AU2017367772B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2020203027A AU2020203027B2 (en) 2016-12-01 2020-05-07 Generating, accessing, and displaying lineage metadata

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US201662428860P 2016-12-01 2016-12-01
US62/428,860 2016-12-01
PCT/US2017/064227 WO2018102691A1 (en) 2016-12-01 2017-12-01 Generating, accessing, and displaying lineage metadata

Related Child Applications (1)

Application Number Title Priority Date Filing Date
AU2020203027A Division AU2020203027B2 (en) 2016-12-01 2020-05-07 Generating, accessing, and displaying lineage metadata

Publications (2)

Publication Number Publication Date
AU2017367772A1 AU2017367772A1 (en) 2019-07-04
AU2017367772B2 true AU2017367772B2 (en) 2020-05-14

Family

ID=61656328

Family Applications (2)

Application Number Title Priority Date Filing Date
AU2017367772A Active AU2017367772B2 (en) 2016-12-01 2017-12-01 Generating, accessing, and displaying lineage metadata
AU2020203027A Active AU2020203027B2 (en) 2016-12-01 2020-05-07 Generating, accessing, and displaying lineage metadata

Family Applications After (1)

Application Number Title Priority Date Filing Date
AU2020203027A Active AU2020203027B2 (en) 2016-12-01 2020-05-07 Generating, accessing, and displaying lineage metadata

Country Status (8)

Country Link
US (2) US11741091B2 (enExample)
EP (1) EP3549036B1 (enExample)
JP (3) JP7170638B2 (enExample)
CN (2) CN110023925B (enExample)
AU (2) AU2017367772B2 (enExample)
CA (1) CA3045810A1 (enExample)
DE (1) DE112017006106T5 (enExample)
WO (1) WO2018102691A1 (enExample)

Families Citing this family (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010065623A1 (en) 2008-12-02 2010-06-10 Ab Initio Software Llc Visualizing relationships between data elements and graphical representations of data element attributes
US10452625B2 (en) * 2016-06-30 2019-10-22 Global Ids, Inc. Data lineage analysis
DE112017006106T5 (de) 2016-12-01 2019-09-19 Ab Initio Technology Llc Erzeugen von, Zugreifen auf und Anzeigen von Abstammungsmetadaten
US10331660B1 (en) * 2017-12-22 2019-06-25 Capital One Services, Llc Generating a data lineage record to facilitate source system and destination system mapping
US11315075B2 (en) * 2018-12-27 2022-04-26 Target Brands, Inc. Computer storage system
US11256701B2 (en) 2019-01-02 2022-02-22 Bank Of America Corporation Interactive lineage mapping system
US11194845B2 (en) 2019-04-19 2021-12-07 Tableau Software, LLC Interactive lineage analyzer for data assets
CN110442604B (zh) * 2019-07-11 2022-03-11 新华三大数据技术有限公司 数据流向查询方法、抽取方法、处理方法及相关装置
US11829421B2 (en) 2019-11-08 2023-11-28 Tableau Software, LLC Dynamic graph generation for interactive data analysis
US11537579B2 (en) * 2020-03-12 2022-12-27 Oracle International Corporation Fast in-memory technique to build a reverse CSR graph index in an RDBMS
US11681721B2 (en) * 2020-05-08 2023-06-20 Jpmorgan Chase Bank, N.A. Systems and methods for spark lineage data capture
WO2022160335A1 (en) * 2021-02-01 2022-08-04 Paypal, Inc. Graphical user interface to depict data lineage information in levels
JP2022140929A (ja) * 2021-03-15 2022-09-29 富士通株式会社 情報処理プログラム、情報処理方法、および情報処理装置
US12229145B2 (en) 2021-06-01 2025-02-18 Tableau Software, LLC Metadata inheritance for data assets
US12423333B2 (en) 2021-07-08 2025-09-23 Tableau Software, LLC Data processing for visualizing hierarchical data
US12105742B2 (en) 2021-08-31 2024-10-01 Tableau Software, LLC Providing data flow directions for data objects
CN113868253B (zh) * 2021-09-28 2024-04-23 中通服创立信息科技有限责任公司 一种数据关系捕获及大数据关系树构建方法
US12393903B2 (en) 2023-01-27 2025-08-19 Tableau Software, LLC Determining shortcut relationships in data models
KR20250096634A (ko) * 2023-12-20 2025-06-27 씨티뱅크, 엔.에이. 인공 지능 모델의 효율적인 배포 자동화

Family Cites Families (86)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3239170B2 (ja) 1994-07-25 2001-12-17 アタカ工業株式会社 攪拌曝気装置における軸流インペラ
US6003040A (en) 1998-01-23 1999-12-14 Mital; Vijay Apparatus and method for storing, navigating among and adding links between data items in computer databases
US7725433B1 (en) 1998-01-26 2010-05-25 International Business Machines Corporation Data navigation system and method employing data transformation lineage model
JPH11307412A (ja) 1998-04-20 1999-11-05 Matsushita Electron Corp 半導体製造データ処理方法
US6742003B2 (en) 2001-04-30 2004-05-25 Microsoft Corporation Apparatus and accompanying methods for visualizing clusters of data and hierarchical cluster classifications
US6725227B1 (en) 1998-10-02 2004-04-20 Nec Corporation Advanced web bookmark database system
US7171646B2 (en) 1999-10-05 2007-01-30 Borland Software Corporation Generating source code for object oriented elements with language neutral transient meta model and correlating display of names, symbols and code
WO2001082068A1 (en) 2000-04-21 2001-11-01 Togethersoft Corporation Methods and systems for identifying dependencies between object-oriented elements
US7117219B1 (en) 2000-05-05 2006-10-03 Group 1 Software, Inc. Method and apparatus for creating a lineage of a data field in a data flow system
US6859217B2 (en) 2000-07-19 2005-02-22 Microsoft Corporation System and method to display and manage data within hierarchies and polyarchies of information
JP2002288403A (ja) 2001-03-27 2002-10-04 Ntt Comware Corp プロジェクト管理システム、プロジェクト管理方法、及びプロジェクト管理プログラム
EP1258814A1 (en) 2001-05-17 2002-11-20 Requisite Technology Inc. Method and apparatus for analyzing the quality of the content of a database
JP3761156B2 (ja) 2001-07-27 2006-03-29 三菱電機株式会社 接続図面の編集表示装置、その動作方法およびその方法をコンピュータに実行させるプログラム
US7970240B1 (en) 2001-12-17 2011-06-28 Google Inc. Method and apparatus for archiving and visualizing digital images
US7401064B1 (en) 2002-11-07 2008-07-15 Data Advantage Group, Inc. Method and apparatus for obtaining metadata from multiple information sources within an organization in real time
US7546226B1 (en) 2003-03-12 2009-06-09 Microsoft Corporation Architecture for automating analytical view of business applications
US7120619B2 (en) 2003-04-22 2006-10-10 Microsoft Corporation Relationship view
US20040255239A1 (en) 2003-06-13 2004-12-16 Ankur Bhatt Generating electronic reports of data displayed in a computer user interface list view
EP1510937A1 (en) 2003-08-29 2005-03-02 Sap Ag A method of providing a visualisation graph on a computer and a computer for providing a visualisation graph
EP1510938B1 (en) 2003-08-29 2014-06-18 Sap Ag A method of providing a visualisation graph on a computer and a computer for providing a visualisation graph
KR100922141B1 (ko) 2003-09-15 2009-10-19 아브 이니티오 소프트웨어 엘엘시 데이터 프로파일링 방법 및 시스템
US7698348B2 (en) 2003-12-19 2010-04-13 Kinaxis Holdings Inc. Extended database engine providing versioning and embedded analytics
US7197502B2 (en) 2004-02-18 2007-03-27 Friendly Polynomials, Inc. Machine-implemented activity management system using asynchronously shared activity data objects and journal data items
US7594227B2 (en) 2004-03-08 2009-09-22 Ab Initio Technology Llc Dependency graph parameter scoping
US7496583B2 (en) 2004-04-30 2009-02-24 Microsoft Corporation Property tree for metadata navigation and assignment
EP2487601A1 (en) 2004-05-04 2012-08-15 Boston Consulting Group, Inc. Method and apparatus for selecting, analyzing and visualizing related database records as a network
US7177883B2 (en) 2004-07-15 2007-02-13 Hitachi, Ltd. Method and apparatus for hierarchical storage management based on data value and user interest
US7456840B2 (en) 2004-08-31 2008-11-25 Oracle International Corporation Displaying information using nodes in a graph
US7844582B1 (en) 2004-10-28 2010-11-30 Stored IQ System and method for involving users in object management
US7899833B2 (en) 2004-11-02 2011-03-01 Ab Initio Technology Llc Managing related data objects
US7650349B2 (en) * 2005-01-05 2010-01-19 Microsoft Corporation Prescribed navigation using topology metadata and navigation path
US7363315B2 (en) 2005-02-22 2008-04-22 Sap Ag Creating, editing, and displaying hierarchical data structures associated with data in a data source
US8176002B2 (en) 2005-03-24 2012-05-08 Microsoft Corporation Method and system for user alteration of the configuration of a data warehouse
US7734619B2 (en) 2005-05-27 2010-06-08 International Business Machines Corporation Method of presenting lineage diagrams representing query plans
US7877350B2 (en) 2005-06-27 2011-01-25 Ab Initio Technology Llc Managing metadata for graph-based computations
US20070061287A1 (en) 2005-09-09 2007-03-15 Jian Le Method, apparatus and program storage device for optimizing a data warehouse model and operation
US7493570B2 (en) 2005-09-12 2009-02-17 International Business Machines Corporation User interface options of a data lineage tool
US8577852B2 (en) 2006-03-23 2013-11-05 Infaxiom Group, Llc Automated records inventory and retention schedule generation system
WO2007127956A2 (en) 2006-04-28 2007-11-08 Business Objects, S.A. Apparatus and method for merging metadata within a repository
GB0608926D0 (en) 2006-05-05 2006-06-14 Ibm An assessment method and apparatus for matching vendor offerings to service provider requirements
US8654125B2 (en) 2006-06-22 2014-02-18 International Business Machines Corporation System and method of chart data layout
EP1883020B1 (en) 2006-07-28 2013-05-22 Dassault Systèmes Method and system for navigating in a database of a computer system
US20080040388A1 (en) * 2006-08-04 2008-02-14 Jonah Petri Methods and systems for tracking document lineage
JP2008134705A (ja) 2006-11-27 2008-06-12 Hitachi Ltd データ処理方法及びデータ分析装置
US7590672B2 (en) 2006-12-11 2009-09-15 Bycast Inc. Identification of fixed content objects in a distributed fixed content storage system
JP4398455B2 (ja) * 2006-12-22 2010-01-13 インターナショナル・ビジネス・マシーンズ・コーポレーション 経路探索方法、プログラム及びシステム
US8640086B2 (en) 2006-12-29 2014-01-28 Sap Ag Graphical user interface system and method for presenting objects
US20080172629A1 (en) 2007-01-17 2008-07-17 Microsoft Corporation Geometric Performance Metric Data Rendering
US7849050B2 (en) 2007-01-29 2010-12-07 Business Objects Data Integration, Inc. Apparatus and method for analyzing impact and lineage of multiple source data objects
US8756673B2 (en) * 2007-03-30 2014-06-17 Ricoh Company, Ltd. Techniques for sharing data
CA2593233A1 (en) 2007-07-06 2009-01-06 Cognos Incorporated System and method for federated member-based data integration and reporting
US8266122B1 (en) 2007-12-19 2012-09-11 Amazon Technologies, Inc. System and method for versioning data in a distributed data store
US8332782B1 (en) 2008-02-22 2012-12-11 Adobe Systems Incorporated Network visualization and navigation
AU2009219299B2 (en) 2008-02-26 2015-05-07 Ab Initio Technology Llc Graphic representations of data relationships
US8797178B2 (en) * 2008-03-10 2014-08-05 Microsoft Corporation Efficient stream sharing for multi-user sensor data collection
WO2010065623A1 (en) 2008-12-02 2010-06-10 Ab Initio Software Llc Visualizing relationships between data elements and graphical representations of data element attributes
US8515911B1 (en) * 2009-01-06 2013-08-20 Emc Corporation Methods and apparatus for managing multiple point in time copies in a file system
US8972899B2 (en) 2009-02-10 2015-03-03 Ayasdi, Inc. Systems and methods for visualization of data analysis
JP2010244157A (ja) 2009-04-02 2010-10-28 Toshiba Corp 機能ブロック図処理装置，機能ブロック図処理方法，およびプログラム
US8819010B2 (en) 2010-06-28 2014-08-26 International Business Machines Corporation Efficient representation of data lineage information
US10394778B2 (en) 2010-09-03 2019-08-27 Robert Lewis Jackson, JR. Minimal representation of connecting walks
US9824091B2 (en) * 2010-12-03 2017-11-21 Microsoft Technology Licensing, Llc File system backup using change journal
US9256350B2 (en) 2011-03-30 2016-02-09 Nexsan Technologies Incorporated System for displaying hierarchical information
US9342579B2 (en) 2011-05-31 2016-05-17 International Business Machines Corporation Visual analysis of multidimensional clusters
US20120310875A1 (en) 2011-06-03 2012-12-06 Prashanth Prahlad Method and system of generating a data lineage repository with lineage visibility, snapshot comparison and version control in a cloud-computing platform
JP5772458B2 (ja) 2011-09-29 2015-09-02 富士通株式会社 データ管理プログラム、ノード、および分散データベースシステム
US9659042B2 (en) 2012-06-12 2017-05-23 Accenture Global Services Limited Data lineage tracking
US10089335B2 (en) 2012-07-10 2018-10-02 Microsoft Technology Licensing, Llc Data lineage across multiple marketplaces
US20160063106A1 (en) 2012-08-08 2016-03-03 Google Inc. Related Entity Search
IN2014DN10461A (enExample) 2012-08-14 2015-08-21 Amadeus Sas
CN102890720A (zh) 2012-10-16 2013-01-23 南京通达海信息技术有限公司 数据库检查维护方法
US9075860B2 (en) 2012-10-18 2015-07-07 Oracle International Corporation Data lineage system
US9928287B2 (en) * 2013-02-24 2018-03-27 Technion Research & Development Foundation Limited Processing query to graph database
AU2014233672B2 (en) * 2013-03-15 2018-03-01 Ab Initio Technology Llc System for metadata management
US20150012315A1 (en) 2013-07-02 2015-01-08 Bank Of America Corporation Data lineage role-based security tools
US9524331B2 (en) * 2013-11-18 2016-12-20 Nuwafin Holdings Ltd Method and system for representing OLAP queries using directed acyclic graph structures in a datagrid to support real-time analytical operations
US20150271267A1 (en) * 2014-03-24 2015-09-24 Palo Alto Research Center Incorporated Content-oriented federated object store
US10705877B2 (en) 2014-05-29 2020-07-07 Ab Initio Technology Llc Workload automation and data lineage analysis
US10175974B2 (en) 2014-07-18 2019-01-08 Ab Initio Technology Llc Managing lineage information
AU2015292700B2 (en) 2014-07-24 2020-03-26 Ab Initio Technology Llc. Data lineage summarization
JP6469878B2 (ja) 2015-02-11 2019-02-13 アビニシオ テクノロジー エルエルシー データ系統図のフィルタリング
SG11201706228UA (en) 2015-02-11 2017-08-30 Ab Initio Technology Llc Filtering data lineage diagrams
US10793895B2 (en) * 2015-08-24 2020-10-06 Seven Bridges Genomics Inc. Systems and methods for epigenetic analysis
US10120923B2 (en) * 2015-11-30 2018-11-06 Bank Of America Corporation Data discovery and analysis tool
US10268753B2 (en) * 2015-12-22 2019-04-23 Opera Solutions Usa, Llc System and method for optimized query execution in computerized data modeling and analysis
DE112017006106T5 (de) 2016-12-01 2019-09-19 Ab Initio Technology Llc Erzeugen von, Zugreifen auf und Anzeigen von Abstammungsmetadaten

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LUCAS ZAMBOULIS ED - HOWARD WILLIAMS ET AL, XML Data Integration by Graph Restructuring, KEY TECHNOLOGIES FOR DATA MANAGEMENT; [LECTURE NOTES], SPRINGER-VERLAG, BERLIN/HEIDELBERG, PAGE(S) 57 - 71, (2004-06-19) *

Also Published As

Publication number Publication date
US11741091B2 (en) 2023-08-29
US12405949B2 (en) 2025-09-02
JP7410919B2 (ja) 2024-01-10
AU2020203027B2 (en) 2021-07-29
CN118916521A (zh) 2024-11-08
EP3549036A1 (en) 2019-10-09
EP3549036B1 (en) 2025-09-03
CA3045810A1 (en) 2018-06-07
JP7170638B2 (ja) 2022-11-14
WO2018102691A1 (en) 2018-06-07
US20240078229A1 (en) 2024-03-07
JP2022033825A (ja) 2022-03-02
AU2017367772A1 (en) 2019-07-04
CN110023925A (zh) 2019-07-16
AU2020203027A1 (en) 2020-05-28
US20180157702A1 (en) 2018-06-07
JP7712998B2 (ja) 2025-07-24
JP2024038033A (ja) 2024-03-19
CN110023925B (zh) 2024-07-12
JP2020501235A (ja) 2020-01-16
DE112017006106T5 (de) 2019-09-19

Similar Documents

Publication Publication Date Title
US12405949B2 (en) Generating, accessing, and displaying lineage metadata
CA2824319C (en) Column smart mechanism for column based database
US20110313969A1 (en) Updating historic data and real-time data in reports
AU2009258015B2 (en) Paging hierarchical data
US8626795B2 (en) Dynamic data association
US10810226B2 (en) Shared comments for visualized data
US20130346426A1 (en) Tracking an ancestry of metadata
US10394844B2 (en) Integrating co-deployed databases for data analytics
US10311049B2 (en) Pattern-based query result enhancement
HK40009080A (en) Generating, accessing, and displaying lineage metadata
HK40009080B (en) Generating, accessing, and displaying lineage metadata
US10606502B2 (en) Data aging infrastructure for automatically determining aging temperature
US10417185B2 (en) Gesture based semantic enrichment
US20120089593A1 (en) Query optimization based on reporting specifications
US12001710B2 (en) Dynamic update of consolidated data based on granular data values
CN103870497A (zh) 用于基于列的数据库的列智能机制

Legal Events

Date Code Title Description
FGA Letters patent sealed or granted (standard patent)