JP6674904B2 - ワークロード自動化およびデータ系統分析 - Google Patents

ワークロード自動化およびデータ系統分析 Download PDF

Info

Publication number
JP6674904B2
JP6674904B2 JP2016567355A JP2016567355A JP6674904B2 JP 6674904 B2 JP6674904 B2 JP 6674904B2 JP 2016567355 A JP2016567355 A JP 2016567355A JP 2016567355 A JP2016567355 A JP 2016567355A JP 6674904 B2 JP6674904 B2 JP 6674904B2
Authority
JP
Japan
Prior art keywords
information
data
job
identifying
data source
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2016567355A
Other languages
English (en)
Japanese (ja)
Other versions
JP2017522630A (ja
JP2017522630A5 (enExample
Inventor
マイケル ウォルフソン，ハリー
マイケル ウォルフソン，ハリー
ゴウルド，ジョエル
イエラカリス，アンソニー
ウェイクリング，ティム
Original Assignee
アビニシオ テクノロジー エルエルシー
アビニシオ テクノロジー エルエルシー
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by アビニシオ テクノロジー エルエルシー, アビニシオ テクノロジー エルエルシー filed Critical アビニシオ テクノロジー エルエルシー
Publication of JP2017522630A publication Critical patent/JP2017522630A/ja
Publication of JP2017522630A5 publication Critical patent/JP2017522630A5/ja
Priority to JP2020039456A priority Critical patent/JP6985441B2/ja
Application granted granted Critical
Publication of JP6674904B2 publication Critical patent/JP6674904B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5038—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Stored Programmes (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Debugging And Monitoring (AREA)
JP2016567355A 2014-05-29 2015-05-22 ワークロード自動化およびデータ系統分析 Active JP6674904B2 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2020039456A JP6985441B2 (ja) 2014-05-29 2020-03-09 ワークロード自動化およびデータ系統分析

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US201462004406P 2014-05-29 2014-05-29
US62/004,406 2014-05-29
US14/470,501 US10705877B2 (en) 2014-05-29 2014-08-27 Workload automation and data lineage analysis
US14/470,501 2014-08-27
PCT/US2015/032193 WO2015183738A1 (en) 2014-05-29 2015-05-22 Workload automation and data lineage analysis

Related Child Applications (1)

Application Number Title Priority Date Filing Date
JP2020039456A Division JP6985441B2 (ja) 2014-05-29 2020-03-09 ワークロード自動化およびデータ系統分析

Publications (3)

Publication Number Publication Date
JP2017522630A JP2017522630A (ja) 2017-08-10
JP2017522630A5 JP2017522630A5 (enExample) 2018-12-27
JP6674904B2 true JP6674904B2 (ja) 2020-04-01

Family

ID=53404861

Family Applications (2)

Application Number Title Priority Date Filing Date
JP2016567355A Active JP6674904B2 (ja) 2014-05-29 2015-05-22 ワークロード自動化およびデータ系統分析
JP2020039456A Active JP6985441B2 (ja) 2014-05-29 2020-03-09 ワークロード自動化およびデータ系統分析

Family Applications After (1)

Application Number Title Priority Date Filing Date
JP2020039456A Active JP6985441B2 (ja) 2014-05-29 2020-03-09 ワークロード自動化およびデータ系統分析

Country Status (7)

Country Link
US (2) US10705877B2 (enExample)
EP (1) EP3149581B1 (enExample)
JP (2) JP6674904B2 (enExample)
AU (3) AU2015267334B2 (enExample)
CA (1) CA2949955C (enExample)
SG (2) SG10201807358YA (enExample)
WO (1) WO2015183738A1 (enExample)

Families Citing this family (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10705877B2 (en) 2014-05-29 2020-07-07 Ab Initio Technology Llc Workload automation and data lineage analysis
US11892913B2 (en) * 2015-01-05 2024-02-06 Rubrik, Inc. Data lineage based multi-data store recovery
US10579627B2 (en) * 2016-01-08 2020-03-03 Microsoft Technology Licensing, Llc Database operation using metadata of data sources
WO2018039251A1 (en) 2016-08-22 2018-03-01 Oracle International Corporation System and method for inferencing of data transformations through pattern decomposition
CA3045810A1 (en) 2016-12-01 2018-06-07 Ab Initio Technology Llc Generating, accessing, and displaying lineage metadata
US10514993B2 (en) 2017-02-14 2019-12-24 Google Llc Analyzing large-scale data processing jobs
US10431002B2 (en) * 2017-02-23 2019-10-01 International Business Machines Corporation Displaying data lineage using three dimensional virtual reality model
US10642801B2 (en) 2017-08-29 2020-05-05 Bank Of America Corporation System for determining the impact to databases, tables and views by batch processing
US10514948B2 (en) * 2017-11-09 2019-12-24 Cloudera, Inc. Information based on run-time artifacts in a distributed computing cluster
US10635700B2 (en) 2017-11-09 2020-04-28 Cloudera, Inc. Design-time information based on run-time artifacts in transient cloud-based distributed computing clusters
US11481583B2 (en) 2017-12-28 2022-10-25 Intel Corporation Algorithm management blockchain
US10719744B2 (en) 2017-12-28 2020-07-21 Intel Corporation Automated semantic inference of visual features and scenes
US10936367B2 (en) * 2018-10-28 2021-03-02 Microsoft Technology Licensing, Llc Provenance driven job relevance assessment
US10445170B1 (en) 2018-11-21 2019-10-15 Fmr Llc Data lineage identification and change impact prediction in a distributed computing environment
US10719336B1 (en) 2019-05-14 2020-07-21 Microsoft Technology Licensing, Llc Dependency version conflict auto-resolution
US11681721B2 (en) * 2020-05-08 2023-06-20 Jpmorgan Chase Bank, N.A. Systems and methods for spark lineage data capture
US11349957B2 (en) 2020-05-14 2022-05-31 Bank Of America Corporation Automatic knowledge management for data lineage tracking
US11520801B2 (en) 2020-11-10 2022-12-06 Bank Of America Corporation System and method for automatically obtaining data lineage in real time
US11789779B2 (en) * 2021-03-01 2023-10-17 Bank Of America Corporation Electronic system for monitoring and automatically controlling batch processing
US11797574B2 (en) 2021-07-30 2023-10-24 Bank Of America Corporation Hierarchic distributed ledger for data lineage
CN118811354B (zh) * 2024-09-19 2024-11-19 联诚科技集团股份有限公司 一种实现模拟量模块烧写程序和自动校准的测试装置

Family Cites Families (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3181994B2 (ja) 1992-09-03 2001-07-03 株式会社日立製作所 ジョブフロー仕様書自動作成方法
US5966072A (en) 1996-07-02 1999-10-12 Ab Initio Software Corporation Executing computations expressed as graphs
JP2000066931A (ja) 1998-08-19 2000-03-03 Sony Corp データベースシステム、データ変更方法およびデータベースプログラムが記録されたコンピュータ読み取り可能な記録媒体
JP2001022695A (ja) 1999-07-12 2001-01-26 Nec Software Chubu Ltd システム構成要素間の対応関係管理・照会システムとその対応関係管理・照会方法および情報記録媒体
US7289964B1 (en) * 1999-08-31 2007-10-30 Accenture Llp System and method for transaction services patterns in a netcentric environment
JP4399127B2 (ja) 2001-05-14 2010-01-13 株式会社日立製作所 文書管理方法及び装置並びにその処理プログラム及びそれを格納した記憶媒体
US20060010425A1 (en) * 2001-10-29 2006-01-12 Willadsen Gloria J Methods and apparatus for automated mangement of software
US20050071842A1 (en) * 2003-08-04 2005-03-31 Totaletl, Inc. Method and system for managing data using parallel processing in a clustered network
US7366735B2 (en) 2004-04-09 2008-04-29 Oracle International Corporation Efficient extraction of XML content stored in a LOB
WO2005116830A1 (en) * 2004-05-27 2005-12-08 Koninklijke Philips Electronics N.V. Signal processing apparatus
JP4866844B2 (ja) 2004-06-16 2012-02-01 オラクル・インターナショナル・コーポレイション Ｌｏｂに格納されたｘｍｌ内容の効率的な抽出
JP2006120021A (ja) 2004-10-22 2006-05-11 Cannac:Kk 問題解決支援装置、問題解決支援方法、及び問題解決支援プログラム
JP2006268509A (ja) 2005-03-24 2006-10-05 Nomura Research Institute Ltd ジョブ設定装置およびジョブ設定方法
US7716630B2 (en) 2005-06-27 2010-05-11 Ab Initio Technology Llc Managing parameters for graph-based computations
JP2007241642A (ja) * 2006-03-08 2007-09-20 Kubota Systems Inc 解析方法、解析装置及びコンピュータプログラム
JP4476233B2 (ja) * 2006-03-24 2010-06-09 日本証券テクノロジー株式会社 バッチ系のシステムリソース管理方式
US20090024111A1 (en) * 2007-07-16 2009-01-22 German Borodulin Urethral catheter assembly for combining catheterization with injection of therapeutic liquid into the urethral channel
US8387066B1 (en) * 2007-09-28 2013-02-26 Emc Corporation Dependency-based task management using set of preconditions to generate scheduling data structure in storage area network
US20090165015A1 (en) * 2007-12-21 2009-06-25 Schlumberger Technology Corporation Managing dependencies among applications using satisfiability engine
JP2009163566A (ja) 2008-01-08 2009-07-23 Nomura Research Institute Ltd ジョブ解析支援装置
US8869165B2 (en) 2008-03-20 2014-10-21 International Business Machines Corporation Integrating flow orchestration and scheduling of jobs and data activities for a batch of workflows over multiple domains subject to constraints
US8261363B2 (en) 2008-04-29 2012-09-04 Ricoh Company, Ltd. Managing electronic data with identification data
US20110119680A1 (en) 2009-11-16 2011-05-19 Yahoo! Inc. Policy-driven schema and system for managing data system pipelines in multi-tenant model
US8510751B2 (en) * 2010-03-18 2013-08-13 International Business Machines Corporation Optimizing workflow engines
KR101809573B1 (ko) 2010-08-25 2018-01-18 아브 이니티오 테크놀로지 엘엘시 데이터플로우 그래프 특징의 평가 방법, 시스템 및 컴퓨터 판독가능 저장 매체
US8856291B2 (en) 2012-02-14 2014-10-07 Amazon Technologies, Inc. Providing configurable workflow capabilities
US10147063B2 (en) * 2012-07-16 2018-12-04 International Business Machines Corporation Transforming project management representations into business process representations
US8943505B2 (en) * 2012-08-24 2015-01-27 National Instruments Corporation Hardware assisted real-time scheduler using memory monitoring
US20140189703A1 (en) * 2012-12-28 2014-07-03 General Electric Company System and method for distributed computing using automated provisoning of heterogeneous computing resources
EP2973051A4 (en) * 2013-03-15 2016-11-16 Amazon Tech Inc SCALABLE ANALYSIS PLATFORM FOR SEMI-STRUCTURED DATA
US9477523B1 (en) * 2013-06-25 2016-10-25 Amazon Technologies, Inc. Scheduling data access jobs based on job priority and predicted execution time using historical execution data
US9304817B2 (en) * 2013-11-25 2016-04-05 Xerox Corporation Method and apparatus for a user-driven priority based job scheduling in a data processing platform
US10310903B2 (en) * 2014-01-17 2019-06-04 Red Hat, Inc. Resilient scheduling of broker jobs for asynchronous tasks in a multi-tenant platform-as-a-service (PaaS) system
US9805326B2 (en) * 2014-04-24 2017-10-31 International Business Machines Corporation Task management integrated design environment for complex data integration applications
US10705877B2 (en) 2014-05-29 2020-07-07 Ab Initio Technology Llc Workload automation and data lineage analysis

Also Published As

Publication number Publication date
US20200319932A1 (en) 2020-10-08
CA2949955C (en) 2022-12-06
AU2019283853A1 (en) 2020-01-23
SG10201807358YA (en) 2018-09-27
AU2019283853B2 (en) 2020-11-19
CA2949955A1 (en) 2015-12-03
JP2017522630A (ja) 2017-08-10
AU2021200669A1 (en) 2021-03-04
AU2015267334B2 (en) 2019-10-03
AU2015267334A1 (en) 2016-11-17
SG11201608958TA (en) 2016-11-29
US10705877B2 (en) 2020-07-07
WO2015183738A1 (en) 2015-12-03
US11748165B2 (en) 2023-09-05
JP2020126656A (ja) 2020-08-20
AU2021200669B2 (en) 2022-11-17
EP3149581B1 (en) 2025-05-14
JP6985441B2 (ja) 2021-12-22
US20150347193A1 (en) 2015-12-03
EP3149581A1 (en) 2017-04-05

Similar Documents

Publication Publication Date Title
JP6985441B2 (ja) ワークロード自動化およびデータ系統分析
EP3552092B1 (en) Differencing of executable dataflow graphs
JP5902185B2 (ja) コンピュータプログラムを表すデータフローグラフ内のデータセットオブジェクトの管理
US10275221B2 (en) Systems and methods for generating data visualization applications
JP6581108B2 (ja) 複数ソースからのデータの処理
Crawl et al. A provenance-based fault tolerance mechanism for scientific workflows
US9189529B2 (en) Queue monitoring and visualization
JP2016525734A (ja) Ｅｔｌマップ設計のためのプロジェクタおよびセレクタコンポーネントタイプの使用
CN108829505A (zh) 一种分布式调度系统及方法
Sturm et al. Distributed Multi-Perspective Declare Discovery.
US10116512B2 (en) Service discovery and/or effort estimation in networked computing environments
CN109033196A (zh) 一种分布式数据调度系统及方法
Silva et al. Adding domain data to code profiling tools to debug workflow parallel execution
HK1229921B (en) Workload automation and data lineage analysis
HK1229921A1 (en) Workload automation and data lineage analysis
Nandakumar Workflow based orchestrations for serverless workloads with ephemeral statestore
US20250181486A1 (en) Automated identification and migration of input and baseline test data
Figueira A framework for dataflow orchestration in lambda architectures
Sundaravarathan et al. MEWSE: Multi-engine workflow submission and execution on apache yarn
US20180060407A1 (en) Data-dependency-driven flow execution
Pearl Creating a SQL Health Repository
Sundaravarathan MEWSE-multi engine workflow submission and execution on Apache yarn
Vallath Real Application Testing

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20180315

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20181116

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20181228

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20190201

A601 Written request for extension of time

Free format text: JAPANESE INTERMEDIATE CODE: A601

Effective date: 20190405

A601 Written request for extension of time

Free format text: JAPANESE INTERMEDIATE CODE: A601

Effective date: 20190626

A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20190801

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20200117

A601 Written request for extension of time

Free format text: JAPANESE INTERMEDIATE CODE: A601

Effective date: 20200128

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20200309

R150 Certificate of patent or registration of utility model

Ref document number: 6674904

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250