CN102142039B - 用于进行数据仓储的系统和方法 - Google Patents

用于进行数据仓储的系统和方法 Download PDF

Info

Publication number
CN102142039B
CN102142039B CN2011101223215A CN201110122321A CN102142039B CN 102142039 B CN102142039 B CN 102142039B CN 2011101223215 A CN2011101223215 A CN 2011101223215A CN 201110122321 A CN201110122321 A CN 201110122321A CN 102142039 B CN102142039 B CN 102142039B
Authority
CN
China
Prior art keywords
data
warehouse
warehouses
data warehouse
given
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
CN2011101223215A
Other languages
English (en)
Chinese (zh)
Other versions
CN102142039A (zh
Inventor
克里斯托弗·R·贝尔
保罗·J·博得
马克·E·邓拉普
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Amazon Technologies Inc
Original Assignee
Amazon Technologies Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Amazon Technologies Inc filed Critical Amazon Technologies Inc
Publication of CN102142039A publication Critical patent/CN102142039A/zh
Application granted granted Critical
Publication of CN102142039B publication Critical patent/CN102142039B/zh
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
    • G06F16/2471—Distributed queries
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00—Data processing: database and file management or data structures
    • Y10S707/99931—Database or file accessing
    • Y10S707/99933—Query processing, i.e. searching
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
    • Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y10S707/00—Data processing: database and file management or data structures
    • Y10S707/99951—File or database maintenance
    • Y10S707/99952—Coherency, e.g. same view to multiple users

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Fuzzy Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
CN2011101223215A 2004-12-17 2005-12-14 用于进行数据仓储的系统和方法 Expired - Lifetime CN102142039B (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US11/016,563 2004-12-17
US11/016,563 US7415487B2 (en) 2004-12-17 2004-12-17 Apparatus and method for data warehousing

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN2005800475052A Division CN101305365B (zh) 2004-12-17 2005-12-14 用于进行数据仓储的设备和方法

Publications (2)

Publication Number Publication Date
CN102142039A CN102142039A (zh) 2011-08-03
CN102142039B true CN102142039B (zh) 2012-12-26

Family

ID=36046631

Family Applications (2)

Application Number Title Priority Date Filing Date
CN2011101223215A Expired - Lifetime CN102142039B (zh) 2004-12-17 2005-12-14 用于进行数据仓储的系统和方法
CN2005800475052A Expired - Lifetime CN101305365B (zh) 2004-12-17 2005-12-14 用于进行数据仓储的设备和方法

Family Applications After (1)

Application Number Title Priority Date Filing Date
CN2005800475052A Expired - Lifetime CN101305365B (zh) 2004-12-17 2005-12-14 用于进行数据仓储的设备和方法

Country Status (7)

Country Link
US (1) US7415487B2 (https=)
EP (2) EP3352103A1 (https=)
JP (1) JP5047806B2 (https=)
KR (2) KR101323500B1 (https=)
CN (2) CN102142039B (https=)
CA (2) CA2879549C (https=)
WO (1) WO2006065953A2 (https=)

Families Citing this family (63)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060167811A1 (en) * 2005-01-24 2006-07-27 Microsoft Corporation Product locker for multi-merchant purchasing environment for downloadable products
US20090171847A2 (en) * 2005-01-24 2009-07-02 Microsoft Corporation Multi-merchant purchasing environment for downloadable products
EP2033122B1 (en) * 2006-06-26 2011-02-23 International Business Machines Corporation Method and system for ensuring consistency over time of data gathered by distinct software applications
US8392358B2 (en) * 2006-06-29 2013-03-05 Nice Systems Technologies Inc. Temporal extent considerations in reporting on facts organized as a dimensionally-modeled fact collection
US20080028028A1 (en) * 2006-07-27 2008-01-31 Gr8 Practice Llc E-mail archive system, method and medium
US8417731B2 (en) 2006-12-28 2013-04-09 Sap Ag Article utilizing a generic update module with recursive calls identify, reformat the update parameters into the identified database table structure
US8606799B2 (en) 2006-12-28 2013-12-10 Sap Ag Software and method for utilizing a generic database query
US7730056B2 (en) * 2006-12-28 2010-06-01 Sap Ag Software and method for utilizing a common database layout
US20080177892A1 (en) * 2007-01-19 2008-07-24 International Business Machines Corporation Method for service oriented data extraction transformation and load
CA2576703C (en) * 2007-02-02 2014-08-12 Cognos Incorporated System and method for optimizing business intelligence data queries within a client-server architecture
ITRM20070161A1 (it) * 2007-03-27 2008-09-28 Uni Del Salento Metodo e formalismo per inviare istruzioni a database distribuiti realizzato mediante programma per computer
US7739547B2 (en) * 2007-06-07 2010-06-15 International Business Machines Corporation Failure recovery and error correction techniques for data loading in information warehouses
US20110004622A1 (en) * 2007-10-17 2011-01-06 Blazent, Inc. Method and apparatus for gathering and organizing information pertaining to an entity
CN101471956B (zh) * 2007-12-28 2011-08-31 英业达股份有限公司 目标端的储存设备状态的识别及动态更新方法
US7933873B2 (en) * 2008-01-17 2011-04-26 International Business Machines Corporation Handling transfer of bad data to database partitions in restartable environments
US8521682B2 (en) * 2008-01-17 2013-08-27 International Business Machines Corporation Transfer of data from transactional data sources to partitioned databases in restartable environments
US8156084B2 (en) * 2008-01-17 2012-04-10 International Business Machines Corporation Transfer of data from positional data sources to partitioned databases in restartable environments
US8688622B2 (en) * 2008-06-02 2014-04-01 The Boeing Company Methods and systems for loading data into a temporal data warehouse
US7917547B2 (en) * 2008-06-10 2011-03-29 Microsoft Corporation Virtualizing objects within queries
US8010648B2 (en) * 2008-10-24 2011-08-30 Microsoft Corporation Replica placement in a distributed storage system
US8380663B2 (en) * 2008-12-17 2013-02-19 Sybase, Inc. Data integrity in a database environment through background synchronization
AU2013202073B2 (en) * 2009-02-12 2014-04-17 Accenture Global Services Limited A data system architecture to analyze distributed data sets
US20100205153A1 (en) * 2009-02-12 2010-08-12 Accenture Global Services Gmbh Data System Architecture to Analyze Distributed Data Sets
US8135666B2 (en) * 2010-03-11 2012-03-13 International Business Machines Corporation Systems and methods for policy based execution of time critical data warehouse triggers
US10162851B2 (en) * 2010-04-19 2018-12-25 Salesforce.Com, Inc. Methods and systems for performing cross store joins in a multi-tenant store
CN102541959B (zh) * 2010-12-31 2014-03-12 中国移动通信集团安徽有限公司 Etl调度方法、装置及系统
EP2490135A1 (en) 2011-02-21 2012-08-22 Amadeus S.A.S. Method and system for providing statistical data from a data warehouse
AU2012203333A1 (en) 2011-06-15 2013-01-10 Agile Software Pty Limited Method and apparatus for testing data warehouses
CN102279886B (zh) * 2011-08-16 2012-10-17 中国民生银行股份有限公司 元数据处理方法及设备
CN102799651B (zh) * 2012-06-28 2015-01-21 用友软件股份有限公司 查询处理装置和查询处理方法
JP6511394B2 (ja) * 2012-09-27 2019-05-15 アマデウス エス．アー．エス．Ａｍａｄｅｕｓ Ｓ．Ａ．Ｓ． データの保存および取得の方法およびシステム
CN103902583B (zh) * 2012-12-27 2019-03-12 方正国际软件(北京)有限公司 一种etl流程执行系统
CN103345468B (zh) * 2013-05-13 2017-03-29 中国科学技术大学 一种基于太阳能建筑的建筑材料热物性数据库系统
US10158579B2 (en) * 2013-06-21 2018-12-18 Amazon Technologies, Inc. Resource silos at network-accessible services
US10198292B2 (en) * 2013-11-27 2019-02-05 Actian Sub Iii, Inc. Scheduling database queries based on elapsed time of queries
US20150169609A1 (en) * 2013-12-06 2015-06-18 Zaius, Inc. System and method for load balancing in a data storage system
US10545917B2 (en) 2014-02-19 2020-01-28 Snowflake Inc. Multi-range and runtime pruning
US11809451B2 (en) 2014-02-19 2023-11-07 Snowflake Inc. Caching systems and methods
US9553762B1 (en) * 2014-06-26 2017-01-24 Altera Corporation Network-on-chip with fixed and configurable functions
US10304025B2 (en) 2015-05-26 2019-05-28 Locanis Ag Controlling industrial trucks in a warehouse
CN106921614B (zh) * 2015-12-24 2020-05-22 北京国双科技有限公司 业务数据处理方法和装置
CN106933913B (zh) * 2015-12-31 2020-05-08 北京国双科技有限公司 数据处理方法和装置
EP3449363B1 (en) * 2016-04-28 2025-02-12 Snowflake Inc. Multi-cluster warehouse
US10785295B2 (en) * 2016-06-30 2020-09-22 Intel Corporation Fabric encapsulated resilient storage
CN107563925A (zh) * 2017-08-31 2018-01-09 上海德衡数据科技有限公司 一种智能化区域急救医疗集成数据中心系统架构
CN107992526B (zh) * 2017-11-10 2021-03-23 广州虎牙信息科技有限公司 主播推荐方法、存储设备及计算机设备
EP3514742A1 (en) * 2018-01-19 2019-07-24 Siemens Aktiengesellschaft Collecting data from a data-source into a mom data warehouse
CN109299180B (zh) * 2018-10-31 2021-08-27 武汉光谷联众大数据技术有限责任公司 一种数据仓库etl操作系统
US11966870B2 (en) 2019-04-18 2024-04-23 Oracle International Corporation System and method for determination of recommendations and alerts in an analytics environment
US12248490B2 (en) 2019-04-18 2025-03-11 Oracle International Corporation System and method for ranking of database tables for use with extract, transform, load processes
US11614976B2 (en) 2019-04-18 2023-03-28 Oracle International Corporation System and method for determining an amount of virtual machines for use with extract, transform, load (ETL) processes
JP7611843B2 (ja) 2019-04-30 2025-01-10 オラクル・インターナショナル・コーポレイション 分析アプリケーション環境を用いたデータアナリティクスのためのシステムおよび方法
US12153595B2 (en) 2019-07-04 2024-11-26 Oracle International Corporation System and method for data pipeline optimization in an analytic applications environment
US11169728B2 (en) 2019-09-10 2021-11-09 Western Digital Technologies, Inc. Replication configuration for multiple heterogeneous data stores
US11321285B2 (en) 2020-10-01 2022-05-03 Bank Of America Corporation Automatic database script generation for copying data between relational databases
US12045246B2 (en) * 2020-11-20 2024-07-23 AtScale, Inc. Distributed queries through dynamic views
US11934670B2 (en) 2021-03-31 2024-03-19 Netapp, Inc. Performing various operations at the granularity of a consistency group within a cross-site storage solution
US11928352B2 (en) 2021-05-05 2024-03-12 Netapp, Inc. Maintaining the benefit of parallel splitting of ops between primary and secondary storage clusters in synchronous replication while adding support for op logging and early engagement of op logging
US11416815B1 (en) * 2021-06-11 2022-08-16 Coupang Corp. Systems and computerized methods for balancing inventory
US11537314B1 (en) 2021-10-07 2022-12-27 Netapp, Inc. Resynchronization of individual volumes of a consistency group (CG) within a cross-site storage solution while maintaining synchronization of other volumes of the CG
US11892982B2 (en) 2021-10-20 2024-02-06 Netapp, Inc. Facilitating immediate performance of volume resynchronization with the use of passive cache entries
CN114595294B (zh) * 2022-03-11 2022-09-20 北京梦诚科技有限公司 一种数据仓库建模和抽取方法及系统
US11907562B2 (en) 2022-07-11 2024-02-20 Netapp, Inc. Methods and storage nodes to decrease delay in resuming input output (I/O) operations after a non-disruptive event for a storage object of a distributed storage system by utilizing asynchronous inflight replay of the I/O operations

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1310824A (zh) * 1999-05-24 2001-08-29 白金技术Ip公司 用于数据仓库的选择聚集层和交叉产品层的方法和装置

Family Cites Families (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4769772A (en) * 1985-02-28 1988-09-06 Honeywell Bull, Inc. Automated query optimization method using both global and parallel local optimizations for materialization access planning for distributed databases
US6263337B1 (en) 1998-03-17 2001-07-17 Microsoft Corporation Scalable system for expectation maximization clustering of large databases
US6374251B1 (en) 1998-03-17 2002-04-16 Microsoft Corporation Scalable system for clustering of large databases
US6167405A (en) * 1998-04-27 2000-12-26 Bull Hn Information Systems Inc. Method and apparatus for automatically populating a data warehouse system
US6178418B1 (en) 1998-07-28 2001-01-23 Noetix Corporation Distributed data warehouse query and resource management system
US6243715B1 (en) * 1998-11-09 2001-06-05 Lucent Technologies Inc. Replicated database synchronization method whereby primary database is selected queries to secondary databases are referred to primary database, primary database is updated, then secondary databases are updated
US6550057B1 (en) 1999-08-31 2003-04-15 Accenture Llp Piecemeal retrieval in an information services patterns environment
US6502095B2 (en) 1999-09-09 2002-12-31 Lucent Technologies Inc. Timestamp-based system and method for serializing lazy updates in a distributed database
US6438538B1 (en) 1999-10-07 2002-08-20 International Business Machines Corporation Data replication in data warehousing scenarios
JP2001297026A (ja) * 2000-04-11 2001-10-26 Hitachi Ltd 複数のデータベースマネージメントシステムを有する計算機システム
US6922685B2 (en) 2000-05-22 2005-07-26 Mci, Inc. Method and system for managing partitioned data resources
US7010553B2 (en) 2002-03-19 2006-03-07 Network Appliance, Inc. System and method for redirecting access to a remote mirrored snapshot
US7149759B2 (en) 2002-03-25 2006-12-12 International Business Machines Corporation Method and system for detecting conflicts in replicated data in a database network
US7627597B2 (en) * 2003-03-13 2009-12-01 International Business Machines Corporation Usage-based optimization of network traffic and data warehouse size
US6973654B1 (en) 2003-05-27 2005-12-06 Microsoft Corporation Systems and methods for the repartitioning of data
EP1501021A1 (en) 2003-07-22 2005-01-26 Sap Ag A system and method for extracting data sets from an online relational database into a data warehouse
US20050240354A1 (en) 2003-08-27 2005-10-27 Ascential Software Corporation Service oriented architecture for an extract function in a data integration platform

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1310824A (zh) * 1999-05-24 2001-08-29 白金技术Ip公司 用于数据仓库的选择聚集层和交叉产品层的方法和装置

Also Published As

Publication number Publication date
CN101305365B (zh) 2011-07-06
KR20120120444A (ko) 2012-11-01
CN101305365A (zh) 2008-11-12
WO2006065953A2 (en) 2006-06-22
US7415487B2 (en) 2008-08-19
US20060136354A1 (en) 2006-06-22
CA2879549A1 (en) 2006-06-22
EP3352103A1 (en) 2018-07-25
EP1828935A2 (en) 2007-09-05
CA2594568A1 (en) 2006-06-22
JP5047806B2 (ja) 2012-10-10
CN102142039A (zh) 2011-08-03
KR101323500B1 (ko) 2013-10-31
KR101266683B1 (ko) 2013-06-26
JP2008524715A (ja) 2008-07-10
KR20080002743A (ko) 2008-01-04
CA2879549C (en) 2016-07-26
CA2594568C (en) 2015-04-14
WO2006065953A3 (en) 2006-09-14
EP1828935B1 (en) 2018-05-09

Similar Documents

Publication Publication Date Title
CN102142039B (zh) 用于进行数据仓储的系统和方法
US10997148B2 (en) Processing transactions on journaled tables
US20200167370A1 (en) Maintaining a relationship between two different items of data
CN107844388B (zh) 从备份系统流式恢复数据库
US7497370B2 (en) Supply chain visibility solution architecture
US8121978B2 (en) Database system providing improved methods for data replication
US20100017184A1 (en) Systems and methods for performing virtual storage operations
CA2419883A1 (en) Discriminatory replay of log files during table space recovery in a database management system
Hobbs et al. Oracle 10g data warehousing
Vadlamani Introduction to PostgreSQL database management
Whitesell et al. Decentralizing Data
Gallucci Adaptation and Implementation of an automated Corporate ETL Framework on Microsoft Fabric: A Technical Approach to Integrating Data Workflows into a Modern Cloud Platform
Frank et al. Architecture for mobile control functions in supplier deliveries for distributed integrated ERP modules
UGLI DEVELOPMENT OF A METHOD FOR DATA MIGRATION FROM XML REPOSITORIES INTO RELATIONAL DATABASE
Ballard et al. Data Warehousing with the Informix Dynamic Server
Grech Designing and Implementing a Distributed Database for a Small Multi-Outlet Business
Business ePublications at Regis Universit y
Babb et al. Oracle Database High Availability Overview, 11g Release 1 (11.1) B28281-01
Bhat et al. Oracle Database High Availability Architecture and Best Practices 10g Release 1 (10.1) Part No. B10726-02 Copyright© 2003, 2004, Oracle. All rights reserved. Primary Author: Cathy Baird Contributing Author: David Austin, Andrew Babb, Mark Bauer, Ruth Baylis, Tammy Bednar, Pradeep
Curtis GoldenGate
Diamond iCDA: Continuous Double Auction on the Web
Babb et al. Oracle Database High Availability Overview 11g Release 1 (11.1) B28281-04
Kasper New method for data replication in distributed heterogeneous database systems

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
CX01 Expiry of patent term
CX01 Expiry of patent term

Granted publication date: 20121226