CN104685497B - 通过过滤方法操作的聚合/分组的硬件实现 - Google Patents

通过过滤方法操作的聚合/分组的硬件实现 Download PDF

Info

Publication number
CN104685497B
CN104685497B CN201380050945.8A CN201380050945A CN104685497B CN 104685497 B CN104685497 B CN 104685497B CN 201380050945 A CN201380050945 A CN 201380050945A CN 104685497 B CN104685497 B CN 104685497B
Authority
CN
China
Prior art keywords
row
value
bit vector
predicate
filter element
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201380050945.8A
Other languages
English (en)
Chinese (zh)
Other versions
CN104685497A (zh
Inventor
J·施奥尔
P·安博格
R·D·霍博金斯二世
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Oracle International Corp
Original Assignee
Oracle International Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oracle International Corp filed Critical Oracle International Corp
Publication of CN104685497A publication Critical patent/CN104685497A/zh
Application granted granted Critical
Publication of CN104685497B publication Critical patent/CN104685497B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/242—Query formulation
    • G06F16/2433—Query languages
    • G06F16/244—Grouping and aggregation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G06F16/24553—Query execution of query operations
    • G06F16/24561—Intermediate data storage techniques for performance improvement
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G06F16/2237—Vectors, bitmaps or matrices

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
CN201380050945.8A 2012-08-20 2013-08-16 通过过滤方法操作的聚合/分组的硬件实现 Active CN104685497B (zh)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US13/590,110 2012-08-20
US13/590,110 US9600522B2 (en) 2012-08-20 2012-08-20 Hardware implementation of the aggregation/group by operation: filter method
PCT/US2013/055316 WO2014031481A1 (en) 2012-08-20 2013-08-16 Hardware implementation of the aggregation/group by operation: filter method

Publications (2)

Publication Number Publication Date
CN104685497A CN104685497A (zh) 2015-06-03
CN104685497B true CN104685497B (zh) 2018-05-08

Family

ID=49083778

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201380050945.8A Active CN104685497B (zh) 2012-08-20 2013-08-16 通过过滤方法操作的聚合/分组的硬件实现

Country Status (5)

Country Link
US (1) US9600522B2 (enExample)
EP (1) EP2885730B1 (enExample)
JP (1) JP6400010B2 (enExample)
CN (1) CN104685497B (enExample)
WO (1) WO2014031481A1 (enExample)

Families Citing this family (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9697174B2 (en) 2011-12-08 2017-07-04 Oracle International Corporation Efficient hardware instructions for processing bit vectors for single instruction multiple data processors
US9792117B2 (en) 2011-12-08 2017-10-17 Oracle International Corporation Loading values from a value vector into subregisters of a single instruction multiple data register
US10534606B2 (en) 2011-12-08 2020-01-14 Oracle International Corporation Run-length encoding decompression
US9563658B2 (en) 2012-08-20 2017-02-07 Oracle International Corporation Hardware implementation of the aggregation/group by operation: hash-table method
US9727606B2 (en) 2012-08-20 2017-08-08 Oracle International Corporation Hardware implementation of the filter/project operations
US9235564B2 (en) * 2013-07-19 2016-01-12 International Business Machines Corporation Offloading projection of fixed and variable length database columns
US9275168B2 (en) * 2013-07-19 2016-03-01 International Business Machines Corporation Hardware projection of fixed and variable length columns of database tables
US11113054B2 (en) 2013-09-10 2021-09-07 Oracle International Corporation Efficient hardware instructions for single instruction multiple data processors: fast fixed-length value compression
US9606921B2 (en) 2013-09-21 2017-03-28 Oracle International Corporation Granular creation and refresh of columnar data
US20150261862A1 (en) * 2014-03-12 2015-09-17 Kaushal MITTAL Search Space Reduction Using Approximate Results
US20160103914A1 (en) * 2014-10-10 2016-04-14 Salesforce.Com, Inc. Offloading search processing against analytic data stores
US10101889B2 (en) 2014-10-10 2018-10-16 Salesforce.Com, Inc. Dashboard builder with live data updating without exiting an edit mode
US9449188B2 (en) 2014-10-10 2016-09-20 Salesforce.Com, Inc. Integration user for analytical access to read only data stores generated from transactional systems
US10049141B2 (en) 2014-10-10 2018-08-14 salesforce.com,inc. Declarative specification of visualization queries, display formats and bindings
US9600548B2 (en) 2014-10-10 2017-03-21 Salesforce.Com Row level security integration of analytical data store with cloud architecture
US10073885B2 (en) 2015-05-29 2018-09-11 Oracle International Corporation Optimizer statistics and cost model for in-memory tables
US10067954B2 (en) 2015-07-22 2018-09-04 Oracle International Corporation Use of dynamic dictionary encoding with an associated hash table to support many-to-many joins and aggregations
US10115213B2 (en) 2015-09-15 2018-10-30 Salesforce, Inc. Recursive cell-based hierarchy for data visualizations
US10089368B2 (en) 2015-09-18 2018-10-02 Salesforce, Inc. Systems and methods for making visual data representations actionable
US10152421B2 (en) * 2015-11-23 2018-12-11 Intel Corporation Instruction and logic for cache control operations
US10235468B2 (en) * 2015-12-30 2019-03-19 Business Objects Software Limited Indirect filtering in blended data operations
US10061832B2 (en) 2016-11-28 2018-08-28 Oracle International Corporation Database tuple-encoding-aware data partitioning in a direct memory access engine
US10055358B2 (en) 2016-03-18 2018-08-21 Oracle International Corporation Run length encoding aware direct memory access filtering engine for scratchpad enabled multicore processors
US10061714B2 (en) 2016-03-18 2018-08-28 Oracle International Corporation Tuple encoding aware direct memory access engine for scratchpad enabled multicore processors
US10402425B2 (en) 2016-03-18 2019-09-03 Oracle International Corporation Tuple encoding aware direct memory access engine for scratchpad enabled multi-core processors
US10489401B2 (en) 2016-05-31 2019-11-26 International Business Machines Corporation Efficient aggregation in a parallel system
US10599488B2 (en) 2016-06-29 2020-03-24 Oracle International Corporation Multi-purpose events for notification and sequence control in multi-core processor systems
US10380058B2 (en) 2016-09-06 2019-08-13 Oracle International Corporation Processor core to coprocessor interface with FIFO semantics
US10783102B2 (en) 2016-10-11 2020-09-22 Oracle International Corporation Dynamically configurable high performance database-aware hash engine
US10311047B2 (en) 2016-10-19 2019-06-04 Salesforce.Com, Inc. Streamlined creation and updating of OLAP analytic databases
US10459859B2 (en) 2016-11-28 2019-10-29 Oracle International Corporation Multicast copy ring for database direct memory access filtering engine
US10176114B2 (en) 2016-11-28 2019-01-08 Oracle International Corporation Row identification number generation in database direct memory access engine
US10725947B2 (en) 2016-11-29 2020-07-28 Oracle International Corporation Bit vector gather row count calculation and handling in direct memory access engine
US10346145B2 (en) * 2017-06-23 2019-07-09 Intel Corporation Loop execution with predicate computing for dataflow machines
US11275731B2 (en) * 2017-12-01 2022-03-15 Singlestore, Inc. Accelerated filtering, grouping and aggregation in a database system
US11226955B2 (en) * 2018-06-28 2022-01-18 Oracle International Corporation Techniques for enabling and integrating in-memory semi-structured data and text document searches with in-memory columnar query processing
US10963460B2 (en) * 2018-12-06 2021-03-30 Xilinx, Inc. Integrated circuits and methods to accelerate data queries
US11514055B2 (en) 2019-09-13 2022-11-29 Oracle International Corporation Querying on hybrid formats and storages
US11507590B2 (en) 2019-09-13 2022-11-22 Oracle International Corporation Techniques for in-memory spatial object filtering
US12080433B2 (en) * 2021-02-10 2024-09-03 International Business Machines Corporation Healthcare application insight compilation sensitivity
US11580111B2 (en) * 2021-04-06 2023-02-14 Thoughtspot, Inc. Distributed pseudo-random subset generation
US12072885B2 (en) * 2021-05-27 2024-08-27 Sap Se Query processing for disk based hybrid transactional analytical processing system
US12423306B1 (en) * 2021-12-10 2025-09-23 Amazon Technologies, Inc Dynamically partition data
CN115563116A (zh) * 2022-10-11 2023-01-03 北京奥星贝斯科技有限公司 一种数据库表扫描方法、装置以及设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5706495A (en) * 1996-05-07 1998-01-06 International Business Machines Corporation Encoded-vector indices for decision support and warehousing
CN1666196A (zh) * 2002-05-10 2005-09-07 甲骨文国际公司 存储和访问数据,以及提高数据库查询语言语句性能的方法和机制
CN1928866A (zh) * 2005-09-09 2007-03-14 国际商业机器公司 用于处理包括半连接的查询的方法和系统
CN102171680A (zh) * 2008-10-05 2011-08-31 微软公司 用于基于列的数据编码结构的查询的高效大规模过滤和/或排序

Family Cites Families (37)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5511190A (en) 1995-01-20 1996-04-23 Tandem Computers, Inc. Hash-based database grouping system and method
US5655080A (en) 1995-08-14 1997-08-05 International Business Machines Corporation Distributed hash group-by cooperative processing
US5884299A (en) 1997-02-06 1999-03-16 Ncr Corporation Optimization of SQL queries involving aggregate expressions using a plurality of local and global aggregation operations
JP2000187668A (ja) 1998-12-22 2000-07-04 Hitachi Ltd グループ化方法と重複排除方法
US6934712B2 (en) * 2000-03-21 2005-08-23 International Business Machines Corporation Tagging XML query results over relational DBMSs
US6598041B1 (en) 2000-09-07 2003-07-22 International Business Machines Corporation Method, system, and program for processing modifications to data in tables in a database system
US6581055B1 (en) 2000-09-11 2003-06-17 Oracle International Corporation Query optimization with switch predicates
CA2327167C (en) 2000-11-30 2007-10-16 Ibm Canada Limited-Ibm Canada Limitee Method and system for composing a query for a database and traversing the database
US6571233B2 (en) 2000-12-06 2003-05-27 International Business Machines Corporation Optimization of SQL queries using filtering predicates
US6996569B1 (en) * 2001-06-20 2006-02-07 Microstrategy Incorporated Systems and methods for custom grouping of data
US6823329B2 (en) * 2002-04-02 2004-11-23 Sybase, Inc. Database system providing methodology for acceleration of queries involving functional expressions against columns having enumerated storage
US6957225B1 (en) 2002-05-07 2005-10-18 Oracle International Corporation Automatic discovery and use of column correlations in tables
US7111025B2 (en) 2003-04-30 2006-09-19 International Business Machines Corporation Information retrieval system and method using index ANDing for improving performance
US7433886B2 (en) * 2003-10-24 2008-10-07 Microsoft Corporation SQL language extensions for modifying collection-valued and scalar valued columns in a single statement
US7469241B2 (en) 2004-11-30 2008-12-23 Oracle International Corporation Efficient data aggregation operations using hash tables
US20080189251A1 (en) 2006-08-25 2008-08-07 Jeremy Branscome Processing elements of a hardware accelerated reconfigurable processor for accelerating database operations and queries
US7974976B2 (en) 2006-11-09 2011-07-05 Yahoo! Inc. Deriving user intent from a user query
US7870398B2 (en) 2007-01-25 2011-01-11 International Business Machines Corporation Integrity assurance of query result from database service provider
US9524317B2 (en) 2007-08-24 2016-12-20 International Business Machines Corporation Optimization of aggregate queries in database management systems using an early out join when processing min and max functions
US20100005077A1 (en) * 2008-07-07 2010-01-07 Kickfire, Inc. Methods and systems for generating query plans that are compatible for execution in hardware
US8078595B2 (en) 2007-10-09 2011-12-13 Oracle International Corporation Secure normal forms
US8150850B2 (en) 2008-01-07 2012-04-03 Akiban Technologies, Inc. Multiple dimensioned database architecture
US20090216757A1 (en) 2008-02-27 2009-08-27 Robi Sen System and Method for Performing Frictionless Collaboration for Criteria Search
US7974967B2 (en) * 2008-04-15 2011-07-05 Sap Ag Hybrid database system using runtime reconfigurable hardware
US8374986B2 (en) * 2008-05-15 2013-02-12 Exegy Incorporated Method and system for accelerated stream processing
US8275761B2 (en) * 2008-05-15 2012-09-25 International Business Machines Corporation Determining a density of a key value referenced in a database query over a range of rows
US8832112B2 (en) 2008-06-17 2014-09-09 International Business Machines Corporation Encoded matrix index
US20100030728A1 (en) 2008-07-29 2010-02-04 Oracle International Corporation Computing selectivities for group of columns and expressions
US9430526B2 (en) 2008-09-29 2016-08-30 Teradata Us, Inc. Method and system for temporal aggregation
US8312038B2 (en) * 2008-12-18 2012-11-13 Oracle International Corporation Criteria builder for query builder
US8745031B2 (en) 2009-05-28 2014-06-03 Oracle International Corporation Cache-based predicate handling for queries on XML data using uncorrelated path-based row sources
EP2290562A1 (en) 2009-08-24 2011-03-02 Amadeus S.A.S. Segmented main-memory stored relational database table system with improved collaborative scan algorithm
US8832142B2 (en) * 2010-08-30 2014-09-09 Oracle International Corporation Query and exadata support for hybrid columnar compressed data
US10311105B2 (en) * 2010-12-28 2019-06-04 Microsoft Technology Licensing, Llc Filtering queried data on data stores
US8676834B2 (en) 2011-02-16 2014-03-18 International Business Machines Corporation Set-level comparisons in dynamically formed groups
US9563658B2 (en) 2012-08-20 2017-02-07 Oracle International Corporation Hardware implementation of the aggregation/group by operation: hash-table method
US9727606B2 (en) 2012-08-20 2017-08-08 Oracle International Corporation Hardware implementation of the filter/project operations

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5706495A (en) * 1996-05-07 1998-01-06 International Business Machines Corporation Encoded-vector indices for decision support and warehousing
CN1666196A (zh) * 2002-05-10 2005-09-07 甲骨文国际公司 存储和访问数据,以及提高数据库查询语言语句性能的方法和机制
CN1928866A (zh) * 2005-09-09 2007-03-14 国际商业机器公司 用于处理包括半连接的查询的方法和系统
CN102171680A (zh) * 2008-10-05 2011-08-31 微软公司 用于基于列的数据编码结构的查询的高效大规模过滤和/或排序

Also Published As

Publication number Publication date
CN104685497A (zh) 2015-06-03
EP2885730A1 (en) 2015-06-24
US9600522B2 (en) 2017-03-21
WO2014031481A1 (en) 2014-02-27
JP2015532749A (ja) 2015-11-12
US20140052713A1 (en) 2014-02-20
JP6400010B2 (ja) 2018-10-03
EP2885730B1 (en) 2021-05-26

Similar Documents

Publication Publication Date Title
CN104685497B (zh) 通过过滤方法操作的聚合/分组的硬件实现
JP6305406B2 (ja) フィルタリング／射影動作のハードウェア実装
CN102822819B (zh) 管理数据查询
US9576028B2 (en) Managing data queries
CN103366015B (zh) 一种基于Hadoop的OLAP数据存储与查询方法
CN109299133A (zh) 数据查询方法、计算机系统及非暂时性计算机可读介质
CN103309958A (zh) Gpu和cpu混合架构下的olap星型连接查询优化方法
CN107918621A (zh) 日志数据处理方法、装置及业务系统
US12169493B2 (en) Workload-aware column inprints
JP2018519578A (ja) ピボットテーブルに組み入れるための限定されたデータセットを提供するためにデータを選択的に取得するための方法およびシステム
CN105740264A (zh) 一种分布式xml数据库的排序方法及装置
CN104537012B (zh) 数据处理方法和装置
KR101955376B1 (ko) 비공유 아키텍처 기반의 분산 스트림 처리 엔진에서 관계형 질의를 처리하는 방법, 이를 수행하기 위한 기록 매체 및 장치
CN113988282A (zh) 用于图神经网络和图应用的可编程访问引擎架构
WO2018119478A1 (en) Meta-join and meta-group-by indexes for big data
CN111177156B (zh) 一种大数据存储方法及系统
US10802796B1 (en) Distributed sorted set
US11822582B2 (en) Metadata clustering
Wang et al. A novel compression algorithm for hardware-oriented gradient boosting decision tree classification model
HK40088869A (en) Managing data queries
CN116541482A (zh) 文本对象索引方法、对象存储系统及相关设备
CN116227585A (zh) 集群任务的并行执行方法、装置、计算机设备和存储介质
CN116610703A (zh) 数据查询方法、装置、计算机设备和存储介质
AU2017202899A1 (en) Managing data queries
CN104408543A (zh) 业务实体的属性状态的处理方法及处理装置

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
EXSB Decision made by sipo to initiate substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant