CN106815260B - 一种索引建立方法及设备 - Google Patents

一种索引建立方法及设备 Download PDF

Info

Publication number
CN106815260B
CN106815260B CN201510868254.XA CN201510868254A CN106815260B CN 106815260 B CN106815260 B CN 106815260B CN 201510868254 A CN201510868254 A CN 201510868254A CN 106815260 B CN106815260 B CN 106815260B
Authority
CN
China
Prior art keywords
index
column
determining
type
time threshold
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201510868254.XA
Other languages
English (en)
Chinese (zh)
Other versions
CN106815260A (zh
Inventor
郑博文
潘岳
魏闯先
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN201510868254.XA priority Critical patent/CN106815260B/zh
Priority to JP2018524442A priority patent/JP6898320B2/ja
Priority to PCT/CN2016/106581 priority patent/WO2017092583A1/zh
Priority to EP16869893.4A priority patent/EP3385864B1/en
Publication of CN106815260A publication Critical patent/CN106815260A/zh
Priority to US15/996,237 priority patent/US11003649B2/en
Application granted granted Critical
Publication of CN106815260B publication Critical patent/CN106815260B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G06F16/2272—Management thereof
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/221—Column-oriented storage; Management thereof
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G06F16/2237—Vectors, bitmaps or matrices
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G06F16/2246—Trees, e.g. B+trees
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G06F16/2255—Hash tables
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/31—Indexing; Data structures therefor; Storage structures
    • G06F16/316—Indexing structures
    • G06F16/319—Inverted lists

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
CN201510868254.XA 2015-12-01 2015-12-01 一种索引建立方法及设备 Active CN106815260B (zh)

Priority Applications (5)

Application Number Priority Date Filing Date Title
CN201510868254.XA CN106815260B (zh) 2015-12-01 2015-12-01 一种索引建立方法及设备
JP2018524442A JP6898320B2 (ja) 2015-12-01 2016-11-21 インデックス確立の方法およびデバイス
PCT/CN2016/106581 WO2017092583A1 (zh) 2015-12-01 2016-11-21 一种索引建立方法及设备
EP16869893.4A EP3385864B1 (en) 2015-12-01 2016-11-21 Method and device for establishing index
US15/996,237 US11003649B2 (en) 2015-12-01 2018-06-01 Index establishment method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510868254.XA CN106815260B (zh) 2015-12-01 2015-12-01 一种索引建立方法及设备

Publications (2)

Publication Number Publication Date
CN106815260A CN106815260A (zh) 2017-06-09
CN106815260B true CN106815260B (zh) 2021-05-04

Family

ID=58796259

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510868254.XA Active CN106815260B (zh) 2015-12-01 2015-12-01 一种索引建立方法及设备

Country Status (5)

Country Link
US (1) US11003649B2 (enExample)
EP (1) EP3385864B1 (enExample)
JP (1) JP6898320B2 (enExample)
CN (1) CN106815260B (enExample)
WO (1) WO2017092583A1 (enExample)

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106815260B (zh) 2015-12-01 2021-05-04 阿里巴巴集团控股有限公司 一种索引建立方法及设备
US11023439B2 (en) * 2016-09-01 2021-06-01 Morphick, Inc. Variable cardinality index and data retrieval
CN110851438B (zh) * 2018-08-20 2025-03-18 北京京东尚科信息技术有限公司 一种数据库索引优化建议与验证的方法和装置
CN110874358B (zh) * 2018-08-30 2023-05-05 阿里巴巴集团控股有限公司 多属性列的存储、检索方法和装置以及电子设备
US10545960B1 (en) * 2019-03-12 2020-01-28 The Governing Council Of The University Of Toronto System and method for set overlap searching of data lakes
CN111046130B (zh) * 2019-11-08 2023-05-23 杭州安恒信息技术股份有限公司 结合ElasticSearch和FSM的关联检索方法
CN113297454B (zh) * 2020-04-14 2025-01-03 阿里巴巴集团控股有限公司 检索方法、查询方法、装置、系统、电子设备和计算机存储介质
CN113535733B (zh) * 2021-07-26 2024-08-06 北京锐安科技有限公司 数据存储、查询方法、装置、计算机设备及存储介质
CN114168800B (zh) * 2021-11-26 2024-09-13 哈尔滨工程大学 一种基于b+树和位图索引融合树的冲突检测方法
US12182093B2 (en) * 2022-09-27 2024-12-31 Ocient Holdings LLC Applying range-based filtering during query execution based on utilizing an inverted index structure
US12321387B2 (en) * 2023-03-10 2025-06-03 Equifax Inc. Automatically generating search indexes for expediting searching of a computerized database
CN116383144A (zh) * 2023-03-23 2023-07-04 中科星图股份有限公司 一种多源异构遥感数据存储方法和装置
CN116719843B (zh) * 2023-05-31 2025-11-07 中电科金仓(北京)科技股份有限公司 数据库系统的查询方法、存储介质及设备
CN117573680B (zh) * 2024-01-17 2024-04-12 深圳市进择科技有限公司 一种基于大数据的定位数据传输管理系统及方法
CN118467669B (zh) * 2024-05-09 2025-02-25 深圳计算科学研究院 索引构建方法、字段搜索方法、装置、设备及介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102467572A (zh) * 2010-11-17 2012-05-23 英业达股份有限公司 支持重复数据删除程序的数据区块查询方法
CN102779180A (zh) * 2012-06-29 2012-11-14 华为技术有限公司 数据存储系统的操作处理方法，数据存储系统
CN104112011A (zh) * 2014-07-16 2014-10-22 深圳市国泰安信息技术有限公司 一种海量数据提取的方法及装置

Family Cites Families (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS63201716A (ja) * 1987-02-17 1988-08-19 Nec Corp インデツクス保守方式
US5404510A (en) 1992-05-21 1995-04-04 Oracle Corporation Database index design based upon request importance and the reuse and modification of similar existing indexes
JPH0785093A (ja) * 1993-09-16 1995-03-31 Nissan Motor Co Ltd インデックス自動設定方法
US5907837A (en) * 1995-07-17 1999-05-25 Microsoft Corporation Information retrieval system in an on-line network including separate content and layout of published titles
US7640244B1 (en) * 2004-06-07 2009-12-29 Teredata Us, Inc. Dynamic partition enhanced joining using a value-count index
US7392266B2 (en) * 2005-03-17 2008-06-24 International Business Machines Corporation Apparatus and method for monitoring usage of components in a database index
JP2007122405A (ja) * 2005-10-28 2007-05-17 Hitachi Ltd データベース管理システムの性能チューニングシステム
JP5162215B2 (ja) * 2007-11-22 2013-03-13 株式会社エヌ・ティ・ティ・データ データ処理装置、データ処理方法、および、プログラム
JP4237813B2 (ja) * 2008-05-26 2009-03-11 株式会社東芝 構造化文書管理システム
US8489565B2 (en) * 2009-03-24 2013-07-16 Microsoft Corporation Dynamic integrated database index management
CN101609460B (zh) * 2009-07-22 2011-12-14 中国科学院地理科学与资源研究所 一种支持异构地学数据资源的检索方法及检索系统
US8655867B2 (en) * 2010-05-13 2014-02-18 Salesforce.Com, Inc. Method and system for optimizing queries in a multi-tenant database environment
US8412701B2 (en) * 2010-09-27 2013-04-02 Computer Associates Think, Inc. Multi-dataset global index
US8396858B2 (en) * 2011-08-11 2013-03-12 International Business Machines Corporation Adding entries to an index based on use of the index
US8825664B2 (en) * 2012-08-17 2014-09-02 Splunk Inc. Indexing preview
CN103810212B (zh) * 2012-11-14 2017-05-24 阿里巴巴集团控股有限公司 一种数据库索引的自动创建方法及系统
US20140317093A1 (en) * 2013-04-22 2014-10-23 Salesforce.Com, Inc. Facilitating dynamic creation of multi-column index tables and management of customer queries in an on-demand services environment
US20150032720A1 (en) * 2013-07-23 2015-01-29 Yahoo! Inc. Optimizing database queries
CN103390066B (zh) * 2013-08-08 2016-02-17 上海新炬网络信息技术有限公司 一种数据库全局性自动化优化预警装置及其处理方法
CN104714984A (zh) 2013-12-17 2015-06-17 中国移动通信集团湖南有限公司 一种数据库优化的方法和装置
CN104182460B (zh) * 2014-07-18 2017-06-13 浙江大学 基于倒排索引的时间序列相似性查询方法
US9846746B2 (en) * 2014-11-20 2017-12-19 Facebook, Inc. Querying groups of users based on user attributes for social analytics
CN104834736A (zh) * 2015-05-19 2015-08-12 深圳证券信息有限公司 构建索引库的方法、装置及检索的方法、装置和系统
CN105045851A (zh) * 2015-07-07 2015-11-11 福建天晴数码有限公司 根据日志分析自动创建数据库索引的方法及系统
CN106815260B (zh) 2015-12-01 2021-05-04 阿里巴巴集团控股有限公司 一种索引建立方法及设备
US10601593B2 (en) * 2016-09-23 2020-03-24 Microsoft Technology Licensing, Llc Type-based database confidentiality using trusted computing

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102467572A (zh) * 2010-11-17 2012-05-23 英业达股份有限公司 支持重复数据删除程序的数据区块查询方法
CN102779180A (zh) * 2012-06-29 2012-11-14 华为技术有限公司 数据存储系统的操作处理方法，数据存储系统
CN104112011A (zh) * 2014-07-16 2014-10-22 深圳市国泰安信息技术有限公司 一种海量数据提取的方法及装置

Also Published As

Publication number Publication date
JP2019502980A (ja) 2019-01-31
US20180276264A1 (en) 2018-09-27
US11003649B2 (en) 2021-05-11
EP3385864B1 (en) 2024-01-03
WO2017092583A1 (zh) 2017-06-08
EP3385864A1 (en) 2018-10-10
CN106815260A (zh) 2017-06-09
EP3385864A4 (en) 2018-10-10
JP6898320B2 (ja) 2021-07-07

Similar Documents

Publication Publication Date Title
CN106815260B (zh) 一种索引建立方法及设备
US7174345B2 (en) Methods and systems for auto-partitioning of schema objects
US11157473B2 (en) Multisource semantic partitioning
US10769126B1 (en) Data entropy reduction across stream shard
US8949222B2 (en) Changing the compression level of query plans
US7895171B2 (en) Compressibility estimation of non-unique indexes in a database management system
US9141666B2 (en) Incremental maintenance of range-partitioned statistics for query optimization
EP2924594A1 (en) Data encoding and corresponding data structure in a column-store database
CN113868230B (zh) 一种基于Spark计算框架的大表连接优化方法
CN100428226C (zh) 实现类内存数据库存取和检索的方法
CN109299101B (zh) 数据检索方法、装置、服务器和存储介质
US12026162B2 (en) Data query method and apparatus, computing device, and storage medium
CN111723089A (zh) 一种基于列式存储格式处理数据的方法和装置
CN111026709A (zh) 基于集群访问的数据处理方法及装置
KR101955376B1 (ko) 비공유 아키텍처 기반의 분산 스트림 처리 엔진에서 관계형 질의를 처리하는 방법, 이를 수행하기 위한 기록 매체 및 장치
CN119537383B (zh) 基于冷热数据分离和多模数据库引擎的存储方法及装置
CN117648391B (zh) 一种gnss轨迹数据存储、查询方法及数据库系统
Suganya et al. Efficient fragmentation and allocation in distributed databases
CN118708608A (zh) 处理引擎的选择方法、装置、计算机设备、存储介质
Jia et al. Research on real time data warehouse architecture
CN115221157A (zh) 数据处理方法及装置、计算机可读存储介质和电子设备
CN113032400B (zh) 海量数据的高性能TopN查询方法、系统及介质
CN109766254B (zh) It系统运维监控数据辅助预处理方法和系统
CN120416113A (zh) 一种数据规模的处理方法和系统
HK40074965A (en) Data processing method and apparatus, computer readable storage medium, and electronic device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant