CN109271408A - 一种分布式数据连接处理方法、装置、设备及存储介质 - Google Patents
一种分布式数据连接处理方法、装置、设备及存储介质 Download PDFInfo
- Publication number
- CN109271408A CN109271408A CN201811015813.2A CN201811015813A CN109271408A CN 109271408 A CN109271408 A CN 109271408A CN 201811015813 A CN201811015813 A CN 201811015813A CN 109271408 A CN109271408 A CN 109271408A
- Authority
- CN
- China
- Prior art keywords
- bloom filter
- data
- acquisition system
- opposite end
- data acquisition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
- G06F16/24534—Query rewriting; Transformation
- G06F16/24537—Query rewriting; Transformation of operators
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2453—Query optimisation
- G06F16/24534—Query rewriting; Transformation
- G06F16/24542—Plan optimisation
- G06F16/24544—Join order optimisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24553—Query execution of query operations
- G06F16/24558—Binary matching operations
- G06F16/2456—Join operations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/56—Provisioning of proxy services
- H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Operations Research (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Transfer Between Computers (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Image Processing (AREA)
Abstract
Description
Claims (13)
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811015813.2A CN109271408B (zh) | 2018-08-31 | 2018-08-31 | 一种分布式数据连接处理方法、装置、设备及存储介质 |
TW108122542A TWI716016B (zh) | 2018-08-31 | 2019-06-27 | 分散式資料連接處理方法、裝置、設備及儲存媒體 |
PCT/CN2019/096478 WO2020042813A1 (zh) | 2018-08-31 | 2019-07-18 | 一种分布式数据连接处理方法、装置、设备及存储介质 |
US17/085,521 US11188535B2 (en) | 2018-08-31 | 2020-10-30 | Distributed join operation processing method, apparatus, device, and storage medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811015813.2A CN109271408B (zh) | 2018-08-31 | 2018-08-31 | 一种分布式数据连接处理方法、装置、设备及存储介质 |
Publications (2)
Publication Number | Publication Date |
---|---|
CN109271408A true CN109271408A (zh) | 2019-01-25 |
CN109271408B CN109271408B (zh) | 2020-07-28 |
Family
ID=65187034
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201811015813.2A Active CN109271408B (zh) | 2018-08-31 | 2018-08-31 | 一种分布式数据连接处理方法、装置、设备及存储介质 |
Country Status (4)
Country | Link |
---|---|
US (1) | US11188535B2 (zh) |
CN (1) | CN109271408B (zh) |
TW (1) | TWI716016B (zh) |
WO (1) | WO2020042813A1 (zh) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110209874A (zh) * | 2019-04-24 | 2019-09-06 | 北京奇艺世纪科技有限公司 | 信息处理方法、装置、电子设备及存储介质 |
WO2020042813A1 (zh) * | 2018-08-31 | 2020-03-05 | 阿里巴巴集团控股有限公司 | 一种分布式数据连接处理方法、装置、设备及存储介质 |
CN113360507A (zh) * | 2021-08-10 | 2021-09-07 | 蚂蚁金服(杭州)网络技术有限公司 | 用于数据库表连接的过滤器传输方法、装置及系统 |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11386089B2 (en) * | 2020-01-13 | 2022-07-12 | The Toronto-Dominion Bank | Scan optimization of column oriented storage |
US11550949B2 (en) * | 2020-07-21 | 2023-01-10 | Constella Intelligence, Inc. | Secure exchange of password presence data |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104572727A (zh) * | 2013-10-22 | 2015-04-29 | 阿里巴巴集团控股有限公司 | 一种数据查询方法及装置 |
CN105187468A (zh) * | 2014-06-11 | 2015-12-23 | 苹果公司 | 用于服务发现的动态布隆过滤器操作 |
CN105554122A (zh) * | 2015-12-18 | 2016-05-04 | 畅捷通信息技术股份有限公司 | 信息更新方法、信息更新装置、终端和服务器 |
CN105718455A (zh) * | 2014-12-01 | 2016-06-29 | 阿里巴巴集团控股有限公司 | 一种数据查询方法及装置 |
US9501527B1 (en) * | 2015-12-28 | 2016-11-22 | International Business Machines Corporation | Bloom filter construction method for use in a table join operation portion of processing a query to a distributed database |
CN106372190A (zh) * | 2016-08-31 | 2017-02-01 | 华北电力大学(保定) | 实时olap查询方法和装置 |
CN106407485A (zh) * | 2016-12-20 | 2017-02-15 | 福建六壬网安股份有限公司 | 一种基于相似度比较的url去重方法和系统 |
US20170060970A1 (en) * | 2015-08-31 | 2017-03-02 | International Business Machines Corporation | Bloom filter utilization for join processing |
CN108156154A (zh) * | 2017-12-25 | 2018-06-12 | 北京工业大学 | 命名数据网络中基于加密和布隆过滤器的访问控制方法 |
Family Cites Families (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2350875A1 (en) * | 2008-09-19 | 2011-08-03 | Oracle International Corporation | Storage-side storage request management |
US9401967B2 (en) * | 2010-06-09 | 2016-07-26 | Brocade Communications Systems, Inc. | Inline wire speed deduplication system |
CN101916261B (zh) * | 2010-07-28 | 2013-07-17 | 北京播思软件技术有限公司 | 一种分布式并行数据库系统的数据分区方法 |
US9226231B2 (en) * | 2013-01-30 | 2015-12-29 | Qualcomm Incorporated | Systems and methods for monitoring the size of a wireless network |
US10372706B2 (en) * | 2015-07-29 | 2019-08-06 | Oracle International Corporation | Tracking and maintaining expression statistics across database queries |
CN106445944A (zh) * | 2015-08-06 | 2017-02-22 | 阿里巴巴集团控股有限公司 | 一种数据查询请求的处理方法、装置及电子设备 |
CN108205571B (zh) * | 2016-12-20 | 2022-04-29 | 航天信息股份有限公司 | 键值数据表的连接方法及装置 |
US10642994B1 (en) * | 2017-05-24 | 2020-05-05 | Amazon Technologies, Inc. | Probabilistic data structures for concordance management |
US10719512B2 (en) * | 2017-10-23 | 2020-07-21 | International Business Machines Corporation | Partitioned bloom filter merge for massively parallel processing clustered data management |
US10642840B1 (en) * | 2018-03-07 | 2020-05-05 | Amazon Technologies, Inc. | Filtered hash table generation for performing hash joins |
CN109271408B (zh) * | 2018-08-31 | 2020-07-28 | 阿里巴巴集团控股有限公司 | 一种分布式数据连接处理方法、装置、设备及存储介质 |
-
2018
- 2018-08-31 CN CN201811015813.2A patent/CN109271408B/zh active Active
-
2019
- 2019-06-27 TW TW108122542A patent/TWI716016B/zh active
- 2019-07-18 WO PCT/CN2019/096478 patent/WO2020042813A1/zh active Application Filing
-
2020
- 2020-10-30 US US17/085,521 patent/US11188535B2/en active Active
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104572727A (zh) * | 2013-10-22 | 2015-04-29 | 阿里巴巴集团控股有限公司 | 一种数据查询方法及装置 |
CN105187468A (zh) * | 2014-06-11 | 2015-12-23 | 苹果公司 | 用于服务发现的动态布隆过滤器操作 |
CN105718455A (zh) * | 2014-12-01 | 2016-06-29 | 阿里巴巴集团控股有限公司 | 一种数据查询方法及装置 |
US20170060970A1 (en) * | 2015-08-31 | 2017-03-02 | International Business Machines Corporation | Bloom filter utilization for join processing |
CN105554122A (zh) * | 2015-12-18 | 2016-05-04 | 畅捷通信息技术股份有限公司 | 信息更新方法、信息更新装置、终端和服务器 |
US9501527B1 (en) * | 2015-12-28 | 2016-11-22 | International Business Machines Corporation | Bloom filter construction method for use in a table join operation portion of processing a query to a distributed database |
CN106372190A (zh) * | 2016-08-31 | 2017-02-01 | 华北电力大学(保定) | 实时olap查询方法和装置 |
CN106407485A (zh) * | 2016-12-20 | 2017-02-15 | 福建六壬网安股份有限公司 | 一种基于相似度比较的url去重方法和系统 |
CN108156154A (zh) * | 2017-12-25 | 2018-06-12 | 北京工业大学 | 命名数据网络中基于加密和布隆过滤器的访问控制方法 |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2020042813A1 (zh) * | 2018-08-31 | 2020-03-05 | 阿里巴巴集团控股有限公司 | 一种分布式数据连接处理方法、装置、设备及存储介质 |
US11188535B2 (en) | 2018-08-31 | 2021-11-30 | Beijing Oceanbase Technology Co., Ltd. | Distributed join operation processing method, apparatus, device, and storage medium |
CN110209874A (zh) * | 2019-04-24 | 2019-09-06 | 北京奇艺世纪科技有限公司 | 信息处理方法、装置、电子设备及存储介质 |
CN113360507A (zh) * | 2021-08-10 | 2021-09-07 | 蚂蚁金服(杭州)网络技术有限公司 | 用于数据库表连接的过滤器传输方法、装置及系统 |
Also Published As
Publication number | Publication date |
---|---|
US11188535B2 (en) | 2021-11-30 |
TW202011223A (zh) | 2020-03-16 |
TWI716016B (zh) | 2021-01-11 |
CN109271408B (zh) | 2020-07-28 |
WO2020042813A1 (zh) | 2020-03-05 |
US20210049173A1 (en) | 2021-02-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109271408A (zh) | 一种分布式数据连接处理方法、装置、设备及存储介质 | |
CN110347684B (zh) | 基于区块链的分级存储方法及装置、电子设备 | |
CN112887437B (zh) | 区块链交易处理方法、区块链节点和区块链系统 | |
CN113743941B (zh) | 一种在区块链中执行交易的方法、区块链和主节点 | |
TWI730690B (zh) | 在區塊鏈中同時執行交易的方法和裝置及電腦可讀儲存媒體與計算設備 | |
CN108280522A (zh) | 一种插件式分布式机器学习计算框架及其数据处理方法 | |
CN108959510B (zh) | 一种分布式数据库的分区级连接方法和装置 | |
CN106569896B (zh) | 一种数据分发及并行处理方法和系统 | |
US10515228B2 (en) | Commit and rollback of data streams provided by partially trusted entities | |
CN105095425B (zh) | 一种数据库的跨库结转方法及装置 | |
CN102609446A (zh) | 一种分布式Bloom过滤系统及其使用方法 | |
CN103440246A (zh) | 用于MapReduce的中间结果数据排序方法及系统 | |
EP3678030A1 (en) | Distributed system for executing machine learning, and method therefor | |
CN110704438B (zh) | 一种区块链中布隆过滤器的生成方法及装置 | |
WO2021036260A1 (zh) | 在区块链中并发执行交易的方法和装置 | |
CN114827165A (zh) | 对多个交易进行分组的方法和区块链节点 | |
AU2020200649B2 (en) | Apparatus and method for managing storage of primary database and replica database | |
JP4362839B1 (ja) | 仮想単一メモリストレージ上におけるメタ情報共有型分散データベース・システム | |
JP2012108875A (ja) | データベースの管理方法 | |
JP2018180688A (ja) | 更新処理プログラム、装置、及び方法 | |
CN111161064A (zh) | 一种区块链交易记录的处理方法及装置 | |
CN110399354B (zh) | 数据库的分区交换方法及装置 | |
JP2013037669A (ja) | データベースの管理方法 | |
CN112000671A (zh) | 基于区块链的数据库表处理方法、装置及系统 | |
CN109447262A (zh) | 一种cpu与gpu协同执行算法优化的方法和相关装置 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
TR01 | Transfer of patent right |
Effective date of registration: 20200924 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Patentee after: Innovative advanced technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Patentee before: Advanced innovation technology Co.,Ltd. Effective date of registration: 20200924 Address after: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Patentee after: Advanced innovation technology Co.,Ltd. Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands Patentee before: Alibaba Group Holding Ltd. |
|
TR01 | Transfer of patent right | ||
TR01 | Transfer of patent right |
Effective date of registration: 20210205 Address after: 801-10, Section B, 8th floor, 556 Xixi Road, Xihu District, Hangzhou City, Zhejiang Province Patentee after: Ant financial (Hangzhou) Network Technology Co.,Ltd. Address before: Cayman Enterprise Centre, 27 Hospital Road, George Town, Grand Cayman Islands Patentee before: Innovative advanced technology Co.,Ltd. |
|
TR01 | Transfer of patent right | ||
TR01 | Transfer of patent right |
Effective date of registration: 20210909 Address after: 100000 unit 02, floor 901, unit 1, building 1, No. 1, Middle East Third Ring Road, Chaoyang District, Beijing Patentee after: Beijing Aoxing Beisi Technology Co.,Ltd. Address before: 801-10, Section B, 8th floor, 556 Xixi Road, Xihu District, Hangzhou City, Zhejiang Province 310000 Patentee before: Ant financial (Hangzhou) Network Technology Co.,Ltd. |
|
TR01 | Transfer of patent right |