CN106649412B - 一种数据处理方法和设备 - Google Patents

一种数据处理方法和设备 Download PDF

Info

Publication number
CN106649412B
CN106649412B CN201510742561.3A CN201510742561A CN106649412B CN 106649412 B CN106649412 B CN 106649412B CN 201510742561 A CN201510742561 A CN 201510742561A CN 106649412 B CN106649412 B CN 106649412B
Authority
CN
China
Prior art keywords
data
snapshot
block
read
data block
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201510742561.3A
Other languages
English (en)
Chinese (zh)
Other versions
CN106649412A (zh
Inventor
魏闯先
潘岳
郑博文
曾杰南
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Cloud Computing Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN201510742561.3A priority Critical patent/CN106649412B/zh
Priority to US15/343,761 priority patent/US10789226B2/en
Priority to PCT/US2016/060616 priority patent/WO2017079619A1/en
Priority to JP2018519365A priority patent/JP6820918B2/ja
Priority to EP16863073.9A priority patent/EP3371692A1/en
Publication of CN106649412A publication Critical patent/CN106649412A/zh
Application granted granted Critical
Publication of CN106649412B publication Critical patent/CN106649412B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G06F16/24553—Query execution of query operations
    • G06F16/24554—Unary operations; Data partitioning operations
    • G06F16/24557—Efficient disk access during query execution
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/11—File system administration, e.g. details of archiving or snapshots
    • G06F16/128—Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2358—Change logging, detection, and notification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2372—Updates performed during offline database operations
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Retry When Errors Occur (AREA)
CN201510742561.3A 2015-11-04 2015-11-04 一种数据处理方法和设备 Active CN106649412B (zh)

Priority Applications (5)

Application Number Priority Date Filing Date Title
CN201510742561.3A CN106649412B (zh) 2015-11-04 2015-11-04 一种数据处理方法和设备
US15/343,761 US10789226B2 (en) 2015-11-04 2016-11-04 Method and apparatus for data processing
PCT/US2016/060616 WO2017079619A1 (en) 2015-11-04 2016-11-04 Method and apparatus for data processing
JP2018519365A JP6820918B2 (ja) 2015-11-04 2016-11-04 データ処理のための方法および装置
EP16863073.9A EP3371692A1 (en) 2015-11-04 2016-11-04 Method and apparatus for data processing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510742561.3A CN106649412B (zh) 2015-11-04 2015-11-04 一种数据处理方法和设备

Publications (2)

Publication Number Publication Date
CN106649412A CN106649412A (zh) 2017-05-10
CN106649412B true CN106649412B (zh) 2021-05-04

Family

ID=58634764

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510742561.3A Active CN106649412B (zh) 2015-11-04 2015-11-04 一种数据处理方法和设备

Country Status (5)

Country Link
US (1) US10789226B2 (enExample)
EP (1) EP3371692A1 (enExample)
JP (1) JP6820918B2 (enExample)
CN (1) CN106649412B (enExample)
WO (1) WO2017079619A1 (enExample)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107291883B (zh) * 2017-06-20 2020-09-18 苏州浪潮智能科技有限公司 一种分布式存储系统的快照方法及装置
CN107330031B (zh) * 2017-06-26 2020-07-07 广州华多网络科技有限公司 一种数据存储的方法、装置及电子设备
CN108959548B (zh) * 2018-07-02 2023-04-18 创新先进技术有限公司 业务请求的处理方法及装置
CN110109894B (zh) * 2019-03-22 2021-08-20 新奥特（北京）视频技术有限公司 非关系型数据库的实现方法、装置、存储介质和设备
CN110008067B (zh) * 2019-04-15 2020-07-07 苏州浪潮智能科技有限公司 一种可写快照链式克隆的方法、装置及分布式存储系统
CN112951437B (zh) * 2019-12-11 2023-08-08 北京懿医云科技有限公司 医疗数据管理方法、装置、存储介质与电子设备
CN114428790A (zh) * 2020-10-29 2022-05-03 深圳云天励飞技术股份有限公司 一种数据并发处理方法、装置、电子设备及存储介质
CN114706894B (zh) * 2022-04-20 2025-06-27 北京百度网讯科技有限公司 信息处理方法、装置、设备、存储介质及程序产品
CN118394979B (zh) * 2024-07-01 2024-09-17 之江实验室 用于Neo4j图数据库闪回查询方法、装置和存储介质

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103473277A (zh) * 2013-08-27 2013-12-25 华为技术有限公司 文件系统的快照方法和装置
CN103761053A (zh) * 2013-12-30 2014-04-30 华为技术有限公司 一种数据处理方法和装置

Family Cites Families (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5287496A (en) * 1991-02-25 1994-02-15 International Business Machines Corporation Dynamic, finite versioning for concurrent transaction and query processing
US5920873A (en) 1996-12-06 1999-07-06 International Business Machines Corporation Data management control system for file and database
US6857001B2 (en) * 2002-06-07 2005-02-15 Network Appliance, Inc. Multiple concurrent active file systems
US7457982B2 (en) * 2003-04-11 2008-11-25 Network Appliance, Inc. Writable virtual disk of read-only snapshot file objects
US7243088B2 (en) * 2003-08-06 2007-07-10 Oracle International Corporation Database management system with efficient version control
US7555504B2 (en) * 2003-09-23 2009-06-30 Emc Corporation Maintenance of a file version set including read-only and read-write snapshot copies of a production file
US7587563B1 (en) * 2006-07-11 2009-09-08 Network Appliance, Inc. Method and system to make a read-only file system appear to be writeable
US7930274B2 (en) * 2007-09-12 2011-04-19 Sap Ag Dual access to concurrent data in a database management system
US8296398B2 (en) 2008-04-29 2012-10-23 Overland Storage, Inc. Peer-to-peer redundant file server system and methods
US8489930B1 (en) * 2010-01-20 2013-07-16 Instavia Software, Inc. Method and system for creating virtual editable data objects by using a read-only data set as baseline
JP5844554B2 (ja) * 2011-06-08 2016-01-20 Ｊｆｅシステムズ株式会社 データ管理保存システム
US8600939B2 (en) * 2011-10-24 2013-12-03 Oracle International Corporation Writable snapshots
EP2795484A4 (en) * 2011-12-23 2015-11-11 Univ Arizona State METHOD OF MICRO-SPECIALIZATION IN DATABASE MANAGEMENT SYSTEMS
US9529808B1 (en) * 2012-07-16 2016-12-27 Tintri Inc. Efficient and flexible organization and management of file metadata
US8768977B2 (en) * 2012-07-31 2014-07-01 Hewlett-Packard Development Company, L.P. Data management using writeable snapshots in multi-versioned distributed B-trees
US9311014B2 (en) * 2012-11-29 2016-04-12 Infinidat Ltd. Storage system and methods of mapping addresses of snapshot families
US9146877B2 (en) * 2012-11-29 2015-09-29 Infinidat Ltd. Storage system capable of managing a plurality of snapshot families and method of snapshot family based read
US10311151B2 (en) * 2013-02-21 2019-06-04 Hitachi Vantara Corporation Object-level replication of cloned objects in a data storage system
US9817835B2 (en) * 2013-03-12 2017-11-14 Tintri Inc. Efficient data synchronization for storage containers
CN104484471B (zh) * 2014-12-31 2017-09-15 天津南大通用数据技术股份有限公司 一种高性能数据存储引擎的实现方法
US9946604B1 (en) * 2015-02-04 2018-04-17 Tintri Inc. Optimized remote cloning
US9928003B2 (en) * 2015-05-01 2018-03-27 Hewlett Packard Enterprise Development Lp Management of writable snapshots in a network storage device
US10296219B2 (en) * 2015-05-28 2019-05-21 Vmware, Inc. Data deduplication in a block-based storage system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103473277A (zh) * 2013-08-27 2013-12-25 华为技术有限公司 文件系统的快照方法和装置
CN103761053A (zh) * 2013-12-30 2014-04-30 华为技术有限公司 一种数据处理方法和装置

Also Published As

Publication number Publication date
JP6820918B2 (ja) 2021-01-27
CN106649412A (zh) 2017-05-10
US10789226B2 (en) 2020-09-29
EP3371692A4 (en) 2018-09-12
WO2017079619A1 (en) 2017-05-11
JP2018538596A (ja) 2018-12-27
US20170124130A1 (en) 2017-05-04
EP3371692A1 (en) 2018-09-12

Similar Documents

Publication Publication Date Title
CN106649412B (zh) 一种数据处理方法和设备
US12277141B2 (en) System and method for analysis and management of data distribution in a distributed database environment
US9898475B1 (en) Tiering with pluggable storage system for parallel query engines
US7353227B2 (en) Conflict resolution engine
CN103514201B (zh) 一种非关系型数据库的数据查询方法和装置
TWI603211B (zh) Construction of inverted index system based on Lucene, data processing method and device
CN113094442B (zh) 全量数据同步方法、装置、设备和介质
KR101740271B1 (ko) 온라인 상에서 실시간으로 업데이트되는 대규모 오디오 핑거프린트 데이터베이스의 구축 방법 및 장치
CN104133867A (zh) 分布式顺序表片内二级索引方法及系统
CN104834688A (zh) 一种二级索引建立方法和装置
US11853279B2 (en) Data storage using vectors of vectors
WO2017096892A1 (zh) 索引构建方法、查询方法及对应装置、设备、计算机存储介质
US20180113889A1 (en) Method and Computing Distinct Values in Analytical Databases
WO2014067449A1 (en) System and method for flexible distributed massively parallel processing (mpp) database
US11416466B2 (en) Data edge platform for improved storage and analytics
US20170270149A1 (en) Database systems with re-ordered replicas and methods of accessing and backing up databases
CN106649602A (zh) 业务对象数据处理方法、装置和服务器
CN111143356B (zh) 报表检索方法及装置
CN110851515B (zh) 一种基于Spark分布式环境下的大数据ETL模型执行方法及介质
CN102955808A (zh) 一种数据获取方法和分布式文件系统
CN110019192B (zh) 数据库的检索方法及装置
US20180144060A1 (en) Processing deleted edges in graph databases
CN115658841B (zh) 一种数据管理方法、装置、计算设备及存储介质
HK1237078A1 (en) Data processing method and device
HK1237078A (en) Data processing method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 1237078

Country of ref document: HK

GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20250907

Address after: Room 1-2-A06, Yungu Park, No. 1008 Dengcai Street, Sandun Town, Xihu District, Hangzhou City, Zhejiang Province, 310030

Patentee after: Aliyun Computing Co.,Ltd.

Country or region after: China

Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands

Patentee before: ALIBABA GROUP HOLDING Ltd.

Country or region before: Cayman Islands