CN106649412B - 一种数据处理方法和设备 - Google Patents

一种数据处理方法和设备 Download PDF

Info

Publication number
CN106649412B
CN106649412B CN201510742561.3A CN201510742561A CN106649412B CN 106649412 B CN106649412 B CN 106649412B CN 201510742561 A CN201510742561 A CN 201510742561A CN 106649412 B CN106649412 B CN 106649412B
Authority
CN
China
Prior art keywords
data
snapshot
block
read
determining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201510742561.3A
Other languages
English (en)
Chinese (zh)
Other versions
CN106649412A (zh
Inventor
魏闯先
潘岳
郑博文
曾杰南
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Cloud Computing Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN201510742561.3A priority Critical patent/CN106649412B/zh
Priority to JP2018519365A priority patent/JP6820918B2/ja
Priority to US15/343,761 priority patent/US10789226B2/en
Priority to PCT/US2016/060616 priority patent/WO2017079619A1/en
Priority to EP16863073.9A priority patent/EP3371692A1/en
Publication of CN106649412A publication Critical patent/CN106649412A/zh
Application granted granted Critical
Publication of CN106649412B publication Critical patent/CN106649412B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22—Indexing; Data structures therefor; Storage structures
    • G06F16/2228—Indexing structures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2455—Query execution
    • G06F16/24553—Query execution of query operations
    • G06F16/24554—Unary operations; Data partitioning operations
    • G06F16/24557—Efficient disk access during query execution
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/11—File system administration, e.g. details of archiving or snapshots
    • G06F16/128—Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2358—Change logging, detection, and notification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23—Updating
    • G06F16/2372—Updates performed during offline database operations
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Computational Linguistics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Retry When Errors Occur (AREA)
CN201510742561.3A 2015-11-04 2015-11-04 一种数据处理方法和设备 Active CN106649412B (zh)

Priority Applications (5)

Application Number Priority Date Filing Date Title
CN201510742561.3A CN106649412B (zh) 2015-11-04 2015-11-04 一种数据处理方法和设备
JP2018519365A JP6820918B2 (ja) 2015-11-04 2016-11-04 データ処理のための方法および装置
US15/343,761 US10789226B2 (en) 2015-11-04 2016-11-04 Method and apparatus for data processing
PCT/US2016/060616 WO2017079619A1 (en) 2015-11-04 2016-11-04 Method and apparatus for data processing
EP16863073.9A EP3371692A1 (en) 2015-11-04 2016-11-04 Method and apparatus for data processing

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510742561.3A CN106649412B (zh) 2015-11-04 2015-11-04 一种数据处理方法和设备

Publications (2)

Publication Number Publication Date
CN106649412A CN106649412A (zh) 2017-05-10
CN106649412B true CN106649412B (zh) 2021-05-04

Family

ID=58634764

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510742561.3A Active CN106649412B (zh) 2015-11-04 2015-11-04 一种数据处理方法和设备

Country Status (5)

Country Link
US (1) US10789226B2 (enExample)
EP (1) EP3371692A1 (enExample)
JP (1) JP6820918B2 (enExample)
CN (1) CN106649412B (enExample)
WO (1) WO2017079619A1 (enExample)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107291883B (zh) * 2017-06-20 2020-09-18 苏州浪潮智能科技有限公司 一种分布式存储系统的快照方法及装置
CN107330031B (zh) * 2017-06-26 2020-07-07 广州华多网络科技有限公司 一种数据存储的方法、装置及电子设备
CN108959548B (zh) * 2018-07-02 2023-04-18 创新先进技术有限公司 业务请求的处理方法及装置
CN110109894B (zh) * 2019-03-22 2021-08-20 新奥特（北京）视频技术有限公司 非关系型数据库的实现方法、装置、存储介质和设备
CN110008067B (zh) * 2019-04-15 2020-07-07 苏州浪潮智能科技有限公司 一种可写快照链式克隆的方法、装置及分布式存储系统
CN112951437B (zh) * 2019-12-11 2023-08-08 北京懿医云科技有限公司 医疗数据管理方法、装置、存储介质与电子设备
CN114428790A (zh) * 2020-10-29 2022-05-03 深圳云天励飞技术股份有限公司 一种数据并发处理方法、装置、电子设备及存储介质
CN114706894B (zh) * 2022-04-20 2025-06-27 北京百度网讯科技有限公司 信息处理方法、装置、设备、存储介质及程序产品
CN118394979B (zh) * 2024-07-01 2024-09-17 之江实验室 用于Neo4j图数据库闪回查询方法、装置和存储介质

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103473277A (zh) * 2013-08-27 2013-12-25 华为技术有限公司 文件系统的快照方法和装置
CN103761053A (zh) * 2013-12-30 2014-04-30 华为技术有限公司 一种数据处理方法和装置

Family Cites Families (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5287496A (en) * 1991-02-25 1994-02-15 International Business Machines Corporation Dynamic, finite versioning for concurrent transaction and query processing
US5920873A (en) * 1996-12-06 1999-07-06 International Business Machines Corporation Data management control system for file and database
US6857001B2 (en) * 2002-06-07 2005-02-15 Network Appliance, Inc. Multiple concurrent active file systems
US7457982B2 (en) * 2003-04-11 2008-11-25 Network Appliance, Inc. Writable virtual disk of read-only snapshot file objects
US7243088B2 (en) * 2003-08-06 2007-07-10 Oracle International Corporation Database management system with efficient version control
US7555504B2 (en) * 2003-09-23 2009-06-30 Emc Corporation Maintenance of a file version set including read-only and read-write snapshot copies of a production file
US7587563B1 (en) * 2006-07-11 2009-09-08 Network Appliance, Inc. Method and system to make a read-only file system appear to be writeable
US7930274B2 (en) * 2007-09-12 2011-04-19 Sap Ag Dual access to concurrent data in a database management system
WO2009134772A2 (en) 2008-04-29 2009-11-05 Maxiscale, Inc Peer-to-peer redundant file server system and methods
US8489930B1 (en) * 2010-01-20 2013-07-16 Instavia Software, Inc. Method and system for creating virtual editable data objects by using a read-only data set as baseline
JP5844554B2 (ja) * 2011-06-08 2016-01-20 Ｊｆｅシステムズ株式会社 データ管理保存システム
US8600939B2 (en) * 2011-10-24 2013-12-03 Oracle International Corporation Writable snapshots
US9607017B2 (en) 2011-12-23 2017-03-28 The Arizona Board Of Regents On Behalf Of The University Of Arizona Methods of micro-specialization in database management systems
US9710475B1 (en) * 2012-07-16 2017-07-18 Tintri Inc. Synchronization of data
US8768977B2 (en) * 2012-07-31 2014-07-01 Hewlett-Packard Development Company, L.P. Data management using writeable snapshots in multi-versioned distributed B-trees
US9146877B2 (en) * 2012-11-29 2015-09-29 Infinidat Ltd. Storage system capable of managing a plurality of snapshot families and method of snapshot family based read
US9311014B2 (en) * 2012-11-29 2016-04-12 Infinidat Ltd. Storage system and methods of mapping addresses of snapshot families
WO2014130035A1 (en) * 2013-02-21 2014-08-28 Bluearc Uk Limited Object-level replication of cloned objects in a data storage system
US9817835B2 (en) * 2013-03-12 2017-11-14 Tintri Inc. Efficient data synchronization for storage containers
CN104484471B (zh) * 2014-12-31 2017-09-15 天津南大通用数据技术股份有限公司 一种高性能数据存储引擎的实现方法
US9946604B1 (en) * 2015-02-04 2018-04-17 Tintri Inc. Optimized remote cloning
US9928003B2 (en) * 2015-05-01 2018-03-27 Hewlett Packard Enterprise Development Lp Management of writable snapshots in a network storage device
US10296219B2 (en) * 2015-05-28 2019-05-21 Vmware, Inc. Data deduplication in a block-based storage system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103473277A (zh) * 2013-08-27 2013-12-25 华为技术有限公司 文件系统的快照方法和装置
CN103761053A (zh) * 2013-12-30 2014-04-30 华为技术有限公司 一种数据处理方法和装置

Also Published As

Publication number Publication date
US20170124130A1 (en) 2017-05-04
WO2017079619A1 (en) 2017-05-11
JP6820918B2 (ja) 2021-01-27
JP2018538596A (ja) 2018-12-27
CN106649412A (zh) 2017-05-10
EP3371692A4 (en) 2018-09-12
US10789226B2 (en) 2020-09-29
EP3371692A1 (en) 2018-09-12

Similar Documents

Publication Publication Date Title
CN106649412B (zh) 一种数据处理方法和设备
TWI603211B (zh) Construction of inverted index system based on Lucene, data processing method and device
KR101740271B1 (ko) 온라인 상에서 실시간으로 업데이트되는 대규모 오디오 핑거프린트 데이터베이스의 구축 방법 및 장치
US11093461B2 (en) Method for computing distinct values in analytical databases
CN104834688A (zh) 一种二级索引建立方法和装置
CN103514201A (zh) 一种非关系型数据库的数据查询方法和装置
WO2017096892A1 (zh) 索引构建方法、查询方法及对应装置、设备、计算机存储介质
EP2901344A1 (en) System and method for flexible distributed massively parallel processing (mpp) database
US11853279B2 (en) Data storage using vectors of vectors
CN113094442A (zh) 全量数据同步方法、装置、设备和介质
US20170270149A1 (en) Database systems with re-ordered replicas and methods of accessing and backing up databases
WO2017092444A1 (zh) 基于Hadoop的日志数据挖掘方法及系统
CN113918807A (zh) 数据推荐方法、装置、计算设备及计算机可读存储介质
CN110851515B (zh) 一种基于Spark分布式环境下的大数据ETL模型执行方法及介质
CN111143356B (zh) 报表检索方法及装置
CN102955808A (zh) 一种数据获取方法和分布式文件系统
CN111143582B (zh) 一种双索引实时更新联想词的多媒体资源推荐方法及装置
US20180144060A1 (en) Processing deleted edges in graph databases
US9230011B1 (en) Index-based querying of archived data sets
CN110019192B (zh) 数据库的检索方法及装置
HK1237078A (en) Data processing method and device
HK1237078A1 (en) Data processing method and device
US11853325B2 (en) Data storage using vectors of vectors
CN119691081B (zh) 关系型数据库表同步方法以及终端设备
HK1237078B (zh) 一种数据处理方法和设备

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 1237078

Country of ref document: HK

GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20250907

Address after: Room 1-2-A06, Yungu Park, No. 1008 Dengcai Street, Sandun Town, Xihu District, Hangzhou City, Zhejiang Province, 310030

Patentee after: Aliyun Computing Co.,Ltd.

Country or region after: China

Address before: A four-storey 847 mailbox in Grand Cayman Capital Building, British Cayman Islands

Patentee before: ALIBABA GROUP HOLDING Ltd.

Country or region before: Cayman Islands