CN111936960A - 分布式存储系统中数据存储方法、装置及计算机程序产品 - Google Patents

分布式存储系统中数据存储方法、装置及计算机程序产品 Download PDF

Info

Publication number
CN111936960A
CN111936960A CN201880003863.0A CN201880003863A CN111936960A CN 111936960 A CN111936960 A CN 111936960A CN 201880003863 A CN201880003863 A CN 201880003863A CN 111936960 A CN111936960 A CN 111936960A
Authority
CN
China
Prior art keywords
data
storage node
strip
stripe
storage
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201880003863.0A
Other languages
English (en)
Other versions
CN111936960B (zh
Inventor
魏明昌
王道辉
宋驰
饶蓉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Cloud Computing Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of CN111936960A publication Critical patent/CN111936960A/zh
Application granted granted Critical
Publication of CN111936960B publication Critical patent/CN111936960B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614—Improving the reliability of storage systems
    • G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G06F3/0631—Configuration or reconfiguration of storage systems by allocating resources to storage systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/0644—Management of space entities, e.g. partitions, extents, pools
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/065—Replication mechanisms
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices
    • G06F3/0688—Non-volatile semiconductor memory arrays
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices
    • G06F3/0689—Disk arrays, e.g. RAID, JBOD

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种分布式存储系统数据存储方法，客户端生成分条的数据，将分条中的条带的数据并发地发送到条带对应的存储节点，减少存储节点间数据交互，提高了写并发度，存储节点接收客户端发送的条带的数据，并且记录相同逻辑地址对应的条带的数据的先后顺序，从而提高了分布式块存储系统的写性能，保证了数据的一致性。

Description

PCT国内申请，说明书已公开。

Claims (31)

  1. PCT国内申请，权利要求书已公开。
CN201880003863.0A 2018-12-25 2018-12-25 分布式存储系统中数据存储方法、装置及计算机程序产品 Active CN111936960B (zh)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/123349 WO2020132849A1 (zh) 2018-12-25 2018-12-25 分布式存储系统中数据存储方法、装置及计算机程序产品

Publications (2)

Publication Number Publication Date
CN111936960A true CN111936960A (zh) 2020-11-13
CN111936960B CN111936960B (zh) 2022-08-19

Family

ID=71126800

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201880003863.0A Active CN111936960B (zh) 2018-12-25 2018-12-25 分布式存储系统中数据存储方法、装置及计算机程序产品

Country Status (4)

Country Link
US (1) US11775194B2 (zh)
EP (1) EP3889752A4 (zh)
CN (1) CN111936960B (zh)
WO (1) WO2020132849A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109814805B (zh) * 2018-12-25 2020-08-25 华为技术有限公司 存储系统中分条重组的方法及分条服务器
CN113311993B (zh) * 2021-03-26 2024-04-26 阿里巴巴创新公司 数据存储方法及数据读取方法

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104932953A (zh) * 2015-06-04 2015-09-23 华为技术有限公司 一种数据分发方法、数据存储方法、相关装置以及系统
CN105242879A (zh) * 2015-09-30 2016-01-13 华为技术有限公司 一种数据存储方法以及协议服务器
CN105404469A (zh) * 2015-10-22 2016-03-16 浙江宇视科技有限公司 一种视频数据的存储方法和系统
CN105471930A (zh) * 2014-07-28 2016-04-06 浙江大华技术股份有限公司 一种读取分布式数据的方法、系统和设备
CN106662983A (zh) * 2015-12-31 2017-05-10 华为技术有限公司 分布式存储系统中的数据重建的方法、装置和系统
WO2017088572A1 (zh) * 2015-11-23 2017-06-01 华为技术有限公司 一种处理数据的方法、装置及系统
US20170293533A1 (en) * 2014-12-27 2017-10-12 Huawei Technologies Co.,Ltd. Data processing method, apparatus, and system
CN107436733A (zh) * 2017-06-29 2017-12-05 华为技术有限公司 分片管理方法和分片管理装置
CN107908499A (zh) * 2017-11-02 2018-04-13 华为技术有限公司 一种数据校验方法及装置

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7363444B2 (en) * 2005-01-10 2008-04-22 Hewlett-Packard Development Company, L.P. Method for taking snapshots of data
TWI476610B (zh) * 2008-04-29 2015-03-11 Maxiscale Inc 同級間冗餘檔案伺服器系統及方法
CN103535014B (zh) * 2013-04-23 2016-09-28 华为技术有限公司 一种网络存储系统、数据处理方法和客户端
US9671971B2 (en) * 2015-03-27 2017-06-06 Intel Corporation Managing prior versions of data for logical addresses in a storage device
CN107436725B (zh) * 2016-05-25 2019-12-20 杭州海康威视数字技术股份有限公司 一种数据写、读方法、装置及分布式对象存储集群
CN108491290B (zh) * 2018-03-28 2021-07-23 新华三技术有限公司 一种数据写入方法及装置

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105471930A (zh) * 2014-07-28 2016-04-06 浙江大华技术股份有限公司 一种读取分布式数据的方法、系统和设备
US20170293533A1 (en) * 2014-12-27 2017-10-12 Huawei Technologies Co.,Ltd. Data processing method, apparatus, and system
CN104932953A (zh) * 2015-06-04 2015-09-23 华为技术有限公司 一种数据分发方法、数据存储方法、相关装置以及系统
CN107844268A (zh) * 2015-06-04 2018-03-27 华为技术有限公司 一种数据分发方法、数据存储方法、相关装置以及系统
CN105242879A (zh) * 2015-09-30 2016-01-13 华为技术有限公司 一种数据存储方法以及协议服务器
CN105404469A (zh) * 2015-10-22 2016-03-16 浙江宇视科技有限公司 一种视频数据的存储方法和系统
WO2017088572A1 (zh) * 2015-11-23 2017-06-01 华为技术有限公司 一种处理数据的方法、装置及系统
CN106662983A (zh) * 2015-12-31 2017-05-10 华为技术有限公司 分布式存储系统中的数据重建的方法、装置和系统
US20170206135A1 (en) * 2015-12-31 2017-07-20 Huawei Technologies Co., Ltd. Data Reconstruction Method in Distributed Storage System, Apparatus, and System
CN107436733A (zh) * 2017-06-29 2017-12-05 华为技术有限公司 分片管理方法和分片管理装置
CN107908499A (zh) * 2017-11-02 2018-04-13 华为技术有限公司 一种数据校验方法及装置

Also Published As

Publication number Publication date
EP3889752A4 (en) 2021-12-01
US20210318826A1 (en) 2021-10-14
CN111936960B (zh) 2022-08-19
EP3889752A1 (en) 2021-10-06
US11775194B2 (en) 2023-10-03
WO2020132849A1 (zh) 2020-07-02

Similar Documents

Publication Publication Date Title
US10169365B2 (en) Multiple deduplication domains in network storage system
US20210365185A1 (en) Snapshot-enabled storage system implementing algorithm for efficient reclamation of snapshot storage space
JP6019513B2 (ja) 記憶リソースを共有する方法およびシステム
US7676628B1 (en) Methods, systems, and computer program products for providing access to shared storage by computing grids and clusters with large numbers of nodes
CN110612511B (zh) 使用流选择性地向分配区域中存储数据
US9176677B1 (en) Virtual provisioning space reservation
CN109144406B (zh) 分布式存储系统中元数据存储方法、系统及存储介质
WO2021017782A1 (zh) 分布式存储系统访问方法、客户端及计算机程序产品
US9946485B1 (en) Efficient data marker representation
US11899533B2 (en) Stripe reassembling method in storage system and stripe server
CN110325958B (zh) 分布式块存储系统中数据存储方法、装置及计算机可读存储介质
US11340829B1 (en) Techniques for log space management involving storing a plurality of page descriptor (PDESC) page block (PB) pairs in the log
US11775194B2 (en) Data storage method and apparatus in distributed storage system, and computer program product
CN112714910B (zh) 分布式存储系统及计算机程序产品
US11079956B2 (en) Storage system and storage control method
US10853257B1 (en) Zero detection within sub-track compression domains
CN111913664B (zh) 一种数据写入方法及装置
US8935488B2 (en) Storage system and storage control method
US11144445B1 (en) Use of compression domains that are more granular than storage allocation units
US9778850B1 (en) Techniques for zeroing non-user data areas on allocation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right

Effective date of registration: 20220228

Address after: 550025 Huawei cloud data center, jiaoxinggong Road, Qianzhong Avenue, Gui'an New District, Guiyang City, Guizhou Province

Applicant after: Huawei Cloud Computing Technology Co.,Ltd.

Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen

Applicant before: HUAWEI TECHNOLOGIES Co.,Ltd.

TA01 Transfer of patent application right
GR01 Patent grant
GR01 Patent grant