CN113544635A - 存储系统中数据处理方法、装置以及存储系统 - Google Patents

存储系统中数据处理方法、装置以及存储系统 Download PDF

Info

Publication number
CN113544635A
CN113544635A CN201980092966.3A CN201980092966A CN113544635A CN 113544635 A CN113544635 A CN 113544635A CN 201980092966 A CN201980092966 A CN 201980092966A CN 113544635 A CN113544635 A CN 113544635A
Authority
CN
China
Prior art keywords
data
metadata
storage node
chunk
stripe
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201980092966.3A
Other languages
English (en)
Inventor
王道辉
宋驰
王同雷
湛云
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Cloud Computing Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of CN113544635A publication Critical patent/CN113544635A/zh
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1076—Parity data used in redundant arrays of independent storages, e.g. in RAID systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1076—Parity data used in redundant arrays of independent storages, e.g. in RAID systems
    • G06F11/108—Parity data distribution in semiconductor storages, e.g. in SSD
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G06F11/1402—Saving, restoring, recovering or retrying
    • G06F11/1471—Saving, restoring, recovering or retrying involving logging of persistent data for recovery
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/835—Timestamp

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

一种存储系统中数据处理方法以及存储系统，涉及数据存储技术领域，通过客户端将条带中的数据分块及数据分块的元数据发送至数据存储节点，将数据分块的元数据以及校验分块发送至校验存储节点，从而将数据分块的元数据在校验存储节点上实现了备份，从而降低了写操作的时间。

Description

PCT国内申请，说明书已公开。

Claims (31)

  1. PCT国内申请，权利要求书已公开。
CN201980092966.3A 2019-09-09 2019-09-09 存储系统中数据处理方法、装置以及存储系统 Pending CN113544635A (zh)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/104981 WO2021046693A1 (zh) 2019-09-09 2019-09-09 存储系统中数据处理方法、装置以及存储系统

Publications (1)

Publication Number Publication Date
CN113544635A true CN113544635A (zh) 2021-10-22

Family

ID=74865933

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201980092966.3A Pending CN113544635A (zh) 2019-09-09 2019-09-09 存储系统中数据处理方法、装置以及存储系统

Country Status (4)

Country Link
US (1) US20220129346A1 (zh)
EP (1) EP3971701A4 (zh)
CN (1) CN113544635A (zh)
WO (1) WO2021046693A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115098046A (zh) * 2022-08-26 2022-09-23 苏州浪潮智能科技有限公司 磁盘阵列初始化方法、系统、电子设备及存储介质

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114968668A (zh) * 2022-06-17 2022-08-30 重庆紫光华山智安科技有限公司 数据处理方法、装置、数据接入端及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102521269A (zh) * 2011-11-22 2012-06-27 清华大学 一种基于索引的计算机连续数据保护方法
CN105353988A (zh) * 2015-11-13 2016-02-24 曙光信息产业(北京)有限公司 一种元数据读写方法及装置
US20170206135A1 (en) * 2015-12-31 2017-07-20 Huawei Technologies Co., Ltd. Data Reconstruction Method in Distributed Storage System, Apparatus, and System
CN107423422A (zh) * 2017-08-01 2017-12-01 武大吉奥信息技术有限公司 基于网格的空间数据分布式存储及检索方法和系统
CN109144406A (zh) * 2017-06-28 2019-01-04 华为技术有限公司 分布式存储系统中元数据存储方法、系统及存储介质
CN109213420A (zh) * 2017-06-29 2019-01-15 杭州海康威视数字技术股份有限公司 数据存储方法、装置及系统
CN109542342A (zh) * 2018-11-09 2019-03-29 锐捷网络股份有限公司 元数据管理与数据重构方法、设备及存储介质

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101694984B1 (ko) * 2010-12-08 2017-01-11 한국전자통신연구원 비대칭 클러스터링 파일시스템에서의 패리티 산출 방법
KR20150061258A (ko) * 2013-11-27 2015-06-04 한국전자통신연구원 분산 raid 시스템에서 패리티 청크 운용 방법과 이를 지원하는 데이터 서버 장치
CN103699494B (zh) * 2013-12-06 2017-03-15 北京奇虎科技有限公司 一种数据存储方法、数据存储设备和分布式存储系统
US10608784B2 (en) * 2016-03-15 2020-03-31 ClineHair Commercial Endeavors Distributed storage system data management and security
US9495255B2 (en) * 2014-08-07 2016-11-15 Pure Storage, Inc. Error recovery in a storage cluster
US10394484B2 (en) * 2016-02-26 2019-08-27 Hitachi, Ltd. Storage system
US9672905B1 (en) * 2016-07-22 2017-06-06 Pure Storage, Inc. Optimize data protection layouts based on distributed flash wear leveling
JP6798007B2 (ja) * 2017-03-28 2020-12-09 株式会社日立製作所 ストレージシステム、コンピュータ読み取り可能な記録媒体、システムの制御方法
US10635529B2 (en) * 2017-05-25 2020-04-28 Western Digital Technologies, Inc. Parity offload for multiple data storage devices
CN109814805B (zh) * 2018-12-25 2020-08-25 华为技术有限公司 存储系统中分条重组的方法及分条服务器
US11074129B2 (en) * 2019-10-31 2021-07-27 Western Digital Technologies, Inc. Erasure coded data shards containing multiple data objects

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102521269A (zh) * 2011-11-22 2012-06-27 清华大学 一种基于索引的计算机连续数据保护方法
CN105353988A (zh) * 2015-11-13 2016-02-24 曙光信息产业(北京)有限公司 一种元数据读写方法及装置
US20170206135A1 (en) * 2015-12-31 2017-07-20 Huawei Technologies Co., Ltd. Data Reconstruction Method in Distributed Storage System, Apparatus, and System
CN109144406A (zh) * 2017-06-28 2019-01-04 华为技术有限公司 分布式存储系统中元数据存储方法、系统及存储介质
CN109213420A (zh) * 2017-06-29 2019-01-15 杭州海康威视数字技术股份有限公司 数据存储方法、装置及系统
CN107423422A (zh) * 2017-08-01 2017-12-01 武大吉奥信息技术有限公司 基于网格的空间数据分布式存储及检索方法和系统
CN109542342A (zh) * 2018-11-09 2019-03-29 锐捷网络股份有限公司 元数据管理与数据重构方法、设备及存储介质

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115098046A (zh) * 2022-08-26 2022-09-23 苏州浪潮智能科技有限公司 磁盘阵列初始化方法、系统、电子设备及存储介质
CN115098046B (zh) * 2022-08-26 2023-01-24 苏州浪潮智能科技有限公司 磁盘阵列初始化方法、系统、电子设备及存储介质

Also Published As

Publication number Publication date
WO2021046693A1 (zh) 2021-03-18
US20220129346A1 (en) 2022-04-28
EP3971701A1 (en) 2022-03-23
EP3971701A4 (en) 2022-06-15

Similar Documents

Publication Publication Date Title
EP3726364B1 (en) Data write-in method and solid-state drive array
US8972779B2 (en) Method of calculating parity in asymetric clustering file system
US11698728B2 (en) Data updating technology
US10778762B2 (en) Cloud computing service architecture
CN109582213B (zh) 数据重构方法及装置、数据存储系统
US20220129346A1 (en) Data processing method and apparatus in storage system, and storage system
CN103761190A (zh) 数据处理方法及装置
WO2019184012A1 (zh) 数据写入方法、客户端服务器和系统
CN112000277B (zh) 精简置备文件拷贝方法、装置、设备及可读存储介质
CN111949210A (zh) 分布式存储系统中元数据存储方法、系统及存储介质
JP2022552804A (ja) データ・ストレージ・システムにおけるガベージ・コレクション
EP3495939B1 (en) Method and device for storing data in distributed block storage system, and computer readable storage medium
US11775194B2 (en) Data storage method and apparatus in distributed storage system, and computer program product
CN109814805B (zh) 存储系统中分条重组的方法及分条服务器
US11487428B2 (en) Storage control apparatus and storage control method
CN113805789A (zh) 存储设备中的元数据处理方法及相关设备
US20230350760A1 (en) Physical size api for snapshots backed up to object store
CN115374057A (zh) 数据快照方法及相关设备
CN109508255B (zh) 一种数据处理的方法及装置
CN112714910B (zh) 分布式存储系统及计算机程序产品
CN108023914B (zh) 一种内存数据共享系统、内存数据的写入以及读取方法
TWI826093B (zh) 虛擬機器備援方法及系統
US20220035560A1 (en) Storage System, Storage Node, and Data Storage Method
JP6605762B2 (ja) 記憶ドライブの故障により消失したデータを復元する装置
CN117812095A (zh) 多数据副本数据一致性的检测方法、装置、设备及介质

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
TA01 Transfer of patent application right

Effective date of registration: 20220211

Address after: 550025 Huawei cloud data center, jiaoxinggong Road, Qianzhong Avenue, Gui'an New District, Guiyang City, Guizhou Province

Applicant after: Huawei Cloud Computing Technology Co.,Ltd.

Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen

Applicant before: HUAWEI TECHNOLOGIES Co.,Ltd.

TA01 Transfer of patent application right