SG11201707304RA - Data check method and storage system - Google Patents

Data check method and storage system

Info

Publication number
SG11201707304RA
SG11201707304RA SG11201707304RA SG11201707304RA SG11201707304RA SG 11201707304R A SG11201707304R A SG 11201707304RA SG 11201707304R A SG11201707304R A SG 11201707304RA SG 11201707304R A SG11201707304R A SG 11201707304RA SG 11201707304R A SG11201707304R A SG 11201707304RA
Authority
SG
Singapore
Prior art keywords
data
node
data write
check
storage system
Prior art date
Application number
SG11201707304RA
Inventor
Mingchang Wei
Original Assignee
Huawei Tech Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Tech Co Ltd filed Critical Huawei Tech Co Ltd
Publication of SG11201707304RA publication Critical patent/SG11201707304RA/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/061—Improving I/O performance
    • G06F3/0611—Improving I/O performance in relation to response time
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1076—Parity data used in redundant arrays of independent storages, e.g. in RAID systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0223—User address space allocation, e.g. contiguous or non contiguous base addressing
    • G06F12/023—Free address space management
    • G06F12/0253—Garbage collection, i.e. reclamation of unreferenced memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/38—Information transfer, e.g. on bus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614—Improving the reliability of storage systems
    • G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0655—Vertical data movement, i.e. input-output transfer; data movement between one or more hosts and one or more storage devices
    • G06F3/0659—Command handling arrangements, e.g. command buffers, queues, command scheduling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices
    • G06F3/0688—Non-volatile semiconductor memory arrays
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices
    • G06F3/0689—Disk arrays, e.g. RAID, JBOD
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/10—Providing a specific technical effect
    • G06F2212/1016—Performance improvement
    • G06F2212/1024—Latency reduction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/10—Providing a specific technical effect
    • G06F2212/1032—Reliability improvement, data loss prevention, degraded operation etc
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/15—Use in a specific computing environment
    • G06F2212/154—Networked environment
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2212/00—Indexing scheme relating to accessing, addressing or allocation within memory systems or architectures
    • G06F2212/25—Using a specific main memory architecture
    • G06F2212/254—Distributed memory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2213/00—Indexing scheme relating to interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F2213/38—Universal adapter
    • G06F2213/3854—Control is performed at the peripheral side

Abstract

DATA CHECK METHOD AND STORAGE SYSTEM A storage system includes a host, a check node, and multiple data nodes. The host is configured to: divide multiple data write requests into multiple data write request sets according to data node identifiers included in the data write requests, where each data write 5 request set includes multiple data write requests having a same data node identifier; and when sizes of all data write requests in each of a specified quantity of data write request sets reach a preset data amount, calculate check data of the specified quantity of data write request sets. The host is further configured to: send each data write request set to a data node indicated by an identifier, and send the check data to the check node. Therefore, on a basis of ensuring 10 data reliability, data of a same type can be stored in a same storage node and the data can be read from only one storage node, thereby improving data reading efficiency. 15 Figure 3
SG11201707304RA 2016-11-25 2016-11-25 Data check method and storage system SG11201707304RA (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2016/107355 WO2018094704A1 (en) 2016-11-25 2016-11-25 Data check method and storage system

Publications (1)

Publication Number Publication Date
SG11201707304RA true SG11201707304RA (en) 2018-06-28

Family

ID=62188856

Family Applications (1)

Application Number Title Priority Date Filing Date
SG11201707304RA SG11201707304RA (en) 2016-11-25 2016-11-25 Data check method and storage system

Country Status (9)

Country Link
US (1) US10303374B2 (en)
EP (1) EP3352071B1 (en)
JP (1) JP6526235B2 (en)
CN (1) CN109074227B (en)
AU (1) AU2016397189B2 (en)
BR (1) BR112017020736B8 (en)
CA (1) CA2978927C (en)
SG (1) SG11201707304RA (en)
WO (1) WO2018094704A1 (en)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111949434B (en) * 2019-05-17 2022-06-14 华为技术有限公司 RAID management method, RAID controller and system
US20210357366A1 (en) * 2020-05-12 2021-11-18 Hewlett Packard Enterprise Development Lp File system metadata
CN112365124A (en) * 2020-10-16 2021-02-12 重庆恢恢信息技术有限公司 Intelligent construction site construction material rapid distribution working method
CN114697349A (en) * 2020-12-28 2022-07-01 华为技术有限公司 Method for processing data by using intermediate device, computer system and intermediate device
CN113419684B (en) * 2021-07-09 2023-02-24 深圳大普微电子科技有限公司 Data processing method, device and equipment and readable storage medium
CN116166179A (en) * 2021-11-25 2023-05-26 华为技术有限公司 Data storage system, intelligent network card and computing node
CN115933995B (en) * 2023-01-09 2023-05-09 苏州浪潮智能科技有限公司 Method and device for writing data into solid state disk, electronic equipment and readable medium

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3713788B2 (en) * 1996-02-28 2005-11-09 株式会社日立製作所 Storage device and storage device system
US6742081B2 (en) * 2001-04-30 2004-05-25 Sun Microsystems, Inc. Data storage array employing block checksums and dynamic striping
JP4383321B2 (en) * 2004-11-09 2009-12-16 富士通株式会社 Storage control device and external storage device
US8332608B2 (en) * 2008-09-19 2012-12-11 Mediatek Inc. Method of enhancing command executing performance of disc drive
JP5317827B2 (en) * 2009-05-19 2013-10-16 日本電信電話株式会社 Distributed data management apparatus, method and program
CN102546755A (en) * 2011-12-12 2012-07-04 华中科技大学 Data storage method of cloud storage system
CN102968498B (en) * 2012-12-05 2016-08-10 华为技术有限公司 Data processing method and device
JP6005533B2 (en) * 2013-01-17 2016-10-12 株式会社東芝 Storage device and storage method
JP2014203233A (en) * 2013-04-04 2014-10-27 株式会社日立製作所 Storage system and method for updating data in storage system
CN104216664B (en) * 2013-06-26 2017-06-09 华为技术有限公司 Network volume creation method, date storage method, storage device and storage system
CN103647797A (en) * 2013-11-15 2014-03-19 北京邮电大学 Distributed file system and data access method thereof
CN103699494B (en) 2013-12-06 2017-03-15 北京奇虎科技有限公司 A kind of date storage method, data storage device and distributed memory system
CA2897129C (en) * 2013-12-31 2022-03-15 Huawei Technologies Co., Ltd. Data processing method and device in distributed file storage system
CN103761058B (en) * 2014-01-23 2016-08-17 天津中科蓝鲸信息技术有限公司 RAID1 and RAID4 mixed structure network store system and method
JP2016184372A (en) 2015-03-27 2016-10-20 富士通株式会社 Storage system, information processing device, parity generation program, and parity generation method
CN107844268B (en) * 2015-06-04 2021-09-14 华为技术有限公司 Data distribution method, data storage method, related device and system
CN105404469B (en) * 2015-10-22 2018-11-13 浙江宇视科技有限公司 A kind of storage method and system of video data
CN105930103B (en) * 2016-05-10 2019-04-16 南京大学 A kind of correcting and eleting codes covering write method of distributed storage CEPH

Also Published As

Publication number Publication date
EP3352071B1 (en) 2019-08-28
AU2016397189B2 (en) 2019-07-25
WO2018094704A1 (en) 2018-05-31
EP3352071A4 (en) 2018-07-25
US20180364920A1 (en) 2018-12-20
BR112017020736B8 (en) 2021-08-17
BR112017020736A2 (en) 2018-07-17
AU2016397189A1 (en) 2018-06-14
US10303374B2 (en) 2019-05-28
CA2978927A1 (en) 2018-05-25
EP3352071A1 (en) 2018-07-25
JP2019504369A (en) 2019-02-14
CA2978927C (en) 2019-09-17
CN109074227B (en) 2020-06-16
BR112017020736B1 (en) 2021-01-19
JP6526235B2 (en) 2019-06-05
CN109074227A (en) 2018-12-21

Similar Documents

Publication Publication Date Title
SG11201707304RA (en) Data check method and storage system
MX2023000186A (en) Systems and methods for creating a universal record.
RU2015104116A (en) METHOD AND STORAGE DEVICE FOR DISTRIBUTED FILE SYSTEM
WO2015066719A3 (en) Use of solid state storage devices and the like in data deduplication
SG11201907619VA (en) Blockchain data storage system and method
SG11201907942QA (en) Blockchain cluster processing system and method, computer device and storage medium
PH12017500241A1 (en) File access method, device and storage system
EP3528124A3 (en) Partition tolerance in cluster membership management
EP3361409B1 (en) Data deduplication method and apparatus
EP4242821A3 (en) Distributed transactions with token-associated execution
CN106687911A8 (en) The online data movement of data integrity is not damaged
GB2550800A (en) Managing data in storage according to a log structure
SA516370693B1 (en) Ascertaining command completion in flash memories
KR20160056380A (en) Storage device and operating method of storage device
WO2016065229A3 (en) Access control for data blocks in a distributed filesystem
GB2492870A (en) Optimizing a file system for different types of applications in a compute cluster using dynamic block size granularity
CN104462240A (en) Method and system for realizing hierarchical storage and management in cloud storage
BR112015018922A8 (en) device, method and one or more computer readable non-transient storage media for routine estimation
WO2014114817A3 (en) System and method for replicating data
EP4273688A3 (en) Data access method, device and system
CN105677595A (en) FPGA method achieving computation speedup and PCIESSD storage simultaneously
SG11201901608VA (en) Method for accessing distributed storage system, related apparatus, and related system
CN103473266A (en) Solid state disk and method for deleting repeating data thereof
GB2532667A (en) Memory management
JP2015057696A5 (en)