CN104541251B - 在存储群集中的擦除编码和复制 - Google Patents

在存储群集中的擦除编码和复制 Download PDF

Info

Publication number
CN104541251B
CN104541251B CN201380043101.0A CN201380043101A CN104541251B CN 104541251 B CN104541251 B CN 104541251B CN 201380043101 A CN201380043101 A CN 201380043101A CN 104541251 B CN104541251 B CN 104541251B
Authority
CN
China
Prior art keywords
storage cluster
digital object
cluster
segment
disk
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201380043101.0A
Other languages
English (en)
Chinese (zh)
Other versions
CN104541251A (zh
Inventor
D.贝克
P.R.M.卡彭捷
A.克拉格
A.皮尔斯
J.林
R.特平
D.约亚克利
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
HyperTrust NV
Original Assignee
HyperTrust NV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by HyperTrust NV filed Critical HyperTrust NV
Publication of CN104541251A publication Critical patent/CN104541251A/zh
Application granted granted Critical
Publication of CN104541251B publication Critical patent/CN104541251B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1076—Parity data used in redundant arrays of independent storages, e.g. in RAID systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604—Improving or facilitating administration, e.g. storage management
    • G06F3/0605—Improving or facilitating administration, e.g. storage management by facilitating the interaction with a user or administrator
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0614—Improving the reliability of storage systems
    • G06F3/0619—Improving the reliability of storage systems in relation to data integrity, e.g. data losses, bit errors
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629—Configuration or reconfiguration of storage systems
    • G06F3/0632—Configuration or reconfiguration of storage systems by initialisation or re-initialisation of storage systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638—Organizing or formatting or addressing of data
    • G06F3/064—Management of blocks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
    • G06F3/065—Replication mechanisms
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0662—Virtualisation aspects
    • G06F3/0667—Virtualisation aspects at data level, e.g. file, record or object virtualisation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/0671—In-line storage system
    • G06F3/0683—Plurality of storage devices
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/03—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words
    • H03M13/05—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words using block codes, i.e. a predetermined number of check bits joined to a predetermined number of information bits
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/25—Error detection or forward error correction by signal space coding, i.e. adding redundancy in the signal constellation, e.g. Trellis Coded Modulation [TCM]
    • H03M13/251—Error detection or forward error correction by signal space coding, i.e. adding redundancy in the signal constellation, e.g. Trellis Coded Modulation [TCM] with block coding
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/27—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes using interleaving techniques
    • H03M13/2703—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes using interleaving techniques the interleaver involving at least two directions
    • H03M13/2707—Simple row-column interleaver, i.e. pure block interleaving
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/35—Unequal or adaptive error protection, e.g. by providing a different level of protection according to significance of source information or by adapting the coding according to the change of transmission channel characteristics
    • H03M13/353—Adaptation to the channel
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/35—Unequal or adaptive error protection, e.g. by providing a different level of protection according to significance of source information or by adapting the coding according to the change of transmission channel characteristics
    • H03M13/356—Unequal error protection [UEP]
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/37—Decoding methods or techniques, not specific to the particular type of coding provided for in groups H03M13/03 - H03M13/35
    • H03M13/373—Decoding methods or techniques, not specific to the particular type of coding provided for in groups H03M13/03 - H03M13/35 with erasure correction and erasure determination, e.g. for packet loss recovery or setting of erasures for the decoding of Reed-Solomon codes
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/37—Decoding methods or techniques, not specific to the particular type of coding provided for in groups H03M13/03 - H03M13/35
    • H03M13/3761—Decoding methods or techniques, not specific to the particular type of coding provided for in groups H03M13/03 - H03M13/35 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14—Error detection or correction of the data by redundancy in operation
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/25—Error detection or forward error correction by signal space coding, i.e. adding redundancy in the signal constellation, e.g. Trellis Coded Modulation [TCM]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Probability & Statistics with Applications (AREA)
  • Quality & Reliability (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
CN201380043101.0A 2012-06-13 2013-06-04 在存储群集中的擦除编码和复制 Active CN104541251B (zh)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US13/517,527 US8799746B2 (en) 2012-06-13 2012-06-13 Erasure coding and replication in storage clusters
US13/517527 2012-06-13
PCT/US2013/044045 WO2013188169A1 (en) 2012-06-13 2013-06-04 Erasure coding and replication in storage clusters

Publications (2)

Publication Number Publication Date
CN104541251A CN104541251A (zh) 2015-04-22
CN104541251B true CN104541251B (zh) 2017-09-05

Family

ID=49757125

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201380043101.0A Active CN104541251B (zh) 2012-06-13 2013-06-04 在存储群集中的擦除编码和复制

Country Status (7)

Country Link
US (4) US8799746B2 (enExample)
EP (1) EP2862085A4 (enExample)
JP (1) JP6280108B2 (enExample)
CN (1) CN104541251B (enExample)
AU (1) AU2013274626A1 (enExample)
CA (1) CA2876638A1 (enExample)
WO (1) WO2013188169A1 (enExample)

Families Citing this family (333)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12008266B2 (en) 2010-09-15 2024-06-11 Pure Storage, Inc. Efficient read by reconstruction
US11614893B2 (en) 2010-09-15 2023-03-28 Pure Storage, Inc. Optimizing storage device access based on latency
US8510267B2 (en) * 2011-03-08 2013-08-13 Rackspace Us, Inc. Synchronization of structured information repositories
US8589640B2 (en) 2011-10-14 2013-11-19 Pure Storage, Inc. Method for maintaining multiple fingerprint tables in a deduplicating storage system
US9146969B2 (en) * 2012-11-26 2015-09-29 The Boeing Company System and method of reduction of irrelevant information during search
US9558206B2 (en) * 2013-02-25 2017-01-31 Datadirect Networks, Inc. Asymmetric distributed data storage system
US9354991B2 (en) * 2013-06-25 2016-05-31 Microsoft Technology Licensing, Llc Locally generated simple erasure codes
US9313021B2 (en) * 2013-08-29 2016-04-12 National Chiao Tung University Secret communication method with self-authentication capability
KR102238590B1 (ko) * 2013-11-29 2021-04-09 삼성전자주식회사 데이터 패킷 전송의 인증 및 검증 방법, 및 상기 방법을 수행하는 장치들
US10685037B2 (en) * 2013-12-18 2020-06-16 Amazon Technology, Inc. Volume cohorts in object-redundant storage systems
US10620830B2 (en) * 2013-12-18 2020-04-14 Amazon Technologies, Inc. Reconciling volumelets in volume cohorts
US8874835B1 (en) 2014-01-16 2014-10-28 Pure Storage, Inc. Data placement based on data properties in a tiered storage device system
US9344120B2 (en) * 2014-02-13 2016-05-17 Quantum Corporation Adjusting redundancy in an erasure code object store to account for varying data value
US10608784B2 (en) 2016-03-15 2020-03-31 ClineHair Commercial Endeavors Distributed storage system data management and security
US9003144B1 (en) 2014-06-04 2015-04-07 Pure Storage, Inc. Mechanism for persisting messages in a storage system
US11652884B2 (en) 2014-06-04 2023-05-16 Pure Storage, Inc. Customized hash algorithms
US9213485B1 (en) 2014-06-04 2015-12-15 Pure Storage, Inc. Storage system architecture
US11068363B1 (en) 2014-06-04 2021-07-20 Pure Storage, Inc. Proactively rebuilding data in a storage cluster
AU2015269370B2 (en) * 2014-06-04 2019-06-06 Pure Storage, Inc. Automatically reconfiguring a storage memory topology
US9612952B2 (en) 2014-06-04 2017-04-04 Pure Storage, Inc. Automatically reconfiguring a storage memory topology
WO2015188008A1 (en) * 2014-06-04 2015-12-10 Pure Storage, Inc. Mechanism for persisting messages in a storage system
US11399063B2 (en) 2014-06-04 2022-07-26 Pure Storage, Inc. Network authentication for a storage system
US12341848B2 (en) 2014-06-04 2025-06-24 Pure Storage, Inc. Distributed protocol endpoint services for data storage systems
US9218244B1 (en) * 2014-06-04 2015-12-22 Pure Storage, Inc. Rebuilding data across storage nodes
US9836234B2 (en) 2014-06-04 2017-12-05 Pure Storage, Inc. Storage cluster
US10574754B1 (en) 2014-06-04 2020-02-25 Pure Storage, Inc. Multi-chassis array with multi-level load balancing
US12137140B2 (en) 2014-06-04 2024-11-05 Pure Storage, Inc. Scale out storage platform having active failover
US11960371B2 (en) 2014-06-04 2024-04-16 Pure Storage, Inc. Message persistence in a zoned system
US8850108B1 (en) * 2014-06-04 2014-09-30 Pure Storage, Inc. Storage cluster
US9367243B1 (en) 2014-06-04 2016-06-14 Pure Storage, Inc. Scalable non-uniform storage sizes
US10114757B2 (en) 2014-07-02 2018-10-30 Pure Storage, Inc. Nonrepeating identifiers in an address space of a non-volatile solid-state storage
US11886308B2 (en) 2014-07-02 2024-01-30 Pure Storage, Inc. Dual class of service for unified file and object messaging
US11604598B2 (en) 2014-07-02 2023-03-14 Pure Storage, Inc. Storage cluster with zoned drives
US9021297B1 (en) 2014-07-02 2015-04-28 Pure Storage, Inc. Redundant, fault-tolerant, distributed remote procedure call cache in a storage system
US9836245B2 (en) 2014-07-02 2017-12-05 Pure Storage, Inc. Non-volatile RAM and flash memory in a non-volatile solid-state storage
US8868825B1 (en) 2014-07-02 2014-10-21 Pure Storage, Inc. Nonrepeating identifiers in an address space of a non-volatile solid-state storage
US12182044B2 (en) 2014-07-03 2024-12-31 Pure Storage, Inc. Data storage in a zone drive
US8874836B1 (en) 2014-07-03 2014-10-28 Pure Storage, Inc. Scheduling policy for queues in a non-volatile solid-state storage
US10853311B1 (en) * 2014-07-03 2020-12-01 Pure Storage, Inc. Administration through files in a storage system
US9747229B1 (en) 2014-07-03 2017-08-29 Pure Storage, Inc. Self-describing data format for DMA in a non-volatile solid-state storage
US9811677B2 (en) 2014-07-03 2017-11-07 Pure Storage, Inc. Secure data replication in a storage grid
US9817750B2 (en) * 2014-07-03 2017-11-14 Pure Storage, Inc. Profile-dependent write placement of data into a non-volatile solid-state storage
US9082512B1 (en) 2014-08-07 2015-07-14 Pure Storage, Inc. Die-level monitoring in a storage cluster
US9495255B2 (en) 2014-08-07 2016-11-15 Pure Storage, Inc. Error recovery in a storage cluster
US9483346B2 (en) 2014-08-07 2016-11-01 Pure Storage, Inc. Data rebuild on feedback from a queue in a non-volatile solid-state storage
US12158814B2 (en) 2014-08-07 2024-12-03 Pure Storage, Inc. Granular voltage tuning
US9558069B2 (en) 2014-08-07 2017-01-31 Pure Storage, Inc. Failure mapping in a storage array
US10983859B2 (en) 2014-08-07 2021-04-20 Pure Storage, Inc. Adjustable error correction based on memory health in a storage unit
US9766972B2 (en) 2014-08-07 2017-09-19 Pure Storage, Inc. Masking defective bits in a storage array
US10079711B1 (en) 2014-08-20 2018-09-18 Pure Storage, Inc. Virtual file server with preserved MAC address
US9923970B2 (en) * 2014-08-22 2018-03-20 Nexenta Systems, Inc. Multicast collaborative erasure encoding and distributed parity protection
US20160219120A1 (en) * 2015-01-23 2016-07-28 Netapp, Inc. Methods for providing a staging area for objects prior to erasure coding and devices thereof
US9665428B2 (en) * 2015-02-05 2017-05-30 Netapp, Inc. Distributing erasure-coded fragments in a geo-distributed storage system
US9948615B1 (en) 2015-03-16 2018-04-17 Pure Storage, Inc. Increased storage unit encryption based on loss of trust
US11294893B2 (en) 2015-03-20 2022-04-05 Pure Storage, Inc. Aggregation of queries
US9940234B2 (en) 2015-03-26 2018-04-10 Pure Storage, Inc. Aggressive data deduplication using lazy garbage collection
US10082985B2 (en) 2015-03-27 2018-09-25 Pure Storage, Inc. Data striping across storage nodes that are assigned to multiple logical arrays
US10178169B2 (en) 2015-04-09 2019-01-08 Pure Storage, Inc. Point to point based backend communication layer for storage processing
US9672125B2 (en) 2015-04-10 2017-06-06 Pure Storage, Inc. Ability to partition an array into two or more logical arrays with independently running software
US12379854B2 (en) 2015-04-10 2025-08-05 Pure Storage, Inc. Two or more logical arrays having zoned drives
US10140149B1 (en) 2015-05-19 2018-11-27 Pure Storage, Inc. Transactional commits with hardware assists in remote memory
US9817576B2 (en) 2015-05-27 2017-11-14 Pure Storage, Inc. Parallel update to NVRAM
US10846275B2 (en) 2015-06-26 2020-11-24 Pure Storage, Inc. Key management in a storage device
US9734028B2 (en) * 2015-06-29 2017-08-15 International Business Machines Corporation Reverse resynchronization by a secondary data source when a data destination has more recent data
US10983732B2 (en) 2015-07-13 2021-04-20 Pure Storage, Inc. Method and system for accessing a file
US11232079B2 (en) * 2015-07-16 2022-01-25 Pure Storage, Inc. Efficient distribution of large directories
US9898470B2 (en) * 2015-08-05 2018-02-20 Bank Of America Corporation Transferring archived data
WO2017039580A1 (en) 2015-08-28 2017-03-09 Hewlett Packard Enterprise Development Lp Collision handling during an asynchronous replication
US10108355B2 (en) 2015-09-01 2018-10-23 Pure Storage, Inc. Erase block state detection
US11341136B2 (en) 2015-09-04 2022-05-24 Pure Storage, Inc. Dynamically resizable structures for approximate membership queries
US11269884B2 (en) 2015-09-04 2022-03-08 Pure Storage, Inc. Dynamically resizable structures for approximate membership queries
WO2017042978A1 (ja) * 2015-09-11 2017-03-16 株式会社日立製作所 計算機システム、ストレージ装置、及びデータの管理方法
US10762069B2 (en) 2015-09-30 2020-09-01 Pure Storage, Inc. Mechanism for a system where data and metadata are located closely together
US10853266B2 (en) 2015-09-30 2020-12-01 Pure Storage, Inc. Hardware assisted data lookup methods
US9768953B2 (en) 2015-09-30 2017-09-19 Pure Storage, Inc. Resharing of a split secret
US12271359B2 (en) 2015-09-30 2025-04-08 Pure Storage, Inc. Device host operations in a storage system
US10360119B2 (en) 2015-10-06 2019-07-23 Netapp, Inc. Data recovery in a distributed storage system
US9843453B2 (en) 2015-10-23 2017-12-12 Pure Storage, Inc. Authorizing I/O commands with I/O tokens
US10007457B2 (en) 2015-12-22 2018-06-26 Pure Storage, Inc. Distributed transactions with token-associated execution
JP6671708B2 (ja) * 2016-02-09 2020-03-25 株式会社日立製作所 バックアップリストアシステム及びバックアップリストア方法
US10931402B2 (en) 2016-03-15 2021-02-23 Cloud Storage, Inc. Distributed storage system data management and security
US9916478B2 (en) * 2016-03-24 2018-03-13 Netapp, Inc. Data protection enhancement using free space
US10133503B1 (en) 2016-05-02 2018-11-20 Pure Storage, Inc. Selecting a deduplication process based on a difference between performance metrics
US10261690B1 (en) 2016-05-03 2019-04-16 Pure Storage, Inc. Systems and methods for operating a storage system
US10552038B2 (en) * 2016-05-13 2020-02-04 International Business Machines Corporation Object storage architecture based on file_heat
US11231858B2 (en) 2016-05-19 2022-01-25 Pure Storage, Inc. Dynamically configuring a storage system to facilitate independent scaling of resources
MX2018014879A (es) * 2016-06-01 2019-10-15 Archive360 Inc Metodos y sistemas para archivar y recuperar datos.
US12235743B2 (en) 2016-06-03 2025-02-25 Pure Storage, Inc. Efficient partitioning for storage system resiliency groups
US10691567B2 (en) 2016-06-03 2020-06-23 Pure Storage, Inc. Dynamically forming a failure domain in a storage system that includes a plurality of blades
US10547681B2 (en) 2016-06-30 2020-01-28 Purdue Research Foundation Functional caching in erasure coded storage
US11861188B2 (en) 2016-07-19 2024-01-02 Pure Storage, Inc. System having modular accelerators
US11706895B2 (en) 2016-07-19 2023-07-18 Pure Storage, Inc. Independent scaling of compute resources and storage resources in a storage system
US12210476B2 (en) 2016-07-19 2025-01-28 Pure Storage, Inc. Disaggregated compute resources and storage resources in a storage system
US10768819B2 (en) 2016-07-22 2020-09-08 Pure Storage, Inc. Hardware support for non-disruptive upgrades
US12487884B1 (en) 2017-10-31 2025-12-02 Pure Storage, Inc. Writing parity data to a targeted wordline
US11449232B1 (en) 2016-07-22 2022-09-20 Pure Storage, Inc. Optimal scheduling of flash operations
US9672905B1 (en) * 2016-07-22 2017-06-06 Pure Storage, Inc. Optimize data protection layouts based on distributed flash wear leveling
US11080155B2 (en) 2016-07-24 2021-08-03 Pure Storage, Inc. Identifying error types among flash memory
US11604690B2 (en) 2016-07-24 2023-03-14 Pure Storage, Inc. Online failure span determination
US10216420B1 (en) 2016-07-24 2019-02-26 Pure Storage, Inc. Calibration of flash channels in SSD
US10203903B2 (en) 2016-07-26 2019-02-12 Pure Storage, Inc. Geometry based, space aware shelf/writegroup evacuation
US11734169B2 (en) 2016-07-26 2023-08-22 Pure Storage, Inc. Optimizing spool and memory space management
US11797212B2 (en) 2016-07-26 2023-10-24 Pure Storage, Inc. Data migration for zoned drives
US11886334B2 (en) 2016-07-26 2024-01-30 Pure Storage, Inc. Optimizing spool and memory space management
US10366004B2 (en) 2016-07-26 2019-07-30 Pure Storage, Inc. Storage system with elective garbage collection to reduce flash contention
WO2018022931A1 (en) * 2016-07-28 2018-02-01 Caringo, Inc. Multi-part upload
AT518910B1 (de) * 2016-08-04 2018-10-15 Ait Austrian Inst Tech Gmbh Verfahren zur Prüfung der Verfügbarkeit und Integrität eines verteilt gespeicherten Datenobjekts
US10191809B2 (en) 2016-08-17 2019-01-29 International Business Machines Corporation Converting a data chunk into a ring algebraic structure for fast erasure coding
CN106407040B (zh) * 2016-09-05 2019-05-24 华为技术有限公司 一种远程数据复制方法及系统
US11422719B2 (en) 2016-09-15 2022-08-23 Pure Storage, Inc. Distributed file deletion and truncation
US9747039B1 (en) 2016-10-04 2017-08-29 Pure Storage, Inc. Reservations over multiple paths on NVMe over fabrics
US20180095788A1 (en) 2016-10-04 2018-04-05 Pure Storage, Inc. Scheduling operations for a storage device
US12039165B2 (en) 2016-10-04 2024-07-16 Pure Storage, Inc. Utilizing allocation shares to improve parallelism in a zoned drive storage system
US10613974B2 (en) 2016-10-04 2020-04-07 Pure Storage, Inc. Peer-to-peer non-volatile random-access memory
US10756816B1 (en) 2016-10-04 2020-08-25 Pure Storage, Inc. Optimized fibre channel and non-volatile memory express access
US10481798B2 (en) 2016-10-28 2019-11-19 Pure Storage, Inc. Efficient flash management for multiple controllers
US10359942B2 (en) 2016-10-31 2019-07-23 Pure Storage, Inc. Deduplication aware scalable content placement
CN106685426B (zh) * 2016-11-28 2021-02-09 北京航天自动控制研究所 一种目标信息的编码方法
US10268538B2 (en) 2016-11-28 2019-04-23 Alibaba Group Holding Limited Efficient and enhanced distributed storage clusters
US11269888B1 (en) * 2016-11-28 2022-03-08 Amazon Technologies, Inc. Archival data storage for structured data
US10740288B2 (en) * 2016-12-02 2020-08-11 International Business Machines Corporation Accessing objects in an erasure code supported object storage environment
US11550481B2 (en) 2016-12-19 2023-01-10 Pure Storage, Inc. Efficiently writing data in a zoned drive storage system
US11307998B2 (en) 2017-01-09 2022-04-19 Pure Storage, Inc. Storage efficiency of encrypted host system data
US9747158B1 (en) 2017-01-13 2017-08-29 Pure Storage, Inc. Intelligent refresh of 3D NAND
US11955187B2 (en) 2017-01-13 2024-04-09 Pure Storage, Inc. Refresh of differing capacity NAND
US10255134B2 (en) * 2017-01-20 2019-04-09 Samsung Electronics Co., Ltd. Control plane method and apparatus for providing erasure code protection across multiple storage devices
US10979223B2 (en) 2017-01-31 2021-04-13 Pure Storage, Inc. Separate encryption for a solid-state drive
CN108628539B (zh) 2017-03-17 2021-03-26 杭州海康威视数字技术股份有限公司 数据存储、分散、重构、回收方法、装置及数据处理系统
US10795760B2 (en) * 2017-03-20 2020-10-06 Samsung Electronics Co., Ltd. Key value SSD
US11275762B2 (en) 2017-03-20 2022-03-15 Samsung Electronics Co., Ltd. System and method for hybrid data reliability for object storage devices
US10530752B2 (en) * 2017-03-28 2020-01-07 Amazon Technologies, Inc. Efficient device provision
US11356445B2 (en) 2017-03-28 2022-06-07 Amazon Technologies, Inc. Data access interface for clustered devices
US10621055B2 (en) 2017-03-28 2020-04-14 Amazon Technologies, Inc. Adaptive data recovery for clustered data devices
US10528488B1 (en) 2017-03-30 2020-01-07 Pure Storage, Inc. Efficient name coding
US11016667B1 (en) 2017-04-05 2021-05-25 Pure Storage, Inc. Efficient mapping for LUNs in storage memory with holes in address space
US10141050B1 (en) 2017-04-27 2018-11-27 Pure Storage, Inc. Page writes for triple level cell flash memory
US10516645B1 (en) 2017-04-27 2019-12-24 Pure Storage, Inc. Address resolution broadcasting in a networked device
US10944671B2 (en) 2017-04-27 2021-03-09 Pure Storage, Inc. Efficient data forwarding in a networked device
US10275180B2 (en) * 2017-05-08 2019-04-30 Samsung Electronics Co., Ltd. Ethernet SSD system including storage offload engine (SoE) controller and ethernet switch
US10579284B2 (en) 2017-06-02 2020-03-03 International Business Machines Corporation Filesystem enhancements for unified file and object access in an object storage cloud
US11467913B1 (en) 2017-06-07 2022-10-11 Pure Storage, Inc. Snapshots with crash consistency in a storage system
WO2018226227A1 (en) 2017-06-08 2018-12-13 Hitachi Data Systems Corporation Fast recall for geographically distributed object data
US11947814B2 (en) 2017-06-11 2024-04-02 Pure Storage, Inc. Optimizing resiliency group formation stability
US11782625B2 (en) 2017-06-11 2023-10-10 Pure Storage, Inc. Heterogeneity supportive resiliency groups
US11138103B1 (en) 2017-06-11 2021-10-05 Pure Storage, Inc. Resiliency groups
US10713117B2 (en) * 2017-06-15 2020-07-14 Hitachi, Ltd. Storage system and method for controlling storage system
US10310943B2 (en) 2017-06-16 2019-06-04 Microsoft Technology Licensing, Llc Distributed data object management system
US11003532B2 (en) 2017-06-16 2021-05-11 Microsoft Technology Licensing, Llc Distributed data object management system operations
US10425473B1 (en) 2017-07-03 2019-09-24 Pure Storage, Inc. Stateful connection reset in a storage cluster with a stateless load balancer
JP6733620B2 (ja) * 2017-07-12 2020-08-05 京セラドキュメントソリューションズ株式会社 画像読取装置、及び画像読取方法
US10761743B1 (en) 2017-07-17 2020-09-01 EMC IP Holding Company LLC Establishing data reliability groups within a geographically distributed data storage environment
US10402266B1 (en) 2017-07-31 2019-09-03 Pure Storage, Inc. Redundant array of independent disks in a direct-mapped flash storage system
US10831935B2 (en) 2017-08-31 2020-11-10 Pure Storage, Inc. Encryption management with host-side data reduction
US10210926B1 (en) 2017-09-15 2019-02-19 Pure Storage, Inc. Tracking of optimum read voltage thresholds in nand flash devices
US10877827B2 (en) 2017-09-15 2020-12-29 Pure Storage, Inc. Read voltage optimization
US10789211B1 (en) 2017-10-04 2020-09-29 Pure Storage, Inc. Feature-based deduplication
US10880040B1 (en) 2017-10-23 2020-12-29 EMC IP Holding Company LLC Scale-out distributed erasure coding
US12032848B2 (en) 2021-06-21 2024-07-09 Pure Storage, Inc. Intelligent block allocation in a heterogeneous storage system
US10884919B2 (en) 2017-10-31 2021-01-05 Pure Storage, Inc. Memory management in a storage system
US11520514B2 (en) 2018-09-06 2022-12-06 Pure Storage, Inc. Optimized relocation of data based on data characteristics
US12067274B2 (en) 2018-09-06 2024-08-20 Pure Storage, Inc. Writing segments and erase blocks based on ordering
US10515701B1 (en) 2017-10-31 2019-12-24 Pure Storage, Inc. Overlapping raid groups
US10545687B1 (en) 2017-10-31 2020-01-28 Pure Storage, Inc. Data rebuild when changing erase block sizes during drive replacement
US11024390B1 (en) 2017-10-31 2021-06-01 Pure Storage, Inc. Overlapping RAID groups
US10496330B1 (en) 2017-10-31 2019-12-03 Pure Storage, Inc. Using flash storage devices with different sized erase blocks
US11354058B2 (en) 2018-09-06 2022-06-07 Pure Storage, Inc. Local relocation of data stored at a storage device of a storage system
US10700711B1 (en) * 2017-11-03 2020-06-30 Caringo Inc. Multi-part upload and editing of erasure-coded objects
US10860475B1 (en) 2017-11-17 2020-12-08 Pure Storage, Inc. Hybrid flash translation layer
US10990566B1 (en) 2017-11-20 2021-04-27 Pure Storage, Inc. Persistent file locks in a storage system
US10929053B2 (en) 2017-12-08 2021-02-23 Pure Storage, Inc. Safe destructive actions on drives
US10719265B1 (en) 2017-12-08 2020-07-21 Pure Storage, Inc. Centralized, quorum-aware handling of device reservation requests in a storage system
US10929031B2 (en) 2017-12-21 2021-02-23 Pure Storage, Inc. Maximizing data reduction in a partially encrypted volume
US10382554B1 (en) 2018-01-04 2019-08-13 Emc Corporation Handling deletes with distributed erasure coding
US11861025B1 (en) 2018-01-08 2024-01-02 Rankin Labs, Llc System and method for receiving and processing a signal within a TCP/IP protocol stack
CN110018783B (zh) 2018-01-09 2022-12-20 阿里巴巴集团控股有限公司 一种数据存储方法、装置及系统
KR102663422B1 (ko) * 2018-01-19 2024-05-07 삼성전자주식회사 객체 스토리지 장치들에 대한 하이브리드 데이터 신뢰성을 위한 시스템 및 방법
TWI750425B (zh) * 2018-01-19 2021-12-21 南韓商三星電子股份有限公司 資料儲存系統和用於寫入鍵值對的物件的方法
US10467527B1 (en) 2018-01-31 2019-11-05 Pure Storage, Inc. Method and apparatus for artificial intelligence acceleration
US10976948B1 (en) 2018-01-31 2021-04-13 Pure Storage, Inc. Cluster expansion mechanism
US10733053B1 (en) 2018-01-31 2020-08-04 Pure Storage, Inc. Disaster recovery for high-bandwidth distributed archives
US11036596B1 (en) 2018-02-18 2021-06-15 Pure Storage, Inc. System for delaying acknowledgements on open NAND locations until durability has been confirmed
US11494109B1 (en) 2018-02-22 2022-11-08 Pure Storage, Inc. Erase block trimming for heterogenous flash memory storage devices
US11847331B2 (en) 2019-12-12 2023-12-19 Pure Storage, Inc. Budgeting open blocks of a storage unit based on power loss prevention
US12393340B2 (en) 2019-01-16 2025-08-19 Pure Storage, Inc. Latency reduction of flash-based devices using programming interrupts
US11416144B2 (en) 2019-12-12 2022-08-16 Pure Storage, Inc. Dynamic use of segment or zone power loss protection in a flash device
US10901845B2 (en) * 2018-04-16 2021-01-26 Microsoft Technology Licensing, Llc Erasure coding for a single-image memory
US12001688B2 (en) 2019-04-29 2024-06-04 Pure Storage, Inc. Utilizing data views to optimize secure data access in a storage system
US11995336B2 (en) 2018-04-25 2024-05-28 Pure Storage, Inc. Bucket views
US12175124B2 (en) 2018-04-25 2024-12-24 Pure Storage, Inc. Enhanced data access using composite data views
US10853146B1 (en) 2018-04-27 2020-12-01 Pure Storage, Inc. Efficient data forwarding in a networked device
US10931450B1 (en) 2018-04-27 2021-02-23 Pure Storage, Inc. Distributed, lock-free 2-phase commit of secret shares using multiple stateless controllers
US12079494B2 (en) 2018-04-27 2024-09-03 Pure Storage, Inc. Optimizing storage system upgrades to preserve resources
US11385792B2 (en) 2018-04-27 2022-07-12 Pure Storage, Inc. High availability controller pair transitioning
US10579297B2 (en) 2018-04-27 2020-03-03 EMC IP Holding Company LLC Scaling-in for geographically diverse storage
US11436023B2 (en) 2018-05-31 2022-09-06 Pure Storage, Inc. Mechanism for updating host file system and flash translation layer based on underlying NAND technology
US11023130B2 (en) 2018-06-15 2021-06-01 EMC IP Holding Company LLC Deleting data in a geographically diverse storage construct
US10936196B2 (en) 2018-06-15 2021-03-02 EMC IP Holding Company LLC Data convolution for geographically diverse storage
US12210904B2 (en) 2018-06-29 2025-01-28 International Business Machines Corporation Hybridized storage optimization for genomic workloads
US10572173B2 (en) * 2018-07-10 2020-02-25 EMC IP Holding Company LLC Data management in an elastic cloud storage environment
US11438279B2 (en) 2018-07-23 2022-09-06 Pure Storage, Inc. Non-disruptive conversion of a clustered service from single-chassis to multi-chassis
US11689543B2 (en) 2018-08-10 2023-06-27 Rankin Labs, Llc System and method for detecting transmission of a covert payload of data
US11652732B2 (en) 2018-08-21 2023-05-16 Rankin Labs, Llc System and method for scattering network traffic across a number of disparate hosts
US11500570B2 (en) 2018-09-06 2022-11-15 Pure Storage, Inc. Efficient relocation of data utilizing different programming modes
US11868309B2 (en) 2018-09-06 2024-01-09 Pure Storage, Inc. Queue management for data relocation
JP7312251B2 (ja) * 2018-10-15 2023-07-20 ネットアップ，インコーポレイテッド 様々なデータ冗長性スキームを備えた、システムにおける利用可能なストレージ空間を改善すること
US20200117362A1 (en) * 2018-10-15 2020-04-16 Netapp, Inc. Erasure coding content driven distribution of data blocks
US10454498B1 (en) 2018-10-18 2019-10-22 Pure Storage, Inc. Fully pipelined hardware engine design for fast and efficient inline lossless data compression
US10976947B2 (en) 2018-10-26 2021-04-13 Pure Storage, Inc. Dynamically selecting segment heights in a heterogeneous RAID group
US11436203B2 (en) 2018-11-02 2022-09-06 EMC IP Holding Company LLC Scaling out geographically diverse storage
US10901635B2 (en) * 2018-12-04 2021-01-26 EMC IP Holding Company LLC Mapped redundant array of independent nodes for data storage with high performance using logical columns of the nodes with different widths and different positioning patterns
US11989320B2 (en) 2018-12-19 2024-05-21 Rankin Labs, Llc Hidden electronic file system within non-hidden electronic file system
US10931777B2 (en) 2018-12-20 2021-02-23 EMC IP Holding Company LLC Network efficient geographically diverse data storage system employing degraded chunks
US11119683B2 (en) 2018-12-20 2021-09-14 EMC IP Holding Company LLC Logical compaction of a degraded chunk in a geographically diverse data storage system
US10892782B2 (en) 2018-12-21 2021-01-12 EMC IP Holding Company LLC Flexible system and method for combining erasure-coded protection sets
JP6947717B2 (ja) 2018-12-27 2021-10-13 株式会社日立製作所 ストレージシステム
US11023331B2 (en) 2019-01-04 2021-06-01 EMC IP Holding Company LLC Fast recovery of data in a geographically distributed storage environment
US11586595B1 (en) * 2019-01-11 2023-02-21 Amazon Technologies, Inc. Space-efficient techniques for generating unique instances of data objects
US11544387B2 (en) * 2019-01-16 2023-01-03 International Business Machines Corporation Hash protection within an object storage library
US11334456B1 (en) * 2019-01-18 2022-05-17 EMC IP Holding Company LLC Space efficient data protection
US10942827B2 (en) 2019-01-22 2021-03-09 EMC IP Holding Company LLC Replication of data in a geographically distributed storage environment
US11194473B1 (en) 2019-01-23 2021-12-07 Pure Storage, Inc. Programming frequently read data to low latency portions of a solid-state storage array
US10979312B2 (en) 2019-01-29 2021-04-13 Dell Products L.P. System and method to assign, monitor, and validate solution infrastructure deployment prerequisites in a customer data center
US10972343B2 (en) 2019-01-29 2021-04-06 Dell Products L.P. System and method for device configuration update
US10911307B2 (en) 2019-01-29 2021-02-02 Dell Products L.P. System and method for out of the box solution-level configuration and diagnostic logging and reporting
US10747522B1 (en) 2019-01-29 2020-08-18 EMC IP Holding Company LLC Method and system for non-disruptive host repurposing
US11182247B2 (en) 2019-01-29 2021-11-23 Cloud Storage, Inc. Encoding and storage node repairing method for minimum storage regenerating codes for distributed storage systems
US10866766B2 (en) 2019-01-29 2020-12-15 EMC IP Holding Company LLC Affinity sensitive data convolution for data storage systems
US10936239B2 (en) 2019-01-29 2021-03-02 EMC IP Holding Company LLC Cluster contraction of a mapped redundant array of independent nodes
US10901641B2 (en) 2019-01-29 2021-01-26 Dell Products L.P. Method and system for inline deduplication
US11442642B2 (en) * 2019-01-29 2022-09-13 Dell Products L.P. Method and system for inline deduplication using erasure coding to minimize read and write operations
US10740023B1 (en) 2019-01-29 2020-08-11 Dell Products L.P. System and method for dynamic application access-based mapping
US20200241781A1 (en) 2019-01-29 2020-07-30 Dell Products L.P. Method and system for inline deduplication using erasure coding
US10846003B2 (en) 2019-01-29 2020-11-24 EMC IP Holding Company LLC Doubly mapped redundant array of independent nodes for data storage
US10942825B2 (en) 2019-01-29 2021-03-09 EMC IP Holding Company LLC Mitigating real node failure in a mapped redundant array of independent nodes
US10764135B2 (en) 2019-01-29 2020-09-01 Dell Products L.P. Method and system for solution integration labeling
US11334254B2 (en) 2019-03-29 2022-05-17 Pure Storage, Inc. Reliability based flash page sizing
US10944826B2 (en) 2019-04-03 2021-03-09 EMC IP Holding Company LLC Selective instantiation of a storage service for a mapped redundant array of independent nodes
US12373340B2 (en) 2019-04-03 2025-07-29 Pure Storage, Inc. Intelligent subsegment formation in a heterogeneous storage system
US11775189B2 (en) 2019-04-03 2023-10-03 Pure Storage, Inc. Segment level heterogeneity
US11029865B2 (en) 2019-04-03 2021-06-08 EMC IP Holding Company LLC Affinity sensitive storage of data corresponding to a mapped redundant array of independent nodes
US12087382B2 (en) 2019-04-11 2024-09-10 Pure Storage, Inc. Adaptive threshold for bad flash memory blocks
US11099986B2 (en) 2019-04-12 2021-08-24 Pure Storage, Inc. Efficient transfer of memory contents
US10936218B2 (en) * 2019-04-18 2021-03-02 EMC IP Holding Company LLC Facilitating an out-of-order transmission of segments of multi-segment data portions for distributed storage devices
US11119686B2 (en) 2019-04-30 2021-09-14 EMC IP Holding Company LLC Preservation of data during scaling of a geographically diverse data storage system
US11113146B2 (en) 2019-04-30 2021-09-07 EMC IP Holding Company LLC Chunk segment recovery via hierarchical erasure coding in a geographically diverse data storage system
US11121727B2 (en) 2019-04-30 2021-09-14 EMC IP Holding Company LLC Adaptive data storing for data storage systems employing erasure coding
US11748004B2 (en) 2019-05-03 2023-09-05 EMC IP Holding Company LLC Data replication using active and passive data storage modes
US11729184B2 (en) 2019-05-28 2023-08-15 Rankin Labs, Llc Detecting covertly stored payloads of data within a network
US11487665B2 (en) 2019-06-05 2022-11-01 Pure Storage, Inc. Tiered caching of data in a storage system
US11714572B2 (en) 2019-06-19 2023-08-01 Pure Storage, Inc. Optimized data resiliency in a modular storage system
US11281394B2 (en) 2019-06-24 2022-03-22 Pure Storage, Inc. Replication across partitioning schemes in a distributed storage system
US11209996B2 (en) 2019-07-15 2021-12-28 EMC IP Holding Company LLC Mapped cluster stretching for increasing workload in a data storage system
US11449399B2 (en) 2019-07-30 2022-09-20 EMC IP Holding Company LLC Mitigating real node failure of a doubly mapped redundant array of independent nodes
US11023145B2 (en) 2019-07-30 2021-06-01 EMC IP Holding Company LLC Hybrid mapped clusters for data storage
US11609820B2 (en) 2019-07-31 2023-03-21 Dell Products L.P. Method and system for redundant distribution and reconstruction of storage metadata
US11328071B2 (en) 2019-07-31 2022-05-10 Dell Products L.P. Method and system for identifying actor of a fraudulent action during legal hold and litigation
US10963345B2 (en) 2019-07-31 2021-03-30 Dell Products L.P. Method and system for a proactive health check and reconstruction of data
US11372730B2 (en) 2019-07-31 2022-06-28 Dell Products L.P. Method and system for offloading a continuous health-check and reconstruction of data in a non-accelerator pool
US11775193B2 (en) 2019-08-01 2023-10-03 Dell Products L.P. System and method for indirect data classification in a storage system operations
CN111399755A (zh) * 2019-09-10 2020-07-10 杭州海康威视系统技术有限公司 数据存储管理的方法和装置
US11228322B2 (en) 2019-09-13 2022-01-18 EMC IP Holding Company LLC Rebalancing in a geographically diverse storage system employing erasure coding
US11494108B2 (en) * 2019-09-23 2022-11-08 Amazon Technologies, Inc. Cross-zone replicated block storage devices
US11537725B2 (en) * 2019-09-23 2022-12-27 Amazon Technologies, Inc. Encrypted cross-zone replication for cross-zone replicated block storage devices
US11449248B2 (en) 2019-09-26 2022-09-20 EMC IP Holding Company LLC Mapped redundant array of independent data storage regions
US11893126B2 (en) 2019-10-14 2024-02-06 Pure Storage, Inc. Data deletion for a multi-tenant environment
US12475041B2 (en) 2019-10-15 2025-11-18 Pure Storage, Inc. Efficient data storage by grouping similar data within a zone
US11119690B2 (en) 2019-10-31 2021-09-14 EMC IP Holding Company LLC Consolidation of protection sets in a geographically diverse data storage environment
US11288139B2 (en) 2019-10-31 2022-03-29 EMC IP Holding Company LLC Two-step recovery employing erasure coding in a geographically diverse data storage system
US11435910B2 (en) 2019-10-31 2022-09-06 EMC IP Holding Company LLC Heterogeneous mapped redundant array of independent nodes for data storage
US11435957B2 (en) 2019-11-27 2022-09-06 EMC IP Holding Company LLC Selective instantiation of a storage service for a doubly mapped redundant array of independent nodes
US11157179B2 (en) 2019-12-03 2021-10-26 Pure Storage, Inc. Dynamic allocation of blocks of a storage device based on power loss protection
US11704192B2 (en) 2019-12-12 2023-07-18 Pure Storage, Inc. Budgeting open blocks based on power loss protection
US12001684B2 (en) 2019-12-12 2024-06-04 Pure Storage, Inc. Optimizing dynamic power loss protection adjustment in a storage system
CN111212111B (zh) * 2019-12-17 2022-05-13 腾讯科技（深圳）有限公司 对象存储服务管理方法及电子设备
WO2021127320A1 (en) * 2019-12-18 2021-06-24 John Rankin Distribution of data over a network with interconnected rings
US11514181B2 (en) * 2020-02-12 2022-11-29 Netapp, Inc. Bin syncing technique for multiple data protection schemes
US11144220B2 (en) 2019-12-24 2021-10-12 EMC IP Holding Company LLC Affinity sensitive storage of data corresponding to a doubly mapped redundant array of independent nodes
US11231860B2 (en) 2020-01-17 2022-01-25 EMC IP Holding Company LLC Doubly mapped redundant array of independent nodes for data storage with high performance
US11210002B2 (en) 2020-01-29 2021-12-28 Samsung Electronics Co., Ltd. Offloaded device-driven erasure coding
US11188432B2 (en) 2020-02-28 2021-11-30 Pure Storage, Inc. Data resiliency by partially deallocating data blocks of a storage device
US11281535B2 (en) 2020-03-06 2022-03-22 Dell Products L.P. Method and system for performing a checkpoint zone operation for a spare persistent storage
US11175842B2 (en) 2020-03-06 2021-11-16 Dell Products L.P. Method and system for performing data deduplication in a data pipeline
US11416357B2 (en) 2020-03-06 2022-08-16 Dell Products L.P. Method and system for managing a spare fault domain in a multi-fault domain data cluster
US11119858B1 (en) 2020-03-06 2021-09-14 Dell Products L.P. Method and system for performing a proactive copy operation for a spare persistent storage
US11301327B2 (en) 2020-03-06 2022-04-12 Dell Products L.P. Method and system for managing a spare persistent storage device and a spare node in a multi-node data cluster
US11507308B2 (en) 2020-03-30 2022-11-22 EMC IP Holding Company LLC Disk access event control for mapped nodes supported by a real cluster storage system
US11507297B2 (en) 2020-04-15 2022-11-22 Pure Storage, Inc. Efficient management of optimal read levels for flash storage systems
US11256587B2 (en) 2020-04-17 2022-02-22 Pure Storage, Inc. Intelligent access to a storage device
US11416338B2 (en) 2020-04-24 2022-08-16 Pure Storage, Inc. Resiliency scheme to enhance storage performance
US11474986B2 (en) 2020-04-24 2022-10-18 Pure Storage, Inc. Utilizing machine learning to streamline telemetry processing of storage media
US20210334241A1 (en) * 2020-04-24 2021-10-28 Netapp, Inc. Non-disrputive transitioning between replication schemes
US12056365B2 (en) 2020-04-24 2024-08-06 Pure Storage, Inc. Resiliency for a storage system
US11418326B2 (en) 2020-05-21 2022-08-16 Dell Products L.P. Method and system for performing secure data transactions in a data cluster
US11288229B2 (en) 2020-05-29 2022-03-29 EMC IP Holding Company LLC Verifiable intra-cluster migration for a chunk storage system
US11768763B2 (en) 2020-07-08 2023-09-26 Pure Storage, Inc. Flash secure erase
US11681448B2 (en) 2020-09-08 2023-06-20 Pure Storage, Inc. Multiple device IDs in a multi-fabric module storage system
US11513974B2 (en) 2020-09-08 2022-11-29 Pure Storage, Inc. Using nonce to control erasure of data blocks of a multi-controller storage system
US12153818B2 (en) 2020-09-24 2024-11-26 Pure Storage, Inc. Bucket versioning snapshots
CN114327239B (zh) * 2020-09-27 2024-08-20 伊姆西Ip控股有限责任公司 存储和访问数据的方法、电子设备和计算机程序产品
US11693983B2 (en) 2020-10-28 2023-07-04 EMC IP Holding Company LLC Data protection via commutative erasure coding in a geographically diverse data storage system
US11561856B2 (en) 2020-12-10 2023-01-24 Nutanix, Inc. Erasure coding of replicated data blocks
WO2022128080A1 (en) * 2020-12-16 2022-06-23 Huawei Technologies Co., Ltd. Storing data in computer storage
US11487455B2 (en) 2020-12-17 2022-11-01 Pure Storage, Inc. Dynamic block allocation to optimize storage system performance
US11847324B2 (en) 2020-12-31 2023-12-19 Pure Storage, Inc. Optimizing resiliency groups for data regions of a storage system
US12093545B2 (en) 2020-12-31 2024-09-17 Pure Storage, Inc. Storage system with selectable write modes
US12067282B2 (en) 2020-12-31 2024-08-20 Pure Storage, Inc. Write path selection
US12229437B2 (en) 2020-12-31 2025-02-18 Pure Storage, Inc. Dynamic buffer for storage system
US11614880B2 (en) 2020-12-31 2023-03-28 Pure Storage, Inc. Storage system with selectable write paths
US11847141B2 (en) 2021-01-19 2023-12-19 EMC IP Holding Company LLC Mapped redundant array of independent nodes employing mapped reliability groups for data storage
US11625174B2 (en) 2021-01-20 2023-04-11 EMC IP Holding Company LLC Parity allocation for a virtual redundant array of independent disks
US12061814B2 (en) 2021-01-25 2024-08-13 Pure Storage, Inc. Using data similarity to select segments for garbage collection
US11630593B2 (en) 2021-03-12 2023-04-18 Pure Storage, Inc. Inline flash memory qualification in a storage system
US12099742B2 (en) 2021-03-15 2024-09-24 Pure Storage, Inc. Utilizing programming page size granularity to optimize data segment storage in a storage system
US11507597B2 (en) 2021-03-31 2022-11-22 Pure Storage, Inc. Data replication to meet a recovery point objective
JP7355778B2 (ja) * 2021-04-27 2023-10-03 株式会社日立製作所 ストレージシステム、ストレージノード仮想計算機復旧方法、及び復旧プログラム
US11449234B1 (en) 2021-05-28 2022-09-20 EMC IP Holding Company LLC Efficient data access operations via a mapping layer instance for a doubly mapped redundant array of independent nodes
US11354191B1 (en) 2021-05-28 2022-06-07 EMC IP Holding Company LLC Erasure coding in a large geographically diverse data storage system
CN115639947A (zh) * 2021-07-19 2023-01-24 华为技术有限公司 数据写入方法、数据读取方法、装置、设备、系统及介质
US12498869B2 (en) 2021-08-10 2025-12-16 Samsung Electronics Co., Ltd. Systems, methods, and apparatus for hierarchical aggregation for computational storage
US12074962B2 (en) 2021-08-10 2024-08-27 Samsung Electronics Co., Ltd. Systems, methods, and apparatus for dividing and encrypting data
US12413243B2 (en) 2021-08-10 2025-09-09 Samsung Electronics Co., Ltd. Systems, methods, and apparatus for dividing and compressing data
US11832410B2 (en) 2021-09-14 2023-11-28 Pure Storage, Inc. Mechanical energy absorbing bracket apparatus
US11726779B2 (en) * 2021-11-03 2023-08-15 Sap Se Code simplification system
US11994723B2 (en) 2021-12-30 2024-05-28 Pure Storage, Inc. Ribbon cable alignment apparatus
US12164477B2 (en) 2022-01-24 2024-12-10 Cohesity, Inc. Container-based erasure coding
US12439544B2 (en) 2022-04-20 2025-10-07 Pure Storage, Inc. Retractable pivoting trap door
US12314163B2 (en) 2022-04-21 2025-05-27 Pure Storage, Inc. Die-aware scheduler
TR2022013118A2 (tr) * 2022-08-19 2022-10-21 Oezyegin Ueniversitesi Modüler yazilim mi̇mari̇si̇ni̇n geri̇ kazanimini sağlamak i̇çi̇n bi̇r yöntem
US12141447B2 (en) 2022-10-04 2024-11-12 Scality, S.A. Erasure coding implementation with reduced parity calculation overhead
US12500690B2 (en) * 2022-12-21 2025-12-16 Qualcomm Incorporated Signaling for enabling erasure coding
WO2024182553A1 (en) 2023-02-28 2024-09-06 Pure Storage, Inc. Data storage system with managed flash
US12204788B1 (en) 2023-07-21 2025-01-21 Pure Storage, Inc. Dynamic plane selection in data storage system
US12487920B2 (en) 2024-04-30 2025-12-02 Pure Storage, Inc. Storage system with dynamic data management functions

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050091450A1 (en) * 2003-10-23 2005-04-28 Svend Frolund Methods of reading and writing data
CN1696936A (zh) * 2004-05-14 2005-11-16 微软公司 使用部分复制的Web内容分布式寄放
TW200901025A (en) * 2007-03-08 2009-01-01 Nec Lab America Inc Method and system for a self managing and scalable grid storage
US7552356B1 (en) * 2004-06-30 2009-06-23 Sun Microsystems, Inc. Distributed data storage system for fixed content

Family Cites Families (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5740465A (en) * 1992-04-08 1998-04-14 Hitachi, Ltd. Array disk controller for grouping host commands into a single virtual host command
JP3011020B2 (ja) * 1993-09-27 2000-02-21 株式会社日立製作所 アレイ型デイスクシステム
US5761402A (en) * 1993-03-08 1998-06-02 Hitachi, Ltd. Array type disk system updating redundant data asynchronously with data access
JP3865775B2 (ja) 1995-04-11 2007-01-10 キネテック インコーポレイテッド データ処理システムにおけるデータの識別
US6925547B2 (en) 2000-12-14 2005-08-02 Silicon Graphics, Inc. Remote address translation in a multiprocessor system
US6862692B2 (en) 2001-01-29 2005-03-01 Adaptec, Inc. Dynamic redistribution of parity groups
US6795903B2 (en) 2002-01-17 2004-09-21 Thomas Licensing S.A. System and method for searching for duplicate data
US7577816B2 (en) 2003-08-18 2009-08-18 Cray Inc. Remote translation mechanism for a multinode system
US6922766B2 (en) 2002-09-04 2005-07-26 Cray Inc. Remote translation mechanism for a multi-node system
WO2004046971A1 (en) * 2002-11-14 2004-06-03 Isilon Systems, Inc. Systems and methods for restriping files in a distributed file system
US20040153841A1 (en) 2003-01-16 2004-08-05 Silicon Graphics, Inc. Failure hierarchy in a cluster filesystem
AU2004214014B2 (en) 2003-02-21 2009-10-22 Datacore Software Corporation Additional hash functions in content-based addressing
US7765381B2 (en) 2003-04-04 2010-07-27 Oracle America, Inc. Multi-node system in which home memory subsystem stores global to local address translation information for replicating nodes
US7363462B2 (en) 2003-04-04 2008-04-22 Sun Microsystems, Inc. Performing virtual to global address translation in processing subsystem
US7069392B2 (en) 2003-06-12 2006-06-27 Newisys, Inc. Methods and apparatus for extended packet communications between multiprocessor clusters
US7310703B2 (en) * 2003-10-23 2007-12-18 Hewlett-Packard Development Company, L.P. Methods of reading and writing data
US7266716B2 (en) * 2003-10-23 2007-09-04 Hewlett-Packard Development Company, L.P. Method and recovery of data using erasure coded data from stripe blocks
US7681105B1 (en) 2004-08-09 2010-03-16 Bakbone Software, Inc. Method for lock-free clustered erasure coding and recovery of data across a plurality of data stores in a network
US8843727B2 (en) 2004-09-30 2014-09-23 Intel Corporation Performance enhancement of address translation using translation tables covering large address spaces
US7647329B1 (en) 2005-12-29 2010-01-12 Amazon Technologies, Inc. Keymap service architecture for a distributed storage system
US7734603B1 (en) 2006-01-26 2010-06-08 Netapp, Inc. Content addressable storage array element
BRPI0604440B1 (pt) 2006-10-10 2018-07-10 EMBRAER - Empresa Brasileira de Aeronáutica S.A. Configuração aplicada em sistema para controle da autoridade do leme de uma aeronave
US7917469B2 (en) 2006-11-08 2011-03-29 Hitachi Data Systems Corporation Fast primary cluster recovery
US7590672B2 (en) 2006-12-11 2009-09-15 Bycast Inc. Identification of fixed content objects in a distributed fixed content storage system
KR100905218B1 (ko) 2007-04-09 2009-07-01 삼성전자주식회사 애드혹 네트워크에서 콘텐츠 중복 검출 방법
US7979645B2 (en) 2007-09-14 2011-07-12 Ricoh Company, Limited Multiprocessor system for memory mapping of processing nodes
US20090089537A1 (en) 2007-09-28 2009-04-02 Sun Microsystems, Inc. Apparatus and method for memory address translation across multiple nodes
JP5046863B2 (ja) * 2007-11-01 2012-10-10 株式会社日立製作所 情報処理システム及びデータ管理方法
US7747663B2 (en) 2008-03-05 2010-06-29 Nec Laboratories America, Inc. System and method for content addressable storage
US8255430B2 (en) 2008-08-26 2012-08-28 Caringo, Inc. Shared namespace for storage clusters
US20100235409A1 (en) 2009-03-10 2010-09-16 Global Relay Communications Inc. System and method for managing data stored in a data network
US8069366B1 (en) 2009-04-29 2011-11-29 Netapp, Inc. Global write-log device for managing write logs of nodes of a cluster storage system
US8296515B1 (en) * 2009-05-22 2012-10-23 Nvidia Corporation RAID-6 computation system and method
US8918365B2 (en) 2009-06-19 2014-12-23 Blekko, Inc. Dedicating disks to reading or writing
CN102341777B (zh) 2009-06-26 2014-09-10 株式会社日立制作所 存储系统及其控制方法
AU2010276389B2 (en) 2009-07-20 2016-03-03 Datacore Software Corporation Adaptive power conservation in storage clusters
US8458287B2 (en) 2009-07-31 2013-06-04 Microsoft Corporation Erasure coded storage aggregation in data centers
US8132043B2 (en) 2009-12-17 2012-03-06 Symantec Corporation Multistage system recovery framework
US20110167045A1 (en) 2010-01-05 2011-07-07 Hitachi, Ltd. Storage system and its file management method
US9047351B2 (en) 2010-04-12 2015-06-02 Sandisk Enterprise Ip Llc Cluster of processing nodes with distributed global flash memory using commodity server technology
US8433685B2 (en) 2010-08-18 2013-04-30 Hewlett-Packard Development Company, L.P. Method and system for parity-page distribution among nodes of a multi-node data-storage system
US8473778B2 (en) 2010-09-08 2013-06-25 Microsoft Corporation Erasure coding immutable data
US8504535B1 (en) * 2010-12-20 2013-08-06 Amazon Technologies, Inc. Erasure coding and redundant replication
US20120266044A1 (en) * 2011-04-18 2012-10-18 The Chinese University Of Hong Kong Network-coding-based distributed file system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050091450A1 (en) * 2003-10-23 2005-04-28 Svend Frolund Methods of reading and writing data
CN1696936A (zh) * 2004-05-14 2005-11-16 微软公司 使用部分复制的Web内容分布式寄放
US7552356B1 (en) * 2004-06-30 2009-06-23 Sun Microsystems, Inc. Distributed data storage system for fixed content
TW200901025A (en) * 2007-03-08 2009-01-01 Nec Lab America Inc Method and system for a self managing and scalable grid storage

Also Published As

Publication number Publication date
EP2862085A4 (en) 2016-03-09
JP6280108B2 (ja) 2018-02-14
CA2876638A1 (en) 2013-12-19
US20150019937A1 (en) 2015-01-15
JP2015519674A (ja) 2015-07-09
US10437672B2 (en) 2019-10-08
US20130339818A1 (en) 2013-12-19
AU2013274626A1 (en) 2015-01-22
CN104541251A (zh) 2015-04-22
US9148174B2 (en) 2015-09-29
WO2013188169A1 (en) 2013-12-19
US20150363269A1 (en) 2015-12-17
US20170351575A1 (en) 2017-12-07
EP2862085A1 (en) 2015-04-22
US8799746B2 (en) 2014-08-05
US9916198B2 (en) 2018-03-13

Similar Documents

Publication Publication Date Title
CN104541251B (zh) 在存储群集中的擦除编码和复制
US10649827B2 (en) Two level addressing in storage clusters
US20170308437A1 (en) Parity protection for data chunks in an object storage system
US10725884B2 (en) Object storage system for an unreliable storage medium
CN102598020B (zh) 用于改进的数据去重的装置、系统及方法
US9330137B2 (en) Cloud data backup storage manager
US20130339314A1 (en) Elimination of duplicate objects in storage clusters
CN106547859A (zh) 一种多租户数据存储系统下的数据文件的存储方法及装置
CN103944981A (zh) 一种基于纠删码技术改进的云存储系统及实现方法
WO2015100627A1 (zh) 一种分布式文件存储系统中的数据处理方法及设备
US10700711B1 (en) Multi-part upload and editing of erasure-coded objects
JP6975228B2 (ja) 標準及び非標準分散ストレージ・ネットワーク・データ・アクセスの方法およびコンピュータ可読メモリ
US20190303035A1 (en) Copying garbage collector for geographically distributed data storage environment
JP6671708B2 (ja) バックアップリストアシステム及びバックアップリストア方法
CN113656363B (zh) 一种基于hdfs的数据去重方法、系统、设备和存储介质

Legal Events

Date Code Title Description
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant