CN104054319A - 在弹性云文件系统中减少延迟和成本 - Google Patents

在弹性云文件系统中减少延迟和成本 Download PDF

Info

Publication number
CN104054319A
CN104054319A CN201380005239.1A CN201380005239A CN104054319A CN 104054319 A CN104054319 A CN 104054319A CN 201380005239 A CN201380005239 A CN 201380005239A CN 104054319 A CN104054319 A CN 104054319A
Authority
CN
China
Prior art keywords
file
chunk
chunks
data center
blocks
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201380005239.1A
Other languages
English (en)
Chinese (zh)
Inventor
K·普塔斯瓦米纳加
T·南达戈帕尔
Y·马
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alcatel Lucent SAS
Original Assignee
Alcatel Lucent SAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alcatel Lucent SAS filed Critical Alcatel Lucent SAS
Publication of CN104054319A publication Critical patent/CN104054319A/zh
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/13—File access structures, e.g. distributed indices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/172—Caching, prefetching or hoarding of files
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/17—Details of further file system functions
    • G06F16/176—Support for shared access to files; File sharing support
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems
    • G06F16/1824—Distributed file systems implemented using Network-attached Storage [NAS] architecture
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L1/00—Arrangements for detecting or preventing errors in the information received
    • H04L1/004—Arrangements for detecting or preventing errors in the information received by using forward error control
    • H04L1/0056—Systems characterized by the type of code used
    • H04L1/0057—Block codes

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Information Transfer Between Computers (AREA)
CN201380005239.1A 2012-01-11 2013-01-09 在弹性云文件系统中减少延迟和成本 Pending CN104054319A (zh)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US13/348,243 US8868647B2 (en) 2012-01-11 2012-01-11 Reducing latency and cost in resilient cloud file systems
US13/348,243 2012-01-11
PCT/US2013/020783 WO2013106400A1 (en) 2012-01-11 2013-01-09 Reducing latency and cost in resilient cloud file systems

Publications (1)

Publication Number Publication Date
CN104054319A true CN104054319A (zh) 2014-09-17

Family

ID=47595087

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201380005239.1A Pending CN104054319A (zh) 2012-01-11 2013-01-09 在弹性云文件系统中减少延迟和成本

Country Status (7)

Country Link
US (2) US8868647B2 (enExample)
EP (1) EP2803180A1 (enExample)
JP (1) JP2015509244A (enExample)
KR (1) KR20140102292A (enExample)
CN (1) CN104054319A (enExample)
IN (1) IN2014CN04641A (enExample)
WO (1) WO2013106400A1 (enExample)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107422980A (zh) * 2016-05-24 2017-12-01 上海共联通信信息发展有限公司 物联网数据文件存储系统及其数据文件存储方法
CN107729176A (zh) * 2017-09-14 2018-02-23 北京京东尚科信息技术有限公司 一种配置文件管理系统的容灾方法及容灾系统

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9479587B2 (en) * 2013-01-23 2016-10-25 Nexenta Systems, Inc. Scalable object storage using multicast transport
EP2830285A1 (en) * 2013-07-25 2015-01-28 Thomson Licensing Method for caching of data items in a cloud based data processing system and corresponding device
EP3053055B1 (en) 2013-09-30 2018-07-18 Northeastern University System and method for joint dynamic forwarding and caching in content distribution networks
US9280678B2 (en) * 2013-12-02 2016-03-08 Fortinet, Inc. Secure cloud storage distribution and aggregation
US9367562B2 (en) 2013-12-05 2016-06-14 Google Inc. Distributing data on distributed storage systems
CA2948815A1 (en) * 2014-05-13 2015-11-19 Cloud Crowding Corp. Distributed secure data storage and transmission of streaming media content
US20150331752A1 (en) * 2014-05-16 2015-11-19 Syed Ali Haider Method of data storage on cloud data center for reducing processing and storage requirements by engaging user equipment
US10547681B2 (en) * 2016-06-30 2020-01-28 Purdue Research Foundation Functional caching in erasure coded storage
US10740198B2 (en) 2016-12-22 2020-08-11 Purdue Research Foundation Parallel partial repair of storage
US10270469B2 (en) * 2017-05-24 2019-04-23 Vmware, Inc. Efficient data write approach for distributed multi-mirror erasure coding system
US11336473B2 (en) * 2017-10-13 2022-05-17 Northeastern University Network and method for delivering content while minimizing congestion costs by jointly optimizing forwarding and caching strategies
US10742736B2 (en) * 2017-12-19 2020-08-11 Western Digital Technologies, Inc. Integrated erasure coding for data storage and transmission
KR102222133B1 (ko) * 2018-01-12 2021-03-03 엔에이치엔 주식회사 모바일 단말기 및 그의 어플리케이션 관리 방법과 이를 활용한 타겟 광고 제공 시스템
WO2021003422A1 (en) 2019-07-02 2021-01-07 Northeastern University Network and method for servicing a computation request
JP2022074807A (ja) * 2020-11-05 2022-05-18 株式会社日立製作所 ファイルストレージ及びコンピュータシステム
CN114416135A (zh) * 2022-01-04 2022-04-29 徐工汉云技术股份有限公司 一种物联网云平台远程终端升级系统及方法
US12438823B2 (en) 2023-11-07 2025-10-07 International Business Machines Corporation Intelligent data ingestion chunk size optimization

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1656749A (zh) * 2002-06-06 2005-08-17 国际商业机器公司 数字内容分发系统、方法、程序及服务器和客户机
CN1744500A (zh) * 2004-09-03 2006-03-08 微软公司 用于可缩放媒体的分布式流传送的系统和方法
CN101815993A (zh) * 2007-09-14 2010-08-25 万年知力有限公司 数据安全设备
US20110078277A1 (en) * 2009-09-29 2011-03-31 Cleversafe, Inc. Handling unavailable memories in distributed storage network

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4685317B2 (ja) * 2002-03-29 2011-05-18 株式会社富士通ソーシアルサイエンスラボラトリ データ分散格納方法、データ分散格納装置、プログラム及びバックアップサイト
US7472334B1 (en) 2003-10-15 2008-12-30 Scott Thomas P Efficient method for the reconstruction of digital information
US20060069800A1 (en) * 2004-09-03 2006-03-30 Microsoft Corporation System and method for erasure coding of streaming media
JP2006228202A (ja) * 2005-01-21 2006-08-31 Sorun Corp 機密データの管理方法及び管理システム
JP4155341B2 (ja) * 2006-11-09 2008-09-24 コニカミノルタホールディングス株式会社 情報の管理方法及び情報処理装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1656749A (zh) * 2002-06-06 2005-08-17 国际商业机器公司 数字内容分发系统、方法、程序及服务器和客户机
CN1744500A (zh) * 2004-09-03 2006-03-08 微软公司 用于可缩放媒体的分布式流传送的系统和方法
CN101815993A (zh) * 2007-09-14 2010-08-25 万年知力有限公司 数据安全设备
US20110078277A1 (en) * 2009-09-29 2011-03-31 Cleversafe, Inc. Handling unavailable memories in distributed storage network

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107422980A (zh) * 2016-05-24 2017-12-01 上海共联通信信息发展有限公司 物联网数据文件存储系统及其数据文件存储方法
CN107422980B (zh) * 2016-05-24 2023-09-22 上海共联通信信息发展有限公司 物联网数据文件存储系统及其数据文件存储方法
CN107729176A (zh) * 2017-09-14 2018-02-23 北京京东尚科信息技术有限公司 一种配置文件管理系统的容灾方法及容灾系统
CN107729176B (zh) * 2017-09-14 2020-09-29 北京京东尚科信息技术有限公司 一种配置文件管理系统的容灾方法及容灾系统

Also Published As

Publication number Publication date
JP2015509244A (ja) 2015-03-26
US20130179490A1 (en) 2013-07-11
US8868647B2 (en) 2014-10-21
US20150012493A1 (en) 2015-01-08
EP2803180A1 (en) 2014-11-19
WO2013106400A1 (en) 2013-07-18
IN2014CN04641A (enExample) 2015-09-18
KR20140102292A (ko) 2014-08-21

Similar Documents

Publication Publication Date Title
CN104054319A (zh) 在弹性云文件系统中减少延迟和成本
CN110291509B (zh) 在分散存储网络的区域中存储数据的方法和系统
CN104272274B (zh) 一种分布式文件存储系统中的数据处理方法及设备
US11074129B2 (en) Erasure coded data shards containing multiple data objects
US11150986B2 (en) Efficient compaction on log-structured distributed file system using erasure coding for resource consumption reduction
US10073658B2 (en) Optimized caching of slices by a DS processing unit
KR101896048B1 (ko) 분산된 보안 데이터 저장 및 스트리밍 매체 콘텐트의 전송
RU2501072C2 (ru) Распределенное хранение восстанавливаемых данных
CN104902009B (zh) 一种基于可擦除编码和链式备份的分布式存储系统
CN105630423A (zh) 一种基于数据缓存的纠删码集群存储扩容方法
US11025965B2 (en) Pre-fetching content among DVRs
CN115268773B (zh) 一种纠删码数据存储方法、装置、设备及介质
CN107153661A (zh) 一种基于hdfs系统的数据的存储、读取方法及其装置
US10956091B2 (en) Expanding a dispersed storage network (DSN)
US10691541B2 (en) Method for partial updating data content in a distributed storage network
CN111149365B (zh) 用于传输数据的装置、系统和方法
US10063256B1 (en) Writing copies of objects in enterprise object storage systems
US11019146B2 (en) Segmenting and merging data in a dispersed storage network
CN114866572A (zh) 一种网络云盘
Zhang et al. BEC: A reliable and efficient mechanism for cloud storage service
CN110856144A (zh) 一种移动边缘计算网络中基于lt码的雾化缓存方法
US11586595B1 (en) Space-efficient techniques for generating unique instances of data objects
CN115408345A (zh) 应用于并行文件系统的数据存储方法及装置
KR20180088991A (ko) 파일 데이터의 중복 저장 방지 방법

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20140917

WD01 Invention patent application deemed withdrawn after publication