CN106657182B - 云端文件处理方法和装置 - Google Patents

云端文件处理方法和装置 Download PDF

Info

Publication number
CN106657182B
CN106657182B CN201510728030.9A CN201510728030A CN106657182B CN 106657182 B CN106657182 B CN 106657182B CN 201510728030 A CN201510728030 A CN 201510728030A CN 106657182 B CN106657182 B CN 106657182B
Authority
CN
China
Prior art keywords
read
data packets
target file
state
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201510728030.9A
Other languages
English (en)
Chinese (zh)
Other versions
CN106657182A (zh
Inventor
杨晔辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba Group Holding Ltd
Original Assignee
Alibaba Group Holding Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
Priority to CN201510728030.9A priority Critical patent/CN106657182B/zh
Priority to PCT/US2016/059561 priority patent/WO2017075520A1/en
Priority to US15/338,178 priority patent/US10382551B2/en
Priority to EP16860995.6A priority patent/EP3369238B1/en
Priority to JP2018519335A priority patent/JP6966432B2/ja
Publication of CN106657182A publication Critical patent/CN106657182A/zh
Application granted granted Critical
Publication of CN106657182B publication Critical patent/CN106657182B/zh
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
    • H04L67/5681—Pre-fetching or pre-delivering data based on network characteristics
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601—Interfaces specially adapted for storage systems
    • G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
    • G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Transfer Between Computers (AREA)
CN201510728030.9A 2015-10-30 2015-10-30 云端文件处理方法和装置 Active CN106657182B (zh)

Priority Applications (5)

Application Number Priority Date Filing Date Title
CN201510728030.9A CN106657182B (zh) 2015-10-30 2015-10-30 云端文件处理方法和装置
PCT/US2016/059561 WO2017075520A1 (en) 2015-10-30 2016-10-28 Cloud file processing method and apparatus
US15/338,178 US10382551B2 (en) 2015-10-30 2016-10-28 Cloud file processing method and apparatus
EP16860995.6A EP3369238B1 (en) 2015-10-30 2016-10-28 Method, apparatus, computer-readable medium and computer program product for cloud file processing
JP2018519335A JP6966432B2 (ja) 2015-10-30 2016-10-28 クラウドファイル処理方法および装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510728030.9A CN106657182B (zh) 2015-10-30 2015-10-30 云端文件处理方法和装置

Publications (2)

Publication Number Publication Date
CN106657182A CN106657182A (zh) 2017-05-10
CN106657182B true CN106657182B (zh) 2020-10-27

Family

ID=58630878

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510728030.9A Active CN106657182B (zh) 2015-10-30 2015-10-30 云端文件处理方法和装置

Country Status (5)

Country Link
US (1) US10382551B2 (https=)
EP (1) EP3369238B1 (https=)
JP (1) JP6966432B2 (https=)
CN (1) CN106657182B (https=)
WO (1) WO2017075520A1 (https=)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10419401B2 (en) * 2016-01-08 2019-09-17 Capital One Services, Llc Methods and systems for securing data in the public cloud
CN111324677A (zh) * 2018-12-13 2020-06-23 中国移动通信集团山西有限公司 用户位置数据的获取方法、装置、设备及介质
CN112395337B (zh) * 2019-08-12 2023-09-26 北京京东振世信息技术有限公司 一种数据导出方法和装置
CN112383632A (zh) * 2020-11-17 2021-02-19 珠海大横琴科技发展有限公司 一种负载均衡的方法和装置
CN112559574B (zh) * 2020-12-25 2023-10-13 北京百度网讯科技有限公司 数据处理方法、装置、电子设备及可读存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102223416A (zh) * 2011-06-24 2011-10-19 Tcl集团股份有限公司 一种媒体文件的传输方法及系统
CN101772938B (zh) * 2007-08-21 2013-03-20 株式会社Ntt都科摩 带有在线缓存和对等转发的媒体流式传输
CN103139241A (zh) * 2011-11-24 2013-06-05 腾讯科技（深圳）有限公司 网络资源文件的离线下载系统和方法

Family Cites Families (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001057571A (ja) * 1998-09-14 2001-02-27 Matsushita Electric Ind Co Ltd ファイルシステム
US6842770B1 (en) * 2000-08-18 2005-01-11 Apple Computer, Inc. Method and system for seamlessly accessing remotely stored files
US7043524B2 (en) 2000-11-06 2006-05-09 Omnishift Technologies, Inc. Network caching system for streamed applications
US7062567B2 (en) 2000-11-06 2006-06-13 Endeavors Technology, Inc. Intelligent network streaming and execution system for conventionally coded applications
US6959320B2 (en) 2000-11-06 2005-10-25 Endeavors Technology, Inc. Client-side performance optimization system for streamed applications
US6918113B2 (en) 2000-11-06 2005-07-12 Endeavors Technology, Inc. Client installation and execution system for streamed applications
JP2004535713A (ja) 2001-06-04 2004-11-25 エヌシーティー グループ インコーポレーテッド 通信回路網の有効なバンド幅を大きくすることためのシステムと方法
US20040210949A1 (en) * 2003-04-21 2004-10-21 Matsushita Electric Industrial Co., Ltd. Data reception and playback apparatus, data reception and playback method, and data reception and playback processing program
US8966000B2 (en) * 2007-06-22 2015-02-24 Microsoft Corporation Aggregation and re-ordering of input/output requests for better performance in remote file systems
TWI383640B (zh) * 2009-10-22 2013-01-21 Wistron Corp 利用網路傳輸檔案之方法與檔案傳輸系統
US8527549B2 (en) 2010-02-22 2013-09-03 Sookasa Inc. Cloud based operating and virtual file system
GB201004449D0 (en) 2010-02-22 2010-05-05 Corbett Sean Data accelerator
US8977661B2 (en) 2010-02-22 2015-03-10 Sookasa Inc. System, method and computer readable medium for file management
US9021229B2 (en) 2010-04-14 2015-04-28 International Business Machines Corporation Optimizing a file system for different types of applications in a compute cluster using dynamic block size granularity
US8799413B2 (en) 2010-05-03 2014-08-05 Panzura, Inc. Distributing data for a distributed filesystem across multiple cloud storage systems
US8788628B1 (en) 2011-11-14 2014-07-22 Panzura, Inc. Pre-fetching data for a distributed filesystem
US8805968B2 (en) 2010-05-03 2014-08-12 Panzura, Inc. Accessing cached data from a peer cloud controller in a distributed filesystem
US9811532B2 (en) * 2010-05-03 2017-11-07 Panzura, Inc. Executing a cloud command for a distributed filesystem
US8805951B1 (en) 2011-02-08 2014-08-12 Emc Corporation Virtual machines and cloud storage caching for cloud computing applications
CN102833294B (zh) 2011-06-17 2015-05-20 阿里巴巴集团控股有限公司 基于云存储的文件处理方法、系统及服务器集群系统
US8683008B1 (en) 2011-08-04 2014-03-25 Google Inc. Management of pre-fetched mapping data incorporating user-specified locations
US8204966B1 (en) 2011-09-26 2012-06-19 Google Inc. Map tile data pre-fetching based on user activity analysis
CN104969604B (zh) 2012-08-14 2020-06-02 俄亥俄州立创新基金会 有效使用网络带宽的系统和方法
US20140223099A1 (en) 2013-02-06 2014-08-07 Adam Kidron Content management platform apparatus, methods, and systems
US9614925B2 (en) * 2014-03-31 2017-04-04 Futurewei Technologies, Inc. Intelligent file pre-fetch based on access patterns
CN104166567A (zh) * 2014-08-12 2014-11-26 广州金山网络科技有限公司 一种下载网络流数据的方法及装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101772938B (zh) * 2007-08-21 2013-03-20 株式会社Ntt都科摩 带有在线缓存和对等转发的媒体流式传输
CN102223416A (zh) * 2011-06-24 2011-10-19 Tcl集团股份有限公司 一种媒体文件的传输方法及系统
CN103139241A (zh) * 2011-11-24 2013-06-05 腾讯科技（深圳）有限公司 网络资源文件的离线下载系统和方法

Also Published As

Publication number Publication date
US10382551B2 (en) 2019-08-13
EP3369238B1 (en) 2021-05-12
US20170126806A1 (en) 2017-05-04
JP6966432B2 (ja) 2021-11-17
EP3369238A4 (en) 2018-09-05
WO2017075520A1 (en) 2017-05-04
JP2018532202A (ja) 2018-11-01
EP3369238A1 (en) 2018-09-05
CN106657182A (zh) 2017-05-10

Similar Documents

Publication Publication Date Title
CN113010818A (zh) 访问限流方法、装置、电子设备及存储介质
CN106657182B (zh) 云端文件处理方法和装置
CN110545246A (zh) 一种基于令牌桶的限流方法和装置
KR101719500B1 (ko) 캐싱된 플로우들에 기초한 가속
US10015253B2 (en) System and method for preemptive request processing
CN103780676B (zh) 一种文件传输方法、设备和系统
CN105337891B (zh) 用于分布式缓存系统的流量控制方法和装置
CN104657411A (zh) 用于信息技术资源管理的方法和系统
US20170153909A1 (en) Methods and Devices for Acquiring Data Using Virtual Machine and Host Machine
CN105468737A (zh) 一种网络服务大数据分析方法、云计算平台及挖掘系统
JP2018506794A (ja) Ｉ／ｏステアリングエンジンを備えるシステムオンチップ
US9716666B2 (en) Process cage providing attraction to distributed storage
CN110650209A (zh) 实现负载均衡的方法和装置
CN110168513B (zh) 在不同存储系统中对大文件的部分存储
WO2016101748A1 (zh) 一种网络连接的缓存方法和装置
CN107633102A (zh) 一种读取元数据的方法、装置、系统及设备
US10901797B2 (en) Resource allocation
CN112887349B (zh) 分发文件的方法和装置
US9137276B2 (en) Managing message distribution in a networked environment
CN114116642A (zh) 一种基于wrr队列的日志去重和产生方法及系统
CN114741165A (zh) 数据处理平台的处理方法、计算机设备及存储装置
CN113849125A (zh) 一种cdn服务器磁盘读取的方法、装置及系统
CN114979411A (zh) 一种分布式图像处理方法、装置、设备及系统
US12335153B2 (en) Dynamic quality-of-service (QOS) configuration and forecasting based on workload intent
CN111030856B (zh) 一种基于云的数据接入方法、电子设备及计算机可读介质

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant