WO2019184096A1 - 一种下载数据资源的方法和系统 - Google Patents

一种下载数据资源的方法和系统 Download PDF

Info

Publication number
WO2019184096A1
WO2019184096A1 PCT/CN2018/091213 CN2018091213W WO2019184096A1 WO 2019184096 A1 WO2019184096 A1 WO 2019184096A1 CN 2018091213 W CN2018091213 W CN 2018091213W WO 2019184096 A1 WO2019184096 A1 WO 2019184096A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
slice
slices
target
resource
Prior art date
Application number
PCT/CN2018/091213
Other languages
English (en)
French (fr)
Inventor
陈永煌
苏宗磊
邹昱芳
Original Assignee
网宿科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 网宿科技股份有限公司 filed Critical 网宿科技股份有限公司
Priority to EP18908285.2A priority Critical patent/EP3672205A4/en
Priority to US16/473,180 priority patent/US20210344971A1/en
Publication of WO2019184096A1 publication Critical patent/WO2019184096A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2181—Source of audio or video content, e.g. local disk arrays comprising remotely distributed storage units, e.g. when movies are replicated over a plurality of video servers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/06—Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/104—Peer-to-peer [P2P] networks
    • H04L67/1074—Peer-to-peer [P2P] networks for supporting data block transmission mechanisms
    • H04L67/1078—Resource delivery mechanisms
    • H04L67/108—Resource delivery mechanisms characterised by resources being split in blocks or fragments
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/60—Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
    • H04L67/63—Routing a service request depending on the request content or context
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/24—Monitoring of processes or resources, e.g. monitoring of server load, available bandwidth, upstream requests
    • H04N21/2407—Monitoring of transmitted content, e.g. distribution time, number of downloads
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
    • H04N21/25808—Management of client data
    • H04N21/25841—Management of client data involving the geographical location of the client
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/262—Content or additional data distribution scheduling, e.g. sending additional data at off-peak times, updating software modules, calculating the carousel transmission frequency, delaying a video stream transmission, generating play-lists

Definitions

  • the present invention relates to the field of data transmission technologies, and in particular, to a method and system for downloading data resources.
  • CDN Content Delivery Network
  • CDN service clusters mostly use the Client/Server mode to realize the transmission of data resources between CDN nodes.
  • the source device can serve as a server (ie, the transmitting end of the data), and the CDN node can serve as a client (ie, a receiving end of the data), and the source device can simultaneously transmit the foregoing data resource to multiple CDN nodes, thereby Deploy data resources on multiple CDN nodes.
  • each of the above CDN nodes may also serve as a Sever, and respectively transmit data resources to different multiple clients.
  • the server often takes a lot of time to allocate a large amount of bandwidth, throughput, and the like. Performance to achieve the transmission of data resources, therefore, the efficiency of transmitting data resources is low.
  • embodiments of the present invention provide a method and system for downloading data resources.
  • the technical solution is as follows:
  • a method of downloading a data resource comprising:
  • the first device determines a plurality of candidate devices storing the data slice of the target data resource, and a slice sequence number of the data slice stored by each of the candidate devices;
  • the first device acquires a total number of slices of the data slice of the homogeneous data resource of the target data resource that has been downloaded from each of the candidate devices;
  • the first device downloads all data slices of the target data resource according to the slice sequence number, the total number of slices, and a preset timeout duration.
  • the first device downloads all data slices of the target data resource according to the slice sequence number, the total number of slices, and a preset timeout duration, including:
  • the first device counts a target slice serial number of the data slice that is not stored locally
  • the first device determines, in the candidate device, at least one device that stores the data slice corresponding to the first sequence number;
  • the first device downloads the data slice corresponding to the first serial number from the at least one device according to the preset timeout switching duration according to the total number of the slices.
  • the first device downloads all data slices of the target data resource from the at least one device according to a preset timeout switching duration, in descending order of the total number of slices, including:
  • the first device downloads, according to the device with the largest total number of slices in the at least one device, the data slice corresponding to the first serial number;
  • the second largest device corresponding to the total number of slices in the at least one device is downloaded.
  • the method further includes:
  • the first device reacquires the data of the target data resource currently stored by each of the candidate devices after the preset time length Sliced slice number;
  • the first device downloads the data slice corresponding to the first serial number from the at least one device according to the preset timeout switching duration according to the total number of the slices.
  • the method further includes:
  • the first device calculates a variance of an average download duration and a download duration of all the data slices
  • the first device updates the timeout switching duration according to the average download duration and the variance.
  • the method before the acquiring, by the first device, the slice sequence number of the data slice of the target data resource that is stored by each candidate device that is recommended by the scheduling device, the method further includes:
  • the source device of the target data resource randomly selects multiple transit devices among all the target devices corresponding to the target data resource;
  • Each of the transit devices randomly acquires a data slice of the target data resource from the source device, and reports the acquisition of the data slice to the scheduling device;
  • the scheduling device recommends, to each of the target devices, an candidate device in the transit device that belongs to the same operator/region as the target device.
  • a system for downloading a data resource comprising a plurality of network devices, the plurality of network devices comprising a first device, wherein the first device is configured to:
  • All data slices of the target data resource are downloaded according to the slice sequence number, the total number of slices, and a preset timeout duration.
  • the first device is specifically configured to:
  • the data slice corresponding to the first serial number is downloaded from the at least one device in an order of the total number of slices.
  • the first device is specifically configured to:
  • the first sequence number corresponding to the second largest device corresponding to the total number of slices in the at least one device is downloaded.
  • the data is sliced.
  • the first device is further configured to:
  • the data slice corresponding to the first serial number is downloaded from the at least one device in an order of the total number of slices.
  • the first device is further configured to:
  • the multiple network devices further include a scheduling device and a source device of the target data resource, where:
  • the source device is configured to randomly select multiple transit devices among all target devices corresponding to the target data resource;
  • Each of the transit devices is configured to randomly acquire a data slice of the target data resource from the source device, and report the acquisition of the data slice to the scheduling device;
  • the scheduling device is configured to recommend, to each of the target devices, an candidate device in the transit device that belongs to the same operator/region as the target device.
  • the first device when starting to download the target data resource, determines a plurality of candidate devices that store the data slice of the target data resource, and a slice serial number of the data slice stored by each candidate device, the first device Obtaining a total number of slices of the data slice of the same type of data resource of the target data resource that has been downloaded from each candidate device, and the first device downloads all the data slices of the target data resource according to the slice serial number, the total number of slices, and the preset timeout duration.
  • the P2P mode is used instead of the Client/Server mode, and the data resources are distributed and distributed to multiple devices, and data sharing is realized through data sharing between the devices, thereby greatly reducing the data resources.
  • the load of the source device, and the more the number of data slices downloaded by the first device from a certain device the faster the first device downloads from the device, and the better the network quality of the transmission path between the first device and the device.
  • the first device is based on sorting the total number of slices from large to small, and can select a transmission path with better network quality to transmit data resources, thereby saving time for data transmission, thereby improving the efficiency of transmitting data resources.
  • the data resources are compressed and sliced and transmitted in the form of data slices. The devices can download complete data resources with only a small amount of data interaction.
  • FIG. 1 is a system framework diagram for downloading data resources according to an embodiment of the present invention
  • FIG. 2 is a flowchart of a method for downloading a data resource according to an embodiment of the present invention
  • FIG. 3 is a flow diagram of a process for downloading a data resource according to an embodiment of the present invention.
  • the embodiment of the present invention provides a method for downloading a data resource, which may be implemented by a device cluster.
  • the device cluster includes a large number of network devices, which may be a CDN service cluster, or other clusters having data resource downloading requirements.
  • the network devices in the device cluster can be mainly divided into scheduling devices, source devices, and resource receiving devices.
  • the scheduling device may be configured to perform scheduling and management on the resource receiving device in the process of downloading the data resource by the resource receiving device, where the source device may be used to store the data resource to be distributed, and preprocess the data resource, and the resource receiving device may Used to download and store data resources.
  • the foregoing network device may include a processor, a memory, and a transceiver.
  • the processor may be used to process the downloaded data resource in the following process
  • the memory may be used to store data and data generated in the following processing.
  • the transceiver can be used to receive and transmit relevant data in the following processing.
  • the device cluster is a CDN service cluster
  • the central server, the source device, and the resource receiving device of the CDN service cluster are both CDN nodes.
  • the processing of other device clusters is similar. .
  • Step 201 When starting to download the target data resource, the first device determines a plurality of candidate devices storing the data slice of the target data resource, and a slice sequence number of the data slice stored by each candidate device.
  • the first device may be any CDN node in the CDN service cluster. For ease of understanding, the following is referred to as a first node.
  • the target data resource when a certain data resource (such as a target data resource) needs to be distributed in the CDN service cluster, the target data resource may be compressed and sliced, and the target data resource is divided into a plurality of data slices of the same size, each Each data slice corresponds to a unique slice number.
  • the data slices of the target data resource can then be stored separately in a portion of the CDN node of the CDN service cluster. In this way, when the first node receives the download instruction of the target data resource or meets the download condition of the other target data resource, the first node may start downloading the target data resource, and at this time, the first node may first determine that the target data is currently stored.
  • a plurality of candidate devices of the data slice of the resource (hereinafter referred to as candidate nodes for ease of understanding), and determining the slice sequence number of the data slice of the target data resource stored in the candidate nodes.
  • the first node may be a CDN node that has stored a data slice of the target data resource, or may be a CDN node that does not store any data slice of the target data resource.
  • Step 202 The first device acquires a total number of slices of data slices of the same type of data resource of the target data resource that has been downloaded from each candidate device.
  • the same data resource of the target data resource may be a data resource that belongs to the same topic as the target data resource.
  • the target data resource and the same data resource may be different versions of the same application, or different sets of the same TV series. Number of video files and more.
  • a CDN node may update the locally recorded data resource downloaded from the corresponding CDN node and the topic(x). The total number of slices of the data slice.
  • the first node can acquire the total number of slices of the data slice of the homogeneous data resource of the target data resource that has been downloaded from each candidate node. Further, the first node may generate a list of the best IP (Internet Protocol) for downloading the data resources under the topic (x), and the best IP list may record multiple nodes according to the total number of slices. IP address.
  • IP Internet Protocol
  • Step 203 The first device downloads all data slices of the target data resource according to the slice sequence number, the total number of slices, and the preset timeout duration.
  • the first node may switch the duration based on the preset timeout, and according to the slice serial number and the total number of slices, All data slices of the target data resource are downloaded at the alternate node.
  • the timeout period may be that the first node has been set before step 201.
  • the first node may preferentially download the data slice through the transmission path with better network quality.
  • the processing of step 203 may be as follows: the first device collects the target slice serial number of the data slice that is not stored locally; a first serial number, the first device determines, in the candidate device, at least one device that stores the data slice corresponding to the first serial number; the first device switches the duration according to the preset timeout, in descending order of the total number of slices, The data slice corresponding to the first serial number is downloaded from the at least one device.
  • the first node may first count the target slice serial number of the data slice that is not stored locally, and then select the candidate node sequence from the candidate node. Download the data slice corresponding to each target slice serial number.
  • the first sequence is used as an example.
  • the first node may determine, in the candidate node, at least one node that stores the data slice corresponding to the first sequence number, and then switch the duration according to the preset timeout, according to the total number of slices. In sequence, the data slice corresponding to the first serial number is downloaded from at least one node.
  • the processing that the first node downloads the data slice corresponding to the first sequence number based on the timeout period of the timeout may be as follows: the first device downloads the data slice corresponding to the first sequence number from the device with the largest total number of slices in the at least one device. If the first device still does not receive the data slice corresponding to the first sequence number, the data slice corresponding to the first sequence number is downloaded from the device with the second largest total number of slices in the at least one device.
  • the node with the largest total number of slices may be selected first, and then the data slice corresponding to the first sequence number is downloaded from the node.
  • the first node may start timing after sending the data slice download request corresponding to the first sequence number. If the first node has not received the data slice corresponding to the first sequence number after the preset timeout period is long, the first node may select The node with the second largest total number of corresponding slices in the at least one node downloads the data slice corresponding to the first serial number.
  • the first node may download the first node from the node with the second largest total number of slices corresponding to the at least one node.
  • the first device reacquires the slice sequence number of the data slice of the target data resource currently stored by each candidate device after the preset time length.
  • the first device re-determines at least one device storing the data slice corresponding to the first sequence number in the candidate device; the first device switches the duration according to the preset timeout, and the at least one device is in descending order of the total number of slices. Download the data slice corresponding to the first serial number.
  • the first node may re-detect after the preset time length.
  • the storage of data slices in the alternate node Specifically, taking the first sequence number as an example, if there is no node storing the data slice corresponding to the first sequence number in the determined candidate node, the first node may reacquire each candidate after the preset time length. The slice number of the data slice of the target data resource currently stored by the node.
  • each candidate node also downloads the data slice from the other node, so that even if the data slice of the first serial number does not exist in the previous candidate node, After the duration is set, the candidate node has a great probability that the data slice of the first sequence number has been obtained from the other nodes.
  • the first node may re-determine at least one node storing the data slice corresponding to the first sequence number in all the candidate nodes, and then switch the duration according to the preset timeout, in order from the total number of slices to the least, from at least one
  • the data slice corresponding to the first serial number is downloaded at the node, and the specific processing can refer to the above.
  • the CDN node may update the timeout period of the data slice according to the downloading condition of the data slice after the data resource is downloaded.
  • the specific processing may be as follows: the first device calculates the average download duration and the variance of the download duration of all the data slices. The first device updates the timeout switching duration according to the average download duration and variance.
  • the first node may obtain the download duration of each data slice, and then calculate the average download duration and the download duration variance of all the data slices, for example, the download of slice 1
  • the time can be recorded as t 1
  • the download time of slice 2 can be recorded as t 2
  • the download time of slice n can be recorded as t n
  • the variance is Further, the first node may update the timeout switching duration according to the average download duration and the variance.
  • the embodiment does not limit the method for calculating the timeout period of the timeout by using the average download duration and the variance, and the timeout switching duration can also be obtained by using other reasonable calculation methods.
  • the first node can apply the updated timeout switching duration to the download process of the next data resource.
  • the foregoing candidate node may be recommended by the central server to the first node.
  • the source device of the target data resource randomly selects all the target devices corresponding to the target data resource.
  • Each transit device randomly acquires a data slice of a target data resource from the source device, and reports the acquisition of the data slice to the scheduling device;
  • the scheduling device recommends to each target device that the transit device belongs to the same carrier as the target device. / Alternative equipment for the area.
  • the technician can select one node in the CDN service cluster as the source device (hereinafter referred to as the source node), and then deposit the target data resource into the source node. . Therefore, the source node may perform compression slicing on the target data resource, and divide the target data resource into a plurality of data slices of the same size, and each data slice corresponds to a unique slice serial number. Considering that each topic data resource has a relatively fixed user group, in the CDN service cluster, all CDN nodes can be classified according to the topic according to the region where the user group is located, and each type of topic corresponds to a different CDN node. .
  • the source node of the target data resource may first determine the topic to which the target data resource belongs, then filter out all target nodes corresponding to the topic, and randomly select multiple transit nodes in the target node. Further, each transit node may randomly acquire a data slice of a target data resource from the source node, and then report the acquisition of the data slice to the central server.
  • the acquisition of the data slice here may be the number of data slices of the acquired target data resource, or may be the slice serial number of the data slice of the acquired target data resource.
  • the central server may recommend, to each target node, an alternate node of all transit nodes that belongs to the same operator/region as the target node.
  • the first device when starting to download the target data resource, determines a plurality of candidate devices that store the data slice of the target data resource, and a slice serial number of the data slice stored by each candidate device, the first device Obtaining a total number of slices of the data slice of the same type of data resource of the target data resource that has been downloaded from each candidate device, and the first device downloads all the data slices of the target data resource according to the slice serial number, the total number of slices, and the preset timeout duration.
  • the P2P mode is used instead of the Client/Server mode, and the data resources are distributed and distributed to multiple devices, and data sharing is realized through data sharing between the devices, thereby greatly reducing the data resources.
  • the load of the source device, and the more the number of data slices downloaded by the first device from a certain device the faster the first device downloads from the device, and the better the network quality of the transmission path between the first device and the device.
  • the first device is based on sorting the total number of slices from large to small, and can select a transmission path with better network quality to transmit data resources, thereby saving time for data transmission, thereby improving the efficiency of transmitting data resources.
  • the data resources are compressed and sliced, and transmitted in the form of data slices. The devices can download complete data resources with only a small amount of data interaction.
  • This embodiment also provides an overall process signaling diagram for downloading data resources. As shown in FIG. 3, the figure includes an active node, a target node (including a transit node), and a central server.
  • the specific content may be as follows:
  • the source node performs task preprocessing, that is, compresses and slices the data resource, determines the target node, and selects the transit node;
  • the source node delivers the task to all target nodes
  • the transit node requests a data slice from the source node
  • the transit node reports the acquisition of the data slice to the central server
  • the central server determines candidate nodes of each target node
  • the central server recommends an alternate node to each target node
  • the target node records the download time and updates the timeout switch duration.
  • an embodiment of the present invention further provides a system for downloading a data resource, where the system includes a plurality of network devices, where the multiple network devices include a first device, where the first device is used for :
  • All data slices of the target data resource are downloaded according to the slice sequence number, the total number of slices, and a preset timeout duration.
  • the first device is specifically configured to:
  • the data slice corresponding to the first serial number is downloaded from the at least one device in an order of the total number of slices.
  • the first device is specifically configured to:
  • the first sequence number corresponding to the second largest device corresponding to the total number of slices in the at least one device is downloaded.
  • the data is sliced.
  • the first device is further configured to:
  • the data slice corresponding to the first serial number is downloaded from the at least one device in an order of the total number of slices.
  • the first device is further configured to:
  • the multiple network devices further include a scheduling device and a source device of the target data resource, where:
  • the source device is configured to randomly select multiple transit devices among all target devices corresponding to the target data resource;
  • Each of the transit devices is configured to randomly acquire a data slice of the target data resource from the source device, and report the acquisition of the data slice to the scheduling device;
  • the scheduling device is configured to recommend, to each of the target devices, an candidate device in the transit device that belongs to the same operator/region as the target device.
  • the first device when starting to download the target data resource, determines a plurality of candidate devices that store the data slice of the target data resource, and a slice serial number of the data slice stored by each candidate device, the first device Obtaining a total number of slices of the data slice of the same type of data resource of the target data resource that has been downloaded from each candidate device, and the first device downloads all the data slices of the target data resource according to the slice serial number, the total number of slices, and the preset timeout duration.
  • the P2P mode is used instead of the Client/Server mode, and the data resources are distributed and distributed to multiple devices, and data sharing is realized through data sharing between the devices, thereby greatly reducing the data resources.
  • the load of the source device, and the more the number of data slices downloaded by the first device from a certain device the faster the first device downloads from the device, and the better the network quality of the transmission path between the first device and the device.
  • the first device is based on sorting the total number of slices from large to small, and can select a transmission path with better network quality to transmit data resources, thereby saving time for data transmission, thereby improving the efficiency of transmitting data resources.
  • the data resources are compressed and sliced, and transmitted in the form of data slices. The devices can download complete data resources with only a small amount of data interaction.
  • a person skilled in the art may understand that all or part of the steps of implementing the above embodiments may be completed by hardware, or may be instructed by a program to execute related hardware, and the program may be stored in a computer readable storage medium.
  • the storage medium mentioned may be a read only memory, a magnetic disk or an optical disk or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • Multimedia (AREA)
  • Computer Graphics (AREA)
  • Information Transfer Between Computers (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

本发明公开了一种下载数据资源的方法和系统，属于数据传输技术领域。所述方法包括：当开始下载目标数据资源时，第一设备确定存储有所述目标数据资源的数据切片的多个备选设备，和每个所述备选设备存储的数据切片的切片序号；第一设备获取已从每个所述备选设备下载的所述目标数据资源的同类数据资源的数据切片的切片总数；第一设备根据所述切片序号、所述切片总数以及预设的超时切换时长，下载所述目标数据资源的所有数据切片。采用本发明，可以提高传输数据资源的效率。

Description

一种下载数据资源的方法和系统 技术领域
本发明涉及数据传输技术领域，特别涉及一种下载数据资源的方法和系统。
背景技术
随着互联网技术的不断进步，CDN(内容分发网络，Content Delivery Network)服务也随之快速发展。CDN服务集群中包括大量用于存储数据资源的缓存设备，用户可以以较短的时间，就近获取其中存储的数据资源。
目前CDN服务集群大多采用Client/Server模式来实现CDN节点间的数据资源的传输。当存在待传输的数据资源时，源设备可以作为Server(即数据的发送端)，CDN节点可以作为Client(即数据的接收端)，源设备可以向多个CDN节点同时传输上述数据资源，从而将数据资源部署在多个CDN节点上。进一步的，上述每个CDN节点也可以作为Sever，分别向不同的多个Client传输数据资源。
在实现本发明的过程中，发明人发现现有技术至少存在以下问题：
随着CDN服务集群的业务范围扩大，作为Client的CDN节点越来越多，同时数据资源的数据量也越来越大，故而，Server往往需要耗费大量时间，分配大量的带宽、吞吐量等传输性能来实现数据资源的传输，因此，传输数据资源的效率较低。
发明内容
为了解决现有技术的问题，本发明实施例提供了一种下载数据资源的方法和系统。所述技术方案如下：
第一方面，提供了一种下载数据资源的方法，所述方法包括：
当开始下载目标数据资源时，第一设备确定存储有所述目标数据资源的数据切片的多个备选设备，和每个所述备选设备存储的数据切片的切片序号；
第一设备获取已从每个所述备选设备下载的所述目标数据资源的同类数据 资源的数据切片的切片总数；
第一设备根据所述切片序号、所述切片总数以及预设的超时切换时长，下载所述目标数据资源的所有数据切片。
可选的，所述第一设备根据所述切片序号、所述切片总数以及预设的超时切换时长，下载所述目标数据资源的所有数据切片，包括：
第一设备统计本地未存储的数据切片的目标切片序号；
对于所述目标切片序号中的第一序号，第一设备在所述备选设备中确定存储有所述第一序号对应的数据切片的至少一个设备；
第一设备根据预设的超时切换时长，按照所述切片总数由大到小的顺序，从所述至少一个设备处下载所述第一序号对应的数据切片。
可选的，所述第一设备根据预设的超时切换时长，按照所述切片总数由大到小的顺序，从所述至少一个设备处下载所述目标数据资源的所有数据切片，包括：
第一设备从所述至少一个设备中对应的所述切片总数最大的设备处下载所述第一序号对应的数据切片；
如果经过预设的超时切换时长，第一设备仍未接收到所述第一序号对应的数据切片，则从所述至少一个设备中对应的所述切片总数第二大的设备处下载所述第一序号对应的数据切片。
可选的，所述第一设备统计本地未存储的数据切片的目标切片序号之后，还包括：
如果所有所述备选设备中均未存储所述第一序号对应的数据切片，第一设备则在预设时长后，重新获取每个所述备选设备当前存储的所述目标数据资源的数据切片的切片序号；
第一设备在所述备选设备中重新确定存储有所述第一序号对应的数据切片的至少一个设备；
第一设备根据预设的超时切换时长，按照所述切片总数由大到小的顺序，从所述至少一个设备处下载所述第一序号对应的数据切片。
可选的，所述下载所述目标数据资源的所有数据切片之后，还包括：
第一设备计算所述所有数据切片的平均下载时长和下载时长的方差；
第一设备根据所述平均下载时长和所述方差更新所述超时切换时长。
可选的，所述第一设备获取调度设备推荐的每个备选设备存储的目标数据资源的数据切片的切片序号之前，还包括：
所述目标数据资源的源设备在所述目标数据资源对应的所有目标设备中，随机选择多个中转设备；
每个所述中转设备从所述源设备处随机获取一片所述目标数据资源的数据切片，向所述调度设备上报所述数据切片的获取情况；
所述调度设备向每个所述目标设备推荐所述中转设备中与所述目标设备属于同一运营商/区域的备选设备。
第二方面，提供了一种下载数据资源的系统，所述系统包括多个网络设备，所述多个网络设备包括第一设备，其中，所述第一设备用于：
当开始下载目标数据资源时，确定存储有所述目标数据资源的数据切片的多个备选设备，和每个所述备选设备存储的数据切片的切片序号；
获取已从每个所述备选设备下载的所述目标数据资源的同类数据资源的数据切片的切片总数；
根据所述切片序号、所述切片总数以及预设的超时切换时长，下载所述目标数据资源的所有数据切片。
可选的，所述第一设备，具体用于：
统计本地未存储的数据切片的目标切片序号；
对于所述目标切片序号中的第一序号，在所述备选设备中确定存储有所述第一序号对应的数据切片的至少一个设备；
根据预设的超时切换时长，按照所述切片总数由大到小的顺序，从所述至少一个设备处下载所述第一序号对应的数据切片。
可选的，所述第一设备，具体用于：
从所述至少一个设备中对应的所述切片总数最大的设备处下载所述第一序号对应的数据切片；
如果经过预设的超时切换时长，仍未接收到所述第一序号对应的数据切片，则从所述至少一个设备中对应的所述切片总数第二大的设备处下载所述第一序号对应的数据切片。
可选的，所述第一设备，还用于：
如果所有所述备选设备中均未存储所述第一序号对应的数据切片，则在预设时长后，重新获取每个所述备选设备当前存储的所述目标数据资源的数据切片的切片序号；
在所述备选设备中重新确定存储有所述第一序号对应的数据切片的至少一个设备；
根据预设的超时切换时长，按照所述切片总数由大到小的顺序，从所述至少一个设备处下载所述第一序号对应的数据切片。
可选的，所述第一设备，还用于：
计算所述所有数据切片的平均下载时长和下载时长的方差；
根据所述平均下载时长和所述方差更新所述超时切换时长。
可选的，所述多个网络设备还包括调度设备和所述目标数据资源的源设备，其中：
所述源设备，用于在所述目标数据资源对应的所有目标设备中，随机选择多个中转设备；
每个所述中转设备，用于从所述源设备处随机获取一片所述目标数据资源的数据切片，向所述调度设备上报所述数据切片的获取情况；
所述调度设备，用于向每个所述目标设备推荐所述中转设备中与所述目标设备属于同一运营商/区域的备选设备。
本发明实施例提供的技术方案带来的有益效果是：
本发明实施例中，当开始下载目标数据资源时，第一设备确定存储有目标数据资源的数据切片的多个备选设备，和每个备选设备存储的数据切片的切片序号，第一设备获取已从每个备选设备下载的目标数据资源的同类数据资源的数据切片的切片总数，第一设备根据切片序号、切片总数以及预设的超时切换时长，下载目标数据资源的所有数据切片。这样，在设备集群下载数据资源的过程中采用P2P模式替代了Client/Server模式，将数据资源分片后分布到多个设备，通过设备之间的数据分享，实现数据资源的下载，大大降低了源设备的负载，同时第一设备从某个设备下载的数据切片数越多，可以判断第一设备从该设备下载速度越快，第一设备到该设备之间传输路径的网络质量越好，第一设备基于对切片总数由大到小的排序，可以选择网络质量较好的传输路径来传输数据资源，节省了数据传输的时间，从而可以提高传输数据资源的效率。另 外，将数据资源进行压缩切片，以数据切片的形式进行传输，设备之间可以仅通过少量数据交互，就能下载到完整的数据资源。
附图说明
为了更清楚地说明本发明实施例中的技术方案，下面将对实施例描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1是本发明实施例提供的一种下载数据资源的系统框架图；
图2是本发明实施例提供的一种下载数据资源的方法流程图；
图3是本发明实施例提供的一种下载数据资源的流程信令图。
具体实施方式
为使本发明的目的、技术方案和优点更加清楚，下面将结合附图对本发明实施方式作进一步地详细描述。
本发明实施例提供了一种下载数据资源的方法，该方法可以由设备集群实现，该设备集群包含有大量的网络设备，可以是CDN服务集群，也可以是其它存在数据资源下载需求的集群。设备集群中的网络设备可以主要分为调度设备、源设备和资源接收设备，具体的系统框架可以参照图1所示。其中，调度设备可以用于在资源接收设备下载数据资源的过程中对资源接收设备进行调度和管理，源设备可以用于存放待分发的数据资源，并对数据资源进行预处理，资源接收设备可以用于下载并存储数据资源。值得一提的是，在不同数据资源的下载过程中，可以存在不同的源设备，资源接收设备可以成为源设备，而源设备也可以实现资源接收设备的功能。上述网络设备中均可以包括处理器、存储器、收发器，处理器可以用于进行下述流程中的下载数据资源的处理，存储器可以用于存储下述处理过程中需要的数据以及产生的数据，收发器可以用于接收和发送下述处理过程中的相关数据。本实施例以设备集群为CDN服务集群、调度设备为CDN服务集群的中心服务器、源设备和资源接收设备均为CDN节点为例进行说明，其它设备集群的处理情况与之类似，不再进行赘述。
下面将结合具体实施方式，对图2所示的处理流程进行详细的说明，内容 可以如下：
步骤201，当开始下载目标数据资源时，第一设备确定存储有目标数据资源的数据切片的多个备选设备，和每个备选设备存储的数据切片的切片序号。
其中，第一设备可以是CDN服务集群中的任意CDN节点，为了便于理解，以下称为第一节点。
在实施中，当需要将某一数据资源(如目标数据资源)在CDN服务集群中进行分发时，可以对目标数据资源进行压缩切片，将目标数据资源分为多个大小相同的数据切片，每个数据切片均对应有唯一的一个切片序号。之后，可以将目标数据资源的数据切片分别存储在CDN服务集群的部分CDN节点中。这样，当第一节点接收到目标数据资源的下载指令，或者满足其它目标数据资源的下载条件时，第一节点可以开始下载目标数据资源，这时，第一节点可以先确定当前存储有目标数据资源的数据切片的多个备选设备(为了便于理解，以下称为备选节点)，并确定这些备选节点中存储的目标数据资源的数据切片的切片序号。需要说明的是，第一节点可以是已存储有目标数据资源的数据切片的CDN节点，也可以是未存储有目标数据资源的任何数据切片的CDN节点。
步骤202，第一设备获取已从每个备选设备下载的目标数据资源的同类数据资源的数据切片的切片总数。
其中，目标数据资源的同类数据资源可以是与目标数据资源属于同一主题(topic)的数据资源，例如目标数据资源和同类数据资源可以为同一应用程序不同版本的更新包、或者为同一电视剧不同集数的视频文件等等。
在实施中，某个CDN节点每次从其它CDN节点下载了某个topic(x)下的数据资源的数据切片后，可以更新本地记录的已从相应CDN节点下载topic(x)下的数据资源的数据切片的切片总数。这样，第一节点在确定了多个备选节点后，可以获取已从每个备选节点下载的目标数据资源的同类数据资源的数据切片的切片总数。进一步的，第一节点可以生成下载topic(x)下的数据资源的最佳IP(网络协议，Internet Protocol)列表，该最佳IP列表中可以按照切片总数由大到小记录有多个节点的IP地址。
步骤203，第一设备根据切片序号、切片总数以及预设的超时切换时长，下载目标数据资源的所有数据切片。
在实施中，第一节点在确定了多个备选节点，以及每个备选节点对应的切 片序号和切片总数之后，可以基于预设的超时切换时长，根据上述切片序号和切片总数从多个备选节点处下载目标数据资源的所有数据切片。此处，超时切换时长可以是第一节点在步骤201前已经设置完成的，第一节点从一个备选节点下载某个数据切片时，如果下载时长超过上述超时切换时长，则会触发切换备选节点重新下载该数据切片的处理。
可选的，第一节点可以优先通过网络质量较好的传输路径下载数据切片，相应的，步骤203的处理可以如下：第一设备统计本地未存储的数据切片的目标切片序号；对于目标切片序号中的第一序号，第一设备在备选设备中确定存储有第一序号对应的数据切片的至少一个设备；第一设备根据预设的超时切换时长，按照切片总数由大到小的顺序，从至少一个设备处下载第一序号对应的数据切片。
在实施中，第一节点在确定了多个备选节点，以及每个备选节点对应的切片序号和切片总数之后，可以先统计本地未存储的数据切片的目标切片序号，然后从备选节点中下载每个目标切片序号对应的数据切片。此处以第一序号为例，第一节点可以在备选节点中确定存储有第一序号对应的数据切片的至少一个节点，之后可以根据预设的超时切换时长，按照切片总数由大到小的顺序，从至少一个节点处下载第一序号对应的数据切片。可以理解，切片总数越大，第一节点从相应节点下载的数据切片的数量越多，则说明第一节点到相应节点之间传输路径的网络质量越好，故而第一节点在下载数据切片时，可以优先选择对应的切片总数多的节点。
可选的，上述第一节点基于超时切换时长下载第一序号对应的数据切片的处理具体可以如下：第一设备从至少一个设备中对应的切片总数最大的设备处下载第一序号对应的数据切片；如果经过预设的超时切换时长，第一设备仍未接收到第一序号对应的数据切片，则从至少一个设备中对应的切片总数第二大的设备处下载第一序号对应的数据切片。
在实施中，第一节点确定存储有第一序号对应的数据切片的至少一个节点之后，可以先选取其中对应的切片总数最大的节点，然后从该节点处下载第一序号对应的数据切片。同时，第一节点可以在发送第一序号对应的数据切片下载请求后开始计时，如果在预设的超时切换时长后，第一节点一直未接收到第一序号对应的数据切片，则可以选择从上述至少一个节点中对应的切片总数第 二大的节点下载第一序号对应的数据切片。同理，如果再次经过预设的超时切换时长后，第一节点仍未接收到第一序号对应的数据切片，则可以从上述至少一个节点中对应的切片总数第二大的节点处下载第一序号对应的数据切片，之后以此类推。
可选的，如果所有备选设备中均未存储第一序号对应的数据切片，第一设备则在预设时长后，重新获取每个备选设备当前存储的目标数据资源的数据切片的切片序号；第一设备在备选设备中重新确定存储有第一序号对应的数据切片的至少一个设备；第一设备根据预设的超时切换时长，按照切片总数由大到小的顺序，从至少一个设备处下载第一序号对应的数据切片。
在实施中，第一节点在统计出本地未存储的数据切片的目标切片序号之后，如果发现某个切片序号均不存在于所有确定出的备选节点中，则可以在预设时长后重新检测备选节点中数据切片的存储情况。具体的，以第一序号为例，如果在确定出的备选节点中不存在存储有第一序号对应的数据切片的节点，第一节点则可以在预设时长后，重新获取每个备选节点当前存储的目标数据资源的数据切片的切片序号。可以理解，在第一节点从备选节点下载数据切片的同时，每个备选节点也在从其它节点下载数据切片，这样，即使之前备选节点中不存在第一序号的数据切片，在预设时长后，备选节点有极大概率已从其它节点处获取到第一序号的数据切片。进而，第一节点可以在所有备选节点中重新确定存储有第一序号对应的数据切片的至少一个节点，然后根据预设的超时切换时长，按照切片总数由大到小的顺序，从至少一个节点处下载第一序号对应的数据切片，具体处理可以参考上文。
可选的，CDN节点可以在每次下载完数据资源后，根据数据切片的下载情况更新超时切换时长，具体的处理可以如下：第一设备计算所有数据切片的平均下载时长和下载时长的方差；第一设备根据平均下载时长和方差更新超时切换时长。
在实施中，第一节点在下载完目标数据资源的所有数据切片后，可以获取每个数据切片的下载时长，然后计算所有数据切片的平均下载时长和下载时长的方差，例如，切片1的下载时间可以记为t 1，切片2的下载时间可以记为t 2，切片n的下载时间可以记为t n，则平均下载时长为t ave＝(t 1+t 2+...+t n)/n，方差为
Figure PCTCN2018091213-appb-000001
进一步的，第一节点可以根 据平均下载时长和方差更新超时切换时长，例如，超时切换时长可以为T＝t ave+2t delta。当然，本实施例并不限定通过平均下载时长和方差计算超时切换时长的方法，还可以采用其它合理的计算方式得到超时切换时长。之后，第一节点可以将更新后的超时切换时长运用在下一次数据资源的下载过程中。
可选的，上述备选节点可以是中心服务器向第一节点推荐的，相应的，步骤201之前可以有如下处理：目标数据资源的源设备在目标数据资源对应的所有目标设备中，随机选择多个中转设备；每个中转设备从源设备处随机获取一片目标数据资源的数据切片，向调度设备上报数据切片的获取情况；调度设备向每个目标设备推荐中转设备中与目标设备属于同一运营商/区域的备选设备。
在实施中，当需要在CDN服务集群内添加目标数据资源时，技术人员可以预先在CDN服务集群中选择一个节点作为源设备(以下称为源节点)，然后将目标数据资源的存入源节点。从而，源节点可以对目标数据资源进行压缩切片，将目标数据资源分为多个大小相同的数据切片，每个数据切片均对应有唯一的一个切片序号。考虑到每个topic的数据资源均会有较为固定的用户群体，故而，CDN服务集群中可以预先根据用户群体所在区域，按照topic对所有CDN节点进行分类，每类topic均对应有不同的CDN节点。这样，目标数据资源的源节点可以先确定目标数据资源所属的topic，然后筛选出该topic对应的所有目标节点，并在目标节点中随机选择多个中转节点。进而，每个中转节点可以从源节点处随机获取一片目标数据资源的数据切片，然后向中心服务器上报数据切片的获取情况。此处数据切片的获取情况可以是获取到的目标数据资源的数据切片的数量，也可以是获取到的目标数据资源的数据切片的切片序号。进一步的，中心服务器在接收到所有中转节点上报的获取情况后，可以向每个目标节点推荐所有中转节点中与该目标节点属于同一运营商/区域的备选节点。
本发明实施例中，当开始下载目标数据资源时，第一设备确定存储有目标数据资源的数据切片的多个备选设备，和每个备选设备存储的数据切片的切片序号，第一设备获取已从每个备选设备下载的目标数据资源的同类数据资源的数据切片的切片总数，第一设备根据切片序号、切片总数以及预设的超时切换时长，下载目标数据资源的所有数据切片。这样，在设备集群下载数据资源的过程中采用P2P模式替代了Client/Server模式，将数据资源分片后分布到多个设备，通过设备之间的数据分享，实现数据资源的下载，大大降低了源设备的 负载，同时第一设备从某个设备下载的数据切片数越多，可以判断第一设备从该设备下载速度越快，第一设备到该设备之间传输路径的网络质量越好，第一设备基于对切片总数由大到小的排序，可以选择网络质量较好的传输路径来传输数据资源，节省了数据传输的时间，从而可以提高传输数据资源的效率。另外，将数据资源进行压缩切片，以数据切片的形式进行传输，设备之间可以仅通过少量数据交互，就能下载到完整的数据资源。
本实施例还提供了下载数据资源的整体流程信令图，如图3所示，图中包含有源节点、目标节点(包括中转节点)、中心服务器，具体内容可以如下：
1.源节点进行任务预处理，即对数据资源进行压缩切片，确定目标节点，选择中转节点；
2.源节点向所有目标节点下发任务；
3.中转节点向源节点请求数据切片；
4.中转节点向中心服务器上报数据切片的获取情况；
5.中心服务器确定各目标节点的备选节点；
6.中心服务器向各目标节点推荐备选节点；
7.目标节点间分享数据切片；
8.目标节点记录下载时间，更新超时切换时长。
基于相同的技术构思，本发明实施例还提供了一种下载数据资源的系统，所述系统包括多个网络设备，所述多个网络设备包括第一设备，其中，所述第一设备用于：
当开始下载目标数据资源时，确定存储有所述目标数据资源的数据切片的多个备选设备，和每个所述备选设备存储的数据切片的切片序号；
获取已从每个所述备选设备下载的所述目标数据资源的同类数据资源的数据切片的切片总数；
根据所述切片序号、所述切片总数以及预设的超时切换时长，下载所述目标数据资源的所有数据切片。
可选的，所述第一设备，具体用于：
统计本地未存储的数据切片的目标切片序号；
对于所述目标切片序号中的第一序号，在所述备选设备中确定存储有所述第一序号对应的数据切片的至少一个设备；
根据预设的超时切换时长，按照所述切片总数由大到小的顺序，从所述至少一个设备处下载所述第一序号对应的数据切片。
可选的，所述第一设备，具体用于：
从所述至少一个设备中对应的所述切片总数最大的设备处下载所述第一序号对应的数据切片；
如果经过预设的超时切换时长，仍未接收到所述第一序号对应的数据切片，则从所述至少一个设备中对应的所述切片总数第二大的设备处下载所述第一序号对应的数据切片。
可选的，所述第一设备，还用于：
如果所有所述备选设备中均未存储所述第一序号对应的数据切片，则在预设时长后，重新获取每个所述备选设备当前存储的所述目标数据资源的数据切片的切片序号；
在所述备选设备中重新确定存储有所述第一序号对应的数据切片的至少一个设备；
根据预设的超时切换时长，按照所述切片总数由大到小的顺序，从所述至少一个设备处下载所述第一序号对应的数据切片。
可选的，所述第一设备，还用于：
计算所述所有数据切片的平均下载时长和下载时长的方差；
根据所述平均下载时长和所述方差更新所述超时切换时长。
可选的，所述多个网络设备还包括调度设备和所述目标数据资源的源设备，其中：
所述源设备，用于在所述目标数据资源对应的所有目标设备中，随机选择多个中转设备；
每个所述中转设备，用于从所述源设备处随机获取一片所述目标数据资源的数据切片，向所述调度设备上报所述数据切片的获取情况；
所述调度设备，用于向每个所述目标设备推荐所述中转设备中与所述目标设备属于同一运营商/区域的备选设备。
本发明实施例中，当开始下载目标数据资源时，第一设备确定存储有目标数据资源的数据切片的多个备选设备，和每个备选设备存储的数据切片的切片序号，第一设备获取已从每个备选设备下载的目标数据资源的同类数据资源的数据切片的切片总数，第一设备根据切片序号、切片总数以及预设的超时切换时长，下载目标数据资源的所有数据切片。这样，在设备集群下载数据资源的过程中采用P2P模式替代了Client/Server模式，将数据资源分片后分布到多个设备，通过设备之间的数据分享，实现数据资源的下载，大大降低了源设备的负载，同时第一设备从某个设备下载的数据切片数越多，可以判断第一设备从该设备下载速度越快，第一设备到该设备之间传输路径的网络质量越好，第一设备基于对切片总数由大到小的排序，可以选择网络质量较好的传输路径来传输数据资源，节省了数据传输的时间，从而可以提高传输数据资源的效率。另外，将数据资源进行压缩切片，以数据切片的形式进行传输，设备之间可以仅通过少量数据交互，就能下载到完整的数据资源。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成，也可以通过程序来指令相关的硬件完成，所述的程序可以存储于一种计算机可读存储介质中，上述提到的存储介质可以是只读存储器，磁盘或光盘等。
以上所述仅为本发明的较佳实施例，并不用以限制本发明，凡在本发明的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本发明的保护范围之内。

Claims (12)

  1. 一种下载数据资源的方法，其特征在于，所述方法包括：
    当开始下载目标数据资源时，第一设备确定存储有所述目标数据资源的数据切片的多个备选设备，和每个所述备选设备存储的数据切片的切片序号；
    第一设备获取已从每个所述备选设备下载的所述目标数据资源的同类数据资源的数据切片的切片总数；
    第一设备根据所述切片序号、所述切片总数以及预设的超时切换时长，下载所述目标数据资源的所有数据切片。
  2. 根据权利要求1所述的方法，其特征在于，所述第一设备根据所述切片序号、所述切片总数以及预设的超时切换时长，下载所述目标数据资源的所有数据切片，包括：
    第一设备统计本地未存储的数据切片的目标切片序号；
    对于所述目标切片序号中的第一序号，第一设备在所述备选设备中确定存储有所述第一序号对应的数据切片的至少一个设备；
    第一设备根据预设的超时切换时长，按照所述切片总数由大到小的顺序，从所述至少一个设备处下载所述第一序号对应的数据切片。
  3. 根据权利要求2所述的方法，其特征在于，所述第一设备根据预设的超时切换时长，按照所述切片总数由大到小的顺序，从所述至少一个设备处下载所述目标数据资源的所有数据切片，包括：
    第一设备从所述至少一个设备中对应的所述切片总数最大的设备处下载所述第一序号对应的数据切片；
    如果经过预设的超时切换时长，第一设备仍未接收到所述第一序号对应的数据切片，则从所述至少一个设备中对应的所述切片总数第二大的设备处下载所述第一序号对应的数据切片。
  4. 根据权利要求2所述的方法，其特征在于，所述第一设备统计本地未存储的数据切片的目标切片序号之后，还包括：
    如果所有所述备选设备中均未存储所述第一序号对应的数据切片，第一设备则在预设时长后，重新获取每个所述备选设备当前存储的所述目标数据资源的数据切片的切片序号；
    第一设备在所述备选设备中重新确定存储有所述第一序号对应的数据切片的至少一个设备；
    第一设备根据预设的超时切换时长，按照所述切片总数由大到小的顺序，从所述至少一个设备处下载所述第一序号对应的数据切片。
  5. 根据权利要求1所述的方法，其特征在于，所述下载所述目标数据资源的所有数据切片之后，还包括：
    第一设备计算所述所有数据切片的平均下载时长和下载时长的方差；
    第一设备根据所述平均下载时长和所述方差更新所述超时切换时长。
  6. 根据权利要求1所述的方法，其特征在于，所述第一设备获取调度设备推荐的每个备选设备存储的目标数据资源的数据切片的切片序号之前，还包括：
    所述目标数据资源的源设备在所述目标数据资源对应的所有目标设备中，随机选择多个中转设备；
    每个所述中转设备从所述源设备处随机获取一片所述目标数据资源的数据切片，向所述调度设备上报所述数据切片的获取情况；
    所述调度设备向每个所述目标设备推荐所述中转设备中与所述目标设备属于同一运营商/区域的备选设备。
  7. 一种下载数据资源的系统，其特征在于，所述系统包括多个网络设备，所述多个网络设备包括第一设备，其中，所述第一设备用于：
    当开始下载目标数据资源时，确定存储有所述目标数据资源的数据切片的多个备选设备，和每个所述备选设备存储的数据切片的切片序号；
    获取已从每个所述备选设备下载的所述目标数据资源的同类数据资源的数据切片的切片总数；
    根据所述切片序号、所述切片总数以及预设的超时切换时长，下载所述目标数据资源的所有数据切片。
  8. 根据权利要求7所述的系统，其特征在于，所述第一设备，具体用于：
    统计本地未存储的数据切片的目标切片序号；
    对于所述目标切片序号中的第一序号，在所述备选设备中确定存储有所述第一序号对应的数据切片的至少一个设备；
    根据预设的超时切换时长，按照所述切片总数由大到小的顺序，从所述至少一个设备处下载所述第一序号对应的数据切片。
  9. 根据权利要求8所述的系统，其特征在于，所述第一设备，具体用于：
    从所述至少一个设备中对应的所述切片总数最大的设备处下载所述第一序号对应的数据切片；
    如果经过预设的超时切换时长，仍未接收到所述第一序号对应的数据切片，则从所述至少一个设备中对应的所述切片总数第二大的设备处下载所述第一序号对应的数据切片。
  10. 根据权利要求8所述的系统，其特征在于，所述第一设备，还用于：
    如果所有所述备选设备中均未存储所述第一序号对应的数据切片，则在预设时长后，重新获取每个所述备选设备当前存储的所述目标数据资源的数据切片的切片序号；
    在所述备选设备中重新确定存储有所述第一序号对应的数据切片的至少一个设备；
    根据预设的超时切换时长，按照所述切片总数由大到小的顺序，从所述至少一个设备处下载所述第一序号对应的数据切片。
  11. 根据权利要求7所述的系统，其特征在于，所述第一设备，还用于：
    计算所述所有数据切片的平均下载时长和下载时长的方差；
    根据所述平均下载时长和所述方差更新所述超时切换时长。
  12. 根据权利要求7所述的系统，其特征在于，所述多个网络设备还包括调度设备和所述目标数据资源的源设备，其中：
    所述源设备，用于在所述目标数据资源对应的所有目标设备中，随机选择多个中转设备；
    每个所述中转设备，用于从所述源设备处随机获取一片所述目标数据资源的数据切片，向所述调度设备上报所述数据切片的获取情况；
    所述调度设备，用于向每个所述目标设备推荐所述中转设备中与所述目标设备属于同一运营商/区域的备选设备。
PCT/CN2018/091213 2018-03-28 2018-06-14 一种下载数据资源的方法和系统 WO2019184096A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP18908285.2A EP3672205A4 (en) 2018-03-28 2018-06-14 METHOD AND SYSTEM FOR DOWNLOADING DATA RESOURCES
US16/473,180 US20210344971A1 (en) 2018-03-28 2018-06-14 A method and system for downloading a data resource

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810265146.7A CN108462755B (zh) 2018-03-28 2018-03-28 一种下载数据资源的方法和系统
CN201810265146.7 2018-03-28

Publications (1)

Publication Number Publication Date
WO2019184096A1 true WO2019184096A1 (zh) 2019-10-03

Family

ID=63237531

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/091213 WO2019184096A1 (zh) 2018-03-28 2018-06-14 一种下载数据资源的方法和系统

Country Status (4)

Country Link
US (1) US20210344971A1 (zh)
EP (1) EP3672205A4 (zh)
CN (1) CN108462755B (zh)
WO (1) WO2019184096A1 (zh)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10949301B2 (en) * 2011-06-06 2021-03-16 Pure Storage, Inc. Pre-positioning pre-stored content in a content distribution system
CN108990039B (zh) * 2018-08-08 2021-04-13 Oppo广东移动通信有限公司 数据传输方法及相关装置
CN111246315B (zh) * 2020-01-21 2022-06-03 咪咕视讯科技有限公司 视频交互方法、系统、边缘节点、客户端及存储介质
CN113347137B (zh) * 2020-03-02 2024-03-05 广州虎牙科技有限公司 数据流切片的补片方法、装置、计算机设备及存储介质
WO2022021357A1 (zh) * 2020-07-31 2022-02-03 华为技术有限公司 一种文件块下载方法及装置
JP2023018365A (ja) * 2021-07-27 2023-02-08 富士通株式会社 情報処理プログラム、情報処理方法及び情報処理装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101150506A (zh) * 2007-08-24 2008-03-26 华为技术有限公司 内容获取方法、装置和内容传输系统
CN103188279A (zh) * 2011-12-27 2013-07-03 中国电信股份有限公司 通过对等网络从多个邻居节点下载文件的方法和装置
US9043489B2 (en) * 2009-10-30 2015-05-26 Cleversafe, Inc. Router-based dispersed storage network method and apparatus
CN106993054A (zh) * 2017-05-05 2017-07-28 腾讯科技（深圳）有限公司 文件分发方法、节点及系统

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100427143B1 (ko) * 2003-01-17 2004-04-14 엔에이치엔(주) 스트리밍 데이터 전송 및 다운로드 방법
US20080189429A1 (en) * 2007-02-02 2008-08-07 Sony Corporation Apparatus and method for peer-to-peer streaming
US8122488B2 (en) * 2007-05-18 2012-02-21 Yangaroo, Inc. Media file distribution system and method
CN101478556B (zh) * 2007-12-31 2014-12-17 突触计算机系统（上海）有限公司 用于下载点对点传输数据分片的方法和装置
CN101588287B (zh) * 2008-05-20 2011-11-16 华为技术有限公司 对等网络数据调度和下载的方法、装置和系统
CN101437044B (zh) * 2008-11-28 2012-05-09 中兴通讯股份有限公司 基于切片的点对点网络技术的文件下载方法及系统
CN101500022B (zh) * 2009-03-09 2013-03-20 北大方正集团有限公司 一种数据访问资源的分配方法及其系统和设备
US9038116B1 (en) * 2009-12-28 2015-05-19 Akamai Technologies, Inc. Method and system for recording streams
CN102404182B (zh) * 2010-09-07 2014-12-31 中国移动通信集团公司 传输控制方法及装置
CN103118113B (zh) * 2013-02-01 2015-09-16 中交四航工程研究院有限公司 一种对等网络及其网络资源定位方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101150506A (zh) * 2007-08-24 2008-03-26 华为技术有限公司 内容获取方法、装置和内容传输系统
US9043489B2 (en) * 2009-10-30 2015-05-26 Cleversafe, Inc. Router-based dispersed storage network method and apparatus
CN103188279A (zh) * 2011-12-27 2013-07-03 中国电信股份有限公司 通过对等网络从多个邻居节点下载文件的方法和装置
CN106993054A (zh) * 2017-05-05 2017-07-28 腾讯科技（深圳）有限公司 文件分发方法、节点及系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP3672205A4 *

Also Published As

Publication number Publication date
EP3672205A1 (en) 2020-06-24
EP3672205A4 (en) 2020-12-16
CN108462755B (zh) 2020-06-05
CN108462755A (zh) 2018-08-28
US20210344971A1 (en) 2021-11-04

Similar Documents

Publication Publication Date Title
WO2019184096A1 (zh) 一种下载数据资源的方法和系统
US20210311781A1 (en) Method and system for scalable job processing
CN108696374B (zh) 更新客户端配置的方法和装置
US11356748B2 (en) Method, apparatus and system for slicing live streaming
US10200495B2 (en) CDN scale down
Sharma et al. Wormhole: Reliable {Pub-Sub} to Support Geo-replicated Internet Services
WO2008025297A1 (fr) Procédé de téléchargement de fichiers selon la technique p2p (pair à pair) et système de téléchargement p2p
EP3447970B1 (en) Live broadcast system in peer-to-peer network and node management method
CN111614748B (zh) 用于可扩展的对等匹配的设备及方法
US11005975B2 (en) Rapid optimization of media stream bitrate
CN112311874B (zh) 媒体数据处理方法、装置、存储介质及电子设备
CN112311897A (zh) 资源文件下载方法、装置、设备及介质
US10637962B2 (en) Data request multiplexing
CN113891175B (zh) 直播推流方法、装置及系统
WO2016044329A1 (en) Real-time, low memory estimation of unique client computers communicating with a server computer
CN111541684B (zh) 直播间的信令发送方法、装置、服务器及存储介质
CN113438129A (zh) 数据采集方法及装置
WO2013120412A1 (zh) 数据下载方法、终端、服务器和系统
US7319671B1 (en) Time division polling scheme for network management systems
CN110839061B (zh) 数据分发方法、装置及存储介质
WO2017215415A1 (zh) 一种资源控制方法、装置和iptv服务器
CN112169312A (zh) 云游戏服务的排队调度方法、装置、设备和存储介质
CN108667920B (zh) 一种雾计算环境业务流量加速系统及其业务流量加速方法
KR100681710B1 (ko) Ｐ2ｐ네트웍에서의 대용량 파일분배 방법
CN114866597B (zh) 一种分组管理客户端连接方法以及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18908285

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE