CN113453038B - Effectiveness optimal collaborative cache management method under CDN-P2P hybrid architecture - Google Patents

Effectiveness optimal collaborative cache management method under CDN-P2P hybrid architecture Download PDF

Info

Publication number
CN113453038B
CN113453038B CN202110709682.3A CN202110709682A CN113453038B CN 113453038 B CN113453038 B CN 113453038B CN 202110709682 A CN202110709682 A CN 202110709682A CN 113453038 B CN113453038 B CN 113453038B
Authority
CN
China
Prior art keywords
node
nodes
super
video
resource
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110709682.3A
Other languages
Chinese (zh)
Other versions
CN113453038A (en
Inventor
刘平山
刘韶星
孙业民
俞中舒
黄桂敏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guilin University of Electronic Technology
Original Assignee
Guilin University of Electronic Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guilin University of Electronic Technology filed Critical Guilin University of Electronic Technology
Priority to CN202110709682.3A priority Critical patent/CN113453038B/en
Publication of CN113453038A publication Critical patent/CN113453038A/en
Application granted granted Critical
Publication of CN113453038B publication Critical patent/CN113453038B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H04N21/23106—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion involving caching operations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/104—Peer-to-peer [P2P] networks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/10—Protocols in which an application is distributed across nodes in the network
    • H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/50—Network services
    • H04L67/56—Provisioning of proxy services
    • H04L67/568—Storing data temporarily at an intermediate stage, e.g. caching
    • H04L67/5682—Policies or rules for updating, deleting or replacing the stored data
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2181—Source of audio or video content, e.g. local disk arrays comprising remotely distributed storage units, e.g. when movies are replicated over a plurality of video servers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H04N21/23113—Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion involving housekeeping operations for stored content, e.g. prioritizing content for deletion because of storage space restrictions

Abstract

The invention discloses a method for managing an optimal collaborative cache under a CDN-P2P hybrid architecture, which comprises the steps of firstly constructing a super node group, supporting the combination of a P2P technology and a content distribution network, and providing services for other nodes by utilizing the characteristics of high expandability and low deployment cost of a P2P overlay network; secondly, constructing a corresponding mathematical model based on the global utility value according to the resource supply and demand relationship among ISPs domains, and optimizing the model by using a resource allocation greedy algorithm; and finally, designing a cooperative cache management method based on an optimal utility model, and placing the video resource with high popularity at the side close to the user through cooperative cache among nodes in the super node group and CDNs based on uploading bandwidth of the sending node and downloading bandwidth and disk capacity of the receiving node, so that the flow of a return link among ISPs is reduced, and the playing smoothness and the cache hit rate of the user are improved.

Description

Effectiveness optimal collaborative cache management method under CDN-P2P hybrid architecture
Technical Field
The invention relates to the technical field of computer networks, peer-to-peer network technologies and streaming media, in particular to a method for managing an optimal collaborative cache under a CDN-P2P hybrid architecture.
Background
Network video is one of the main applications of the internet and one of the main traffic sources in the network, and the user scale is large and growing, and the large user scale generates huge bandwidth cost. In recent years, the HTTP adaptive Streaming media technology is abbreviated as DASH, which combines the advantages of the conventional Real Time Streaming Protocol (RTSP)/(Real Time Protocol, RTP) technology and the HTTP progressive download technology. The method has the characteristics of simple deployment, easy expansion and the like, and is widely applied. Representative products of commercial deployment include Smooth Streaming technology (SS) by microsoft, HLS (HTTP Live Streaming) by apple inc, HDS (HTTP Dynamic Streaming) by Adobe. The streaming media transmission has the greatest advantages that the whole media file cannot be downloaded before playing, only part of media content is cached, a user can play the streaming media without waiting for the completion of downloading of the whole streaming media resource, and the user can watch the streaming media resource only by waiting for a short start delay, so that the streaming media can be played simultaneously.
The P2P technology is combined with a Content Delivery Network (CDN) to build a hybrid CDN-P2P architecture. The CDNs nodes are organized in a P2P mode, and data sharing and caching are carried out among the CDNs nodes in a P2P connection mode, so that the pressure of a central server can be effectively relieved. However, CDN technology for video delivery is still a terminal/server architecture in nature, and when users are large, the bandwidth cost becomes expensive. The peer-to-peer network technology has been proven to be a technology for reducing bandwidth cost and improving system expansibility in the streaming media field, and if the peer-to-peer network technology is fused with the HTTP adaptive streaming media technology, the advantages of low cost, high expansibility and the like of the peer-to-peer network technology can be utilized, and the advantages of simple deployment, easy expansion and the like of the peer-to-peer network technology can also be utilized.
In the P2P streaming system, the streaming video segments required by the nodes come from nearby nodes or servers. When the adjacent nodes of one node have no corresponding streaming media segment resource, the request is sent to the server. With the popularization of vast intelligent terminal devices (smart phones, ipads, portable computers, and the like), many devices have large capacity and high bandwidth. If multiple types of terminals can adopt the same streaming media technology and a content distribution strategy of mutual cooperation, a cooperation caching scheme is adopted based on the equipment caching characteristic, and the transmission streaming media video sharing is facilitated. Many fixed terminal devices (desktop computers, televisions, etc.) currently exist with a disk buffer that stores and holds video for a significant period of time, which makes some popular videos even longer to serve. However, not every end node has the ability to serve other nodes, and therefore, selecting some well-performing nodes from the end devices for serving can offload a portion of the backhaul traffic. Considering that the service capability of a single terminal node is weak, how to organize the terminal nodes to be aggregated into a node group is a problem to be considered. Most of the traffic cost existing in the network is generated by transmitting data across ISPs, and how to construct a utility mathematics-based model and optimize the model is an important problem according to the supply-demand relationship of video resources among ISPs. Based on the utility optimal model, how to use the CDNs nodes and the terminal nodes to decide the leaving of the video to be cached is an important problem.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a method for managing an optimal collaborative cache under a CDN-P2P hybrid architecture.
The technical scheme for realizing the purpose of the invention is as follows:
a method for managing an optimal collaborative cache under a CDN-P2P hybrid architecture comprises the steps of firstly constructing a super node group; secondly, constructing a corresponding mathematical model based on a global optimization target according to the resource supply and demand relationship among ISPs domains, and optimizing the model by using a resource allocation greedy algorithm; finally, designing a collaborative cache management method based on the utility optimal model;
the super node cluster takes network time delay between super nodes as a distance between the two nodes, the super nodes establish the super node cluster based on a principle of proximity, the super nodes in the cluster are neighbor nodes, each super node maintains a group of neighbor super node information, a sharing area between the nodes is relatively transparent, the super nodes can acquire video resources from the sharing area of the neighbor nodes, and the specific steps of establishing the super node cluster are as follows:
1-1) starting;
1-2) calculating the Ability value Abiliity of the common node ii；
1-3) Ability to comply with Abiliity valuesiArranging the nodes in descending order to obtain an ordered node list orderSu Peer List；
1-4) put the List order Su Peer List in front
Figure GDA0003506419120000021
Each node is marked as a super node, and a mark symbol SPflag is 1, wherein Numsp represents the number of super nodes, N represents the total number of nodes, wherein
Figure GDA0003506419120000023
Represents a super node occupation ratio, and
Figure GDA0003506419120000022
1-5) Each super node k1Sending a detection packet to other super nodes to obtain a super node list order Peer Rtt Dist with transmission delay less than delta, and sequencing the list according to the ascending order of delay; delta represents the maximum acceptable time delay, and when the time delay is too large, a large amount of time is needed for periodic information exchange between nodes.
1-6) sequentially inquiring whether a super node k exists in the list order Peer Rtt DistspWhether the size of the group is smaller than tau; if yes, the super node k1Adding kspThe group in which the gene belongs; otherwise, the super node k1Establishing a new group; τ represents the maximum number of nodes in a super node cluster, and if the number of super nodes in a cluster is too small, the benefit brought by the shared service is not very large, and when the number of nodes is too large, the nodes in the cluster are difficult to maintain.
1-7) finishing;
the mathematical model is established according to the supply and demand relationship among ISPs domains, and is optimized through a resource allocation greedy algorithm to ensure that the effectiveness of ISP domain request resources is optimal, wherein the mathematical model expression is as follows:
Figure GDA0003506419120000031
in the above formula (1), QvIndicating the missing target resource, tωRepresenting a link cost;
the resource allocation greedy algorithm is to set a resource priority acquisition path, preferentially acquire resources from a super node group of a neighbor ISP domain, and then acquire resources from a CDN node, and specifically comprises the following steps:
2-1) starting;
2-2) initialization Utility value Utility, t1，t2，t3，t4(t1,t2,t3,t4∈tω)；
2-3) obtaining target resource Q lacking in ISP domainv；
2-4) acquiring neighbor ISPs list information, and executing steps 2-5) -2-7) on each neighbor ISP domain;
2-5) judging whether Q exists in the super node groupvIf not, executing step 6), if yes, updating QvAnd Utility, judgment QvIf the result is empty, executing the step 2-8), otherwise, executing the step 2-6);
2-6) judging whether Q exists in CDN nodevIf not, executing the step 2-7), and if so, updating the QvAnd Utility, judgment QvIf the result is empty, executing the step 2-8) if the result is empty, and executing the step 2-7) if the result is not empty;
2-7) judging whether Q exists in neighbor nodes of the CDN nodevIf not, executing the step 2-8), and if so, updating the QvAnd Utility;
2-8) ending;
the cooperative cache management method comprises a cooperative cache mode of a super node in a super node group, a cooperative cache mode of the super node group and CDNs nodes and a cache replacement method;
the cooperative caching mode of the super nodes in the super node clusters is that each super node cluster is regarded as a whole, the nodes in the clusters can share resources, and the nodes in the clusters can store more video resources in limited capacity by caching different video resources, so that the diversity of the resources in the clusters is improved;
the method is characterized in that a coordinated cache mode of the super node group and the CDNs nodes is that target resources are divided into popular resources and secondary popular resources according to popularity degrees, the popular resources are stored in the super node group close to a user side preferentially, and the secondary popular resources are stored in the CDNs nodes secondly;
the cache replacement method specifically comprises the following steps:
3-1) starting;
3-2) judging whether the current node has enough bandwidth, if not, executing the step 3-5), otherwise, judging whether the disk capacity of the node is enough, if so, directly caching the resource, otherwise, executing the step 3-3) -the step 3-4);
3-3) calculating the Energy value Energy of the video resource in the diskvSorting the video resources in a descending order according to the energy values;
3-4) replacing the video resource to be cached with the video resource with the minimum energy value;
3-5) finishing.
In step 1-2), the Ability value AbiliityiThe evaluation index representing the service performance of the node is comprehensively determined by four factors of historical online time, bandwidth, storage capacity and exit frequency of the node. The historical online time, the bandwidth and the capacity of the node serve as positive attributes, and the historical exit frequency serves as negative attributes, but the positive attributes are not weighted as much as possible. The longer the historical online time of a node is, the longer the continuous service time may be, and it is not absolute that when the bandwidth and the capacity of the node are small, the node has no free bandwidth and capacity resource service request node although the historical online time of the node is large. The historical exit frequency of the node reflects the online stable state of the node, when the bandwidth and the capacity of the node are large, the higher the historical exit frequency is, the request node cannot obtain stable service guarantee, in addition, the historical online time and the historical exit frequency are mutually restricted, the larger the historical exit frequency is, and the larger the historical online time is, the node cannot be used as a service node, and the capacity value of the node is defined as follows:
Abilityi＝ρ1OTime+ρ2Bandwidth+ρ3diskCapacity+ρ4exitFreq (2)
in the formula (2), OTime represents the historical online time of the node, Bandwidth represents the Bandwidth of the node, diskCapacity represents the disk capacity of the node, exitFreq represents the exit frequency of the node, and rho1、ρ2、ρ3And ρ4Coefficients representing historical online duration, bandwidth, disk capacity, and historical exit frequency attributes, respectively, and ρ1+ρ2+ρ3+ρ4＝1。
The expression of the mathematical model meets the following constraint conditions:
constraint 1:
Figure GDA0003506419120000051
ensuring that the available bandwidth of the node is not less than 0 in the sending period and bandwidthi,avaiIs the available bandwidth of node i, sjIs the length of video segment j, Γ is the maximum allowed transmission duration;
constraint 2: q. q.si,v∈{0,1}，
Figure GDA0003506419120000052
v∈QvThe solution used to represent the problem is discrete.
In step 2-3), resource QvFrom super node clusters and CDNs nodes, QvThe calculation formula of (2) is as follows:
Figure GDA0003506419120000053
in the above formula (3), G represents a neighbor ISP set; g _ sps represents a set of super node clusters within an ISP domain, g _ CDN represents a CDN node within an ISP domain, Xv,sE {0,1}, is a binary representation of the resource in the supernode, Xv,sWhen the value is 1, the resource v exists in the super node s, and otherwise, the resource v does not exist;
Figure GDA0003506419120000054
is a resource binary representation of a neighboring super node, Yv,cBeing CDN nodesThe binary representation of the resources is represented by,
Figure GDA0003506419120000055
is a resource binary representation of a neighboring CDN node.
In the step 2-7), the calculation formula of the Utility value Utility is as follows:
Figure GDA0003506419120000056
in the above formula (4), t1Representing the link cost, t, of acquiring resources from a super node s2Represents the link cost, t, of acquiring resources from s' neighbor nodes2＝at1,t2>t1，(a＝1，2，…，n)，t3Represents the link cost, t, of acquiring resources from a CDN node4Represents the link cost, t, of acquiring resources from CDN neighbor nodes4＝bt3，t4>t3And b is 1, 2, …, n, and a and b respectively represent the number of super node hops and the number of CDN hops.
In step 3-3), the Energy value EnergyvThe method comprehensively considers the popularity, the scarcity, the request frequency and the video size of the video, carries out weighted summation on the four factors, and obtains the Energy value Energy of the videovThe calculation formula is as follows:
Energyv＝σ·Pv+β·Scarcityv+γ·Reqv,T+λ·Sizev (5)
in the above formula (5), PvRepresenting the popularity of the video v, ScarcityvIndicating the degree of scarcity, Size, of the video vvRepresenting the size of the video, Reqv,TRepresents the number of times that the video v is requested in a period T, and sigma, beta, gamma and lambda represent the parameters of the popularity, the Scarcity, the request frequency and the video size of the video respectively, ScarcityvThe calculation formula of (a) is as follows:
Figure GDA0003506419120000061
num in the above formula (6)vRepresenting the number of copies of video v and TotalNum representing the number of copies of all videos.
The invention provides an optimal-utility collaborative cache management method under a CDN-P2P hybrid architecture, which comprises the steps of firstly, considering the limitation of service capability of a single node, constructing a super node group, supporting the combination of a P2P technology and content distribution, and utilizing the characteristics of high expandability and low deployment cost of a P2P overlay network to serve other nodes; secondly, constructing a corresponding mathematical model based on the global utility value according to the resource supply and demand relationship among ISPs domains, and optimizing the model by using a resource allocation greedy algorithm to achieve the optimal global utility; and finally, designing a cooperative cache management method based on an optimal utility model, preferentially placing the video resources with high popularity in the super node group close to the side of the common user and placing the video resources with low popularity in the CDNs nodes through the cooperative cache among the nodes in the super node group and the cooperative cache among the super node group and the CDNs nodes under the constraints of uploading bandwidth of the sending node and downloading bandwidth and disk capacity of the receiving node, thereby reducing the flow of a backhaul link between ISPs and improving the playing fluency and cache hit rate of the user.
Drawings
FIG. 1 is a CDN-P2P hybrid stream architecture diagram;
FIG. 2 is a flow chart of a process for constructing a cluster of supernodes;
FIG. 3 is a flow diagram of a resource allocation greedy algorithm;
FIG. 4 is a flow chart of a cache replacement method.
Detailed Description
The invention will be further elucidated with reference to the drawings and examples, without however being limited thereto.
Example (b):
fig. 1 is a CDN-P2P hybrid streaming architecture diagram according to the present invention, which is composed of a video source server, a tracking server, edge CDN nodes, super nodes (special user nodes), and general nodes (user nodes). The video source server firstly distributes the video to edge CDNs nodes in different ISPs, and then the video is distributed to common nodes by the edge CDNs nodes. The architecture comprises two layers of overlay networks, one layer is an overlay network organized by CDNs nodes, and the other layer is an overlay network formed by super nodes.
The super node group is composed of fixed terminal nodes (fixed nodes) with good performance, the fixed nodes are provided with a disk cache region which can be used for storing resources for a long time, and a sharing region which can be used for sharing the resources with the adjacent nodes and playing the cache region for playing the requested resources. Each super node is a request node and a service node, and stores a neighbor node information list and a neighbor node entry list, which respectively record an IP address, a port number, a URL address, an IP address, a URL address and a video entry.
When a super node leaves the network, it firstly sends the information of exiting the network to the tracking server of the ISP domain and the neighbor super nodes in the cluster, and the tracking server and the neighbor super nodes update the self list information.
When a super node suddenly leaves the network due to network disconnection or failure, the tracking server and the neighbor super nodes do not receive periodic broadcast information from the node, the node leaves the network due to abnormality, and at the moment, the tracking server and the neighbor super nodes delete abnormal node information.
Specifically, for explaining the embodiment of the present invention, in the embodiment, it is first assumed that there are one video source server and three tracking servers corresponding to three network service providers (ISPs) in the CDN-P2P hybrid architecture, the number of CDN nodes in each ISP domain is 10, the CDN node cache size is 50, and the long transmission/download bandwidth is 20. Further, assuming that the number of common nodes in each ISP domain is 2000,
Figure GDA0003506419120000071
setting 10 super node groups in each ISP domain, wherein the maximum number of nodes in the groups is 20, the cache size of the super nodes is 4, the uploading/downloading bandwidth is 1, the total number of resources is 1000, and the coefficient rho of the super nodes is set1＝0.1，ρ2＝ρ3＝ρ40.3 as a moldSetting t to simulate the difference of different link costs1＝1，t3＝4。
Further assuming that σ ═ γ ═ 0.4 and β ═ λ ═ 0.1, in the cache replacement phase, the ultimate goal is to store more popular resources, replacing the worthless video resources in the disk. For the resources in the cache with large size and small popularity and request times, the resources need to be replaced so as to avoid the waste of capacity; when the popularity is high, the number of requests is large, but the copy resources are insufficient, more copy caching is needed.
A method for managing an optimal collaborative cache under a CDN-P2P hybrid architecture comprises the steps of firstly constructing a super node group; secondly, constructing a corresponding mathematical model based on a global optimization target according to the resource supply and demand relationship among ISPs domains, and optimizing the model by using a resource allocation greedy algorithm; finally, designing a collaborative cache management method based on the utility optimal model;
the super node cluster takes network time delay between super nodes as a distance between the two nodes, the super nodes establish the super node cluster based on a principle of proximity, the super nodes in the cluster are neighbor nodes, each super node maintains a group of neighbor super node information, a shared area between the nodes is relatively transparent, the super nodes can acquire video resources from the shared area of the neighbor nodes, and as shown in fig. 2, the specific steps of establishing the super node cluster are as follows:
1-1) starting;
1-2) calculating the Ability value Abiliity of the common node ii；
1-3) Ability to comply with Abiliity valuesiArranging the nodes in a descending order to obtain an ordered node List order Su Peer List;
1-4) put the List order Su Peer List in front
Figure GDA0003506419120000081
Each node is marked as a super node, and a mark symbol SPflag is 1, wherein Numsp represents the number of super nodes, N represents the total number of nodes, wherein
Figure GDA0003506419120000083
Represents a super node occupation ratio, and
Figure GDA0003506419120000082
1-5) Each super node k1Sending a detection packet to other super nodes to obtain a super node list orderPeerRttDist with transmission delay less than delta, and sequencing the list according to ascending time delay; delta represents the maximum acceptable time delay, and when the time delay is too large, a large amount of time is needed for periodic information exchange between nodes.
1-6) sequentially inquiring whether a super node k exists in the list orderPeerRttDistspWhether the size of the group is smaller than tau; if yes, the super node k1Adding kspThe group in which the gene belongs; otherwise, the super node k1Establishing a new group; τ represents the maximum number of nodes in a super node cluster, and if the number of super nodes in a cluster is too small, the benefit brought by the shared service is not very large, and when the number of nodes is too large, the nodes in the cluster are difficult to maintain.
1-7) finishing;
in step 1-2), the Ability value AbilityiThe evaluation index representing the service performance of the node is comprehensively determined by four factors of historical online time, bandwidth, storage capacity and exit frequency of the node. The longer the historical online time is, the longer the node service duration may be, however, when the bandwidth and capacity of the node are small, although the node is in an online state for a long time, the available bandwidth and capacity resources cannot be well contributed, the exit frequency represents the online stability of the node, the higher the exit frequency is, indicating that the node cannot provide a stable service, and the capability value of the node is defined as follows:
Abilityi＝ρ1OTime+ρ2Bandwidth+ρ3diskCapacity+ρ4exitFreq (1)
in formula (1), OTime represents the historical online time of the node, Bandwidth represents the Bandwidth of the node, diskCapacity represents the disk capacity of the node, exitFreq represents the exit frequency of the node, and ρ1、ρ2、ρ3And ρ4Coefficients representing historical online duration, bandwidth, disk capacity, and historical exit frequency attributes, respectively, and ρ1+ρ2+ρ3+ρ4＝1。
As shown in fig. 3, the greedy resource allocation algorithm sets a resource priority path, preferentially obtains resources from a super node group in a neighboring ISP domain, and then obtains resources from a CDN node, and specifically includes the following steps:
2-1) starting;
2-2) initialization Utility value Utility, t1，t2，t3，t4(t1,t2,t3,t4∈tω)；
2-3) obtaining target resource Q lacking in ISP domainv；
2-4) acquiring neighbor ISPs list information, and executing steps 2-5) -2-7) on each neighbor ISP domain;
2-5) judging whether Q exists in the super node groupvIf not, executing step 6), if yes, updating QvAnd Utility, judgment QvIf the result is empty, executing the step 2-8), otherwise, executing the step 2-6);
2-6) judging whether Q exists in CDN nodevIf not, executing the step 2-7), and if so, updating the QvAnd Utility, judgment QvIf the result is empty, executing the step 2-8) if the result is empty, and executing the step 2-7) if the result is not empty;
2-7) judging whether Q exists in neighbor nodes of the CDN nodevIf not, executing the step 2-8), and if so, updating the QvAnd Utility;
2-8) ending;
in step 2-3), resource QvFrom super node clusters and CDNs nodes, QvThe calculation formula of (2) is as follows:
Figure GDA0003506419120000091
in the above formula (3), G representsA neighbor ISP set; g _ sps represents a set of super node clusters within an ISP domain, g _ CDN represents a set of CDN nodes within an ISP domain; xv,sE {0,1}, is a binary representation of the resource in the supernode, Xv,sWhen the value is 1, the resource v exists in the super node s, and otherwise, the resource v does not exist;
Figure GDA0003506419120000092
is a resource binary representation of a neighboring super node, Yv,cIs a resource binary representation of the CDN node,
Figure GDA0003506419120000093
is a resource binary representation of a neighboring CDN node.
In the step 2-7), the calculation formula of the Utility value Utility is as follows:
Figure GDA0003506419120000094
in the above formula (4), t1Representing the link cost, t, of acquiring resources from a super node s2Represents the link cost, t, of acquiring resources from s' neighbor nodes2＝at1,t2>t1，(a＝1，2，…，n)，t3Represents the link cost, t, of acquiring resources from a CDN node4Represents the link cost, t, of acquiring resources from CDN neighbor nodes4＝bt3，t4>t3And b is 1, 2, …, n, and a and b respectively represent the hop count of the super node and the hop count between CDNs.
The cooperative cache management method comprises a cooperative cache mode of a super node in a super node group, a cooperative cache mode of the super node group and CDNs nodes and a cache replacement method;
the cooperative caching mode of the super nodes in the super node clusters is that each super node cluster is regarded as a whole, the nodes in the clusters can share resources, and the nodes in the clusters can store more video resources in limited capacity by caching different video resources, so that the diversity of the resources in the clusters is improved;
the method is characterized in that a coordinated cache mode of the super node group and the CDNs nodes is that target resources are divided into popular resources and secondary popular resources according to popularity degrees, the popular resources are stored in the super node group close to a user side preferentially, and the secondary popular resources are stored in the CDNs nodes secondly;
as shown in fig. 4, the cache replacement method specifically includes the following steps:
3-1) starting;
3-2) judging whether the current node has enough bandwidth, if not, executing the step 3-5), otherwise, judging whether the disk capacity of the node is enough, if so, directly caching the resource, otherwise, executing the step 3-3) -the step 3-4);
3-3) calculating the Energy value Energy of the video resource in the diskvSorting the video resources in a descending order according to the energy values;
3-4) replacing the video resource to be cached with the video resource with the minimum energy value;
3-5) finishing.
In step 3-3), the Energy value EnergyvThe method comprehensively considers the popularity, the scarcity, the request frequency and the video size of the video, carries out weighted summation on the four factors, and obtains the Energy value Energy of the videovThe calculation formula is as follows:
Energyv＝σ·Pv+β·Scarcityv+γ·Reqv,T+λ·Sizev (5)
in the above formula (5), PvRepresenting the popularity of the video v, ScarcityvIndicating the degree of scarcity, Size, of the video vvRepresenting the size of the video, Reqv,TRepresents the number of times that the video v is requested in a period T, and sigma, beta, gamma and lambda represent the parameters of the popularity, the Scarcity, the request frequency and the video size of the video respectively, ScarcityvThe calculation formula of (a) is as follows:
Figure GDA0003506419120000101
as described aboveIn the formula (6), NumvRepresenting the number of copies of video v and TotalNum representing the number of copies of all videos.

Claims (6)

1. A method for managing an optimal collaborative cache under a CDN-P2P hybrid architecture is characterized in that the method comprises the steps of firstly constructing a super node group; secondly, constructing a corresponding mathematical model based on the global utility value according to the resource supply and demand relationship among ISPs domains, and optimizing the model by using a resource allocation greedy algorithm; finally, designing a collaborative cache management method based on the utility optimal model;
the super node cluster takes network time delay between super nodes as a distance between the two nodes, the super nodes establish the super node cluster based on a principle of proximity, the super nodes in the cluster are neighbor nodes, each super node maintains a group of neighbor super node information, a sharing area between the nodes is relatively transparent, the super nodes can acquire video resources from the sharing area of the neighbor nodes, and the specific steps of establishing the super node cluster are as follows:
1-1) starting;
1-2) calculating the Ability value Abiliity of the common node ii；
1-3) Ability to comply with Abiliity valuesiArranging the nodes in a descending order to obtain an ordered node List order Su Peer List;
1-4) put the List order Su Peer List in front
Figure FDA0003512152550000011
Each node is marked as a super node, and a mark symbol SP flag is 1, wherein Numsp represents the number of super nodes, N represents the total number of nodes, wherein
Figure FDA0003512152550000012
Represents a super node occupation ratio, and
Figure FDA0003512152550000013
1-5) Each super node k1Sending detection packets to other super classesThe nodes obtain a super node list order Peer Rtt Dist with transmission delay less than delta, and the lists are sorted according to ascending delay; δ represents the maximum acceptable delay;
1-6) sequentially inquiring whether a super node k exists in the list order Peer Rtt DistspWhether the size of the group is smaller than tau; if yes, the super node k1Adding kspThe group in which the gene belongs; otherwise, the super node k1Establishing a new group; wherein τ represents the maximum number of nodes in the super node cluster;
1-7) finishing;
the mathematical model is established according to the supply and demand relationship among ISPs domains, and is optimized through a resource allocation greedy algorithm to ensure that the effectiveness of ISP domain request resources is optimal, wherein the mathematical model expression is as follows:
Figure FDA0003512152550000014
in the above formula (1), QvRepresenting the missing target resource, qi,v∈{0,1}，tωRepresenting a link cost;
the resource allocation greedy algorithm is to set a resource priority acquisition path, preferentially acquire resources from a super node group of a neighbor ISP domain, and then acquire resources from a CDN node, and specifically comprises the following steps:
2-1) starting;
2-2) initialization Utility value Utility, t1，t2，t3，t4(t1,t2,t3,t4∈tω)；
2-3) obtaining target resource Q lacking in ISP domainv；
2-4) acquiring neighbor ISPs list information, and executing steps 2-5) -2-7) on each neighbor ISP domain;
2-5) judging whether Q exists in the super node groupvIf not, executing step 6), if yes, updating QvAnd Utility, judgment QvIf empty, if yes, execute step 2-8), otherwiseThen, step 2-6) is performed;
2-6) judging whether Q exists in CDN nodevIf not, executing the step 2-7), and if so, updating the QvAnd Utility, judgment QvIf the result is empty, executing the step 2-8) if the result is empty, and executing the step 2-7) if the result is not empty;
2-7) judging whether Q exists in neighbor nodes of the CDN nodevIf not, executing the step 2-8), and if so, updating the QvAnd Utility;
2-8) ending;
the cooperative cache management method comprises a cooperative cache mode of a super node in a super node group, a cooperative cache mode of the super node group and CDNs nodes and a cache replacement method;
the cooperative caching mode of the super nodes in the super node clusters is that each super node cluster is regarded as a whole, the nodes in the clusters can share resources, and the nodes in the clusters can store more video resources in limited capacity by caching different video resources;
the method is characterized in that a coordinated cache mode of the super node group and the CDNs nodes is that target resources are divided into popular resources and secondary popular resources according to popularity degrees, the popular resources are stored in the super node group close to a user side preferentially, and the secondary popular resources are stored in the CDNs nodes secondly;
the cache replacement method specifically comprises the following steps:
3-1) starting;
3-2) judging whether the current node has enough bandwidth, if not, executing the step 3-5), otherwise, judging whether the disk capacity of the node is enough, if so, directly caching the resource, otherwise, executing the step 3-3) -the step 3-4);
3-3) calculating the Energy value Energy of the video resource in the diskvSorting the video resources in a descending order according to the energy values;
3-4) replacing the video resource to be cached with the video resource with the minimum energy value;
3-5) finishing.
2. According to the claimsSolving 1 the method for managing the optimal collaborative cache under the CDN-P2P hybrid architecture is characterized in that in the step 1-2), the capability value Ability isiThe evaluation index representing the service performance of the node is comprehensively determined by four factors of historical online time, bandwidth, storage capacity and exit frequency of the node, and the capacity value of the node is defined as follows:
Abilityi＝ρ1OTime+ρ2Bandwidth+ρ3diskCapacity+ρ4exitFreq (2)
in the formula (2), OTime represents the historical online time of the node, Bandwidth represents the Bandwidth of the node, diskCapacity represents the disk capacity of the node, exitFreq represents the exit frequency of the node, and rho1、ρ2、ρ3And ρ4Coefficients representing historical online duration, bandwidth, disk capacity, and historical exit frequency attributes, respectively, and ρ1+ρ2+ρ3+ρ4＝1。
3. The method of claim 1, wherein the mathematical model has an expression that satisfies the following constraints:
constraint 1:
Figure FDA0003512152550000031
ensuring that the available bandwidth of the node is not less than 0 in the sending period and bandwidthi,avaiIs the available bandwidth of node i, sjIs the length of video segment j, Γ is the maximum allowed transmission duration;
constraint 2:
Figure FDA0003512152550000032
the solutions used to represent the problem are discrete.
4. The method of claim 1, wherein in step 2-3), resource Q is used as a resource for managing the most effective cooperative cache in the CDN-P2P hybrid architecturevFrom super node clusters and CDsNs node, QvThe calculation formula of (2) is as follows:
Figure FDA0003512152550000033
in the above formula (3), G represents a neighbor ISP set; g _ sps represents a set of super node clusters within an ISP domain, g _ CDN represents a CDN node within an ISP domain, Xv,sE {0,1}, is a binary representation of the resource in the supernode, Xv,sWhen the value is 1, the resource v exists in the super node s, and otherwise, the resource v does not exist;
Figure FDA0003512152550000034
is a resource binary representation of a neighboring super node, Yv,cIs a resource binary representation of the CDN node,
Figure FDA0003512152550000035
is a resource binary representation of a neighboring CDN node.
5. The method for managing the optimal collaborative cache under the CDN-P2P hybrid architecture according to claim 1, wherein in step 2-7), a calculation formula of the Utility value Utility is:
Figure FDA0003512152550000041
in the above formula (4), t1Representing the link cost, t, of acquiring resources from a super node s2Represents the link cost, t, of acquiring resources from s' neighbor nodes2＝at1,t2＞t1，(a＝1，2，…，n)，t3Represents the link cost, t, of acquiring resources from a CDN node4Represents the link cost, t, of acquiring resources from CDN neighbor nodes4＝bt3，t4＞t3And b is 1, 2, …, n, and a and b respectively represent the number of hops between super nodes and the number of hops between CDNs.
6. The method of claim 1, wherein in step 3-3), the Energy value Energy is used in the method for managing the most effective collaborative cache in the CDN-P2P hybrid architecturevThe method comprehensively considers the popularity, the scarcity, the request frequency and the video size of the video, carries out weighted summation on the four factors, and obtains the Energy value Energy of the videovThe calculation formula is as follows:
Energyv＝σ·Pv+β·Scarcityv +γ·Reqv,T+λ·Sizev (5)
in the above formula (5), PvRepresenting the popularity of the video v, ScarcityvIndicating the degree of scarcity, Size, of the video vvRepresenting the size of the video, Reqv,TRepresents the number of times that the video v is requested in a period T, and sigma, beta, gamma and lambda represent the parameters of the popularity, the Scarcity, the request frequency and the video size of the video respectively, ScarcityvThe calculation formula of (a) is as follows:
Figure FDA0003512152550000042
num in the above formula (6)vRepresenting the number of copies of video v and TotalNum representing the number of copies of all videos.
CN202110709682.3A 2021-06-25 2021-06-25 Effectiveness optimal collaborative cache management method under CDN-P2P hybrid architecture Active CN113453038B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110709682.3A CN113453038B (en) 2021-06-25 2021-06-25 Effectiveness optimal collaborative cache management method under CDN-P2P hybrid architecture

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110709682.3A CN113453038B (en) 2021-06-25 2021-06-25 Effectiveness optimal collaborative cache management method under CDN-P2P hybrid architecture

Publications (2)

Publication Number Publication Date
CN113453038A CN113453038A (en) 2021-09-28
CN113453038B true CN113453038B (en) 2022-03-29

Family

ID=77812739

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110709682.3A Active CN113453038B (en) 2021-06-25 2021-06-25 Effectiveness optimal collaborative cache management method under CDN-P2P hybrid architecture

Country Status (1)

Country Link
CN (1) CN113453038B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114124971B (en) * 2021-10-19 2023-11-24 陕西悟空云信息技术有限公司 Content copy placement method of CDN-P2P network based on edge cache
CN114095573B (en) * 2021-10-19 2023-11-28 陕西悟空云信息技术有限公司 Content copy placement method of CDN-P2P network based on edge cache
CN113992653B (en) * 2021-10-19 2023-09-15 西安邮电大学 CDN-P2P network content downloading, pre-storing and replacing method based on edge cache
CN114124939B (en) * 2021-11-25 2024-03-26 北京奇艺世纪科技有限公司 Method and device for processing pre-cached file and file pre-caching system
CN116405563B (en) * 2023-06-08 2023-08-18 湖南快乐阳光互动娱乐传媒有限公司 Resource acquisition method and system based on point-to-point content distribution network

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102547395A (en) * 2011-12-31 2012-07-04 上海聚力传媒技术有限公司 Method and device for determining video data source of network player
CN103873282A (en) * 2012-12-14 2014-06-18 中国移动通信集团福建有限公司 Method and device for realizing P2P data traffic optimization
CN107483614A (en) * 2017-08-31 2017-12-15 京东方科技集团股份有限公司 Content scheduling method and communication network based on CDN Yu P2P networks
CN110336843A (en) * 2015-02-24 2019-10-15 深圳梨享计算有限公司 A kind of content distribution method, central node and fringe node for crowdsourcing
CN112565796A (en) * 2019-09-25 2021-03-26 北京硬核聚视科技有限公司 Video content decentralized access method and system

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8090860B2 (en) * 2007-11-05 2012-01-03 Limelight Networks, Inc. Origin request with peer fulfillment
JP2009042944A (en) * 2007-08-07 2009-02-26 Grid Solutions Inc File distribution system
CN101159580B (en) * 2007-11-29 2011-08-31 中国电信股份有限公司 Content P2P method and system in content distribution network
EP2073503A1 (en) * 2007-12-17 2009-06-24 Alcatel Lucent Method for distributing content data packages originated by users of a super peer-to-peer network
CN101667999B (en) * 2008-09-04 2012-09-05 华为技术有限公司 Method and system for transmitting peer-to-peer broadcast stream, data signature device and client
KR20100042778A (en) * 2008-10-17 2010-04-27 주식회사 아이토비 Content delivery network based on multiple p2p protocol
CN101841553B (en) * 2009-03-17 2014-03-12 日电（中国）有限公司 Method, user node and server for requesting location information of resources on network
TW201210284A (en) * 2010-08-27 2012-03-01 Ind Tech Res Inst Architecture and method for hybrid Peer To Peer/client-server data transmission
CN103475719B (en) * 2013-09-12 2016-08-31 北京科技大学 The content distribution method that in a kind of CDN-P2P UNE, cross-domain flow rate minimizes
CN107181734B (en) * 2017-04-07 2020-04-07 南京邮电大学 Streaming media cache replacement method of CDN-P2P network architecture
US10771524B1 (en) * 2019-07-31 2020-09-08 Theta Labs, Inc. Methods and systems for a decentralized data streaming and delivery network
CN111212114B (en) * 2019-12-19 2021-08-27 网宿科技股份有限公司 Method and device for downloading resource file
CN111200657B (en) * 2020-01-03 2021-08-27 网宿科技股份有限公司 Method for managing resource state information and resource downloading system
CN111372100B (en) * 2020-04-21 2023-07-14 白杨 Distributed election-based end-to-end content distribution network system and distribution method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102547395A (en) * 2011-12-31 2012-07-04 上海聚力传媒技术有限公司 Method and device for determining video data source of network player
CN103873282A (en) * 2012-12-14 2014-06-18 中国移动通信集团福建有限公司 Method and device for realizing P2P data traffic optimization
CN110336843A (en) * 2015-02-24 2019-10-15 深圳梨享计算有限公司 A kind of content distribution method, central node and fringe node for crowdsourcing
CN107483614A (en) * 2017-08-31 2017-12-15 京东方科技集团股份有限公司 Content scheduling method and communication network based on CDN Yu P2P networks
CN112565796A (en) * 2019-09-25 2021-03-26 北京硬核聚视科技有限公司 Video content decentralized access method and system

Also Published As

Publication number Publication date
CN113453038A (en) 2021-09-28

Similar Documents

Publication Publication Date Title
CN113453038B (en) Effectiveness optimal collaborative cache management method under CDN-P2P hybrid architecture
JP5390413B2 (en) Hierarchically clustered P2P streaming system
Jiang et al. Orchestrating massively distributed CDNs
US20080040420A1 (en) Content distribution network
Shen et al. A DHT-aided chunk-driven overlay for scalable and efficient peer-to-peer live streaming
CN102883190B (en) Optimal bandwidth distribution demand method and device
Hefeeda et al. On-demand media streaming over the internet
CN102158767B (en) Scalable-coding-based peer to peer live media streaming system
Kim et al. Efficient peer-to-peer overlay networks for mobile IPTV services
CN101888403A (en) The method and system of storage and distributing electronic content
Markakis et al. A p2p home-box overlay for efficient content distribution
Zhang et al. A distributed multichannel demand-adaptive P2P VoD system with optimized caching and neighbor-selection
Yang et al. A P2P network framework for interactive streaming media
CN103179191B (en) P2P network control device and P2P network managing and control system
Dubin et al. Hybrid clustered peer-assisted DASH-SVC system
Jiang et al. A replica placement algorithm for hybrid CDN-P2P architecture
CN110139126B (en) Mobile video system resource sharing method based on user interaction behavior perception
Muscat et al. A Hybrid CDN-P2P Architecture for Live Video Streaming
Lee et al. A vEB-tree-based architecture for interactive video on demand services in peer-to-peer networks
Ma et al. A cloud‐assisted publish/subscribe service for time‐critical dissemination of bulk content
Hisada et al. P2P live streaming system suitable for private contents distribution
Jia et al. Modelling of P2P-based video sharing performance for Content-oriented community-based VoD Systems in Wireless Mobile Networks
Jia et al. Modeling and optimization of bandwidth supply performance for cloud-assisted video systems under flash crowd
Zhang et al. Geo-edge: Geographical resource allocation on edge caches for video-on-demand streaming
Tu et al. An efficient data scheduling scheme for P2P storage-constrained IPTV system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant